2023-09-23T04:16:59.201 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-23T04:16:59.201 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504 --name vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} -- /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/orig.config.yaml 2023-09-23T04:16:59.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T04:16:59.347 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504 branch: wip-vshankar-testing-20230922.144532 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7401504' kernel: client: branch: testing flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20230922.144532 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230922.144532 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8196 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a sleep_before_teardown: 0 subset: 57/1024 suite: fs suite_branch: wip-vshankar-testing-20230922.144532 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-23_02:50:30 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-23T04:16:59.347 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa; will attempt to use it 2023-09-23T04:16:59.348 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks 2023-09-23T04:16:59.348 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-23T04:16:59.350 INFO:teuthology.task.internal:Checking packages... 2023-09-23T04:16:59.372 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '862663913a74a99e1c52d50e6a7e59dd6705749a' 2023-09-23T04:16:59.372 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-23T04:16:59.372 INFO:teuthology.packaging:ref: None 2023-09-23T04:16:59.372 INFO:teuthology.packaging:tag: None 2023-09-23T04:16:59.372 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230922.144532 2023-09-23T04:16:59.372 INFO:teuthology.packaging:sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:16:59.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20230922.144532 2023-09-23T04:16:59.548 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6336.g86266391 2023-09-23T04:16:59.548 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-23T04:16:59.555 INFO:teuthology.task.internal:no buildpackages task found 2023-09-23T04:16:59.555 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-23T04:16:59.621 INFO:teuthology.task.internal:Saving configuration 2023-09-23T04:16:59.641 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-23T04:16:59.646 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T04:16:59.671 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.825070', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T04:16:59.691 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.822168', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T04:16:59.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.823808', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T04:16:59.711 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-23T04:16:59.717 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-23T04:16:59.717 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-23T04:16:59.717 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-23T04:16:59.717 INFO:teuthology.run_tasks:Running task console_log... 2023-09-23T04:16:59.867 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83ef28fb80>, signals=[15]) 2023-09-23T04:16:59.868 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-23T04:16:59.874 INFO:teuthology.task.internal:Opening connections... 2023-09-23T04:16:59.874 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:16:59.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:16:59.943 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2023-09-23T04:16:59.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:17:00.010 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2023-09-23T04:17:00.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:17:00.082 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-23T04:17:00.088 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2023-09-23T04:17:00.110 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2023-09-23T04:17:00.110 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2023-09-23T04:17:00.170 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T04:17:00.170 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:17:00.171 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T04:17:00.172 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T04:17:00.172 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T04:17:00.172 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T04:17:00.172 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2023-09-23T04:17:00.199 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2023-09-23T04:17:00.219 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2023-09-23T04:17:00.219 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:ID="rhel" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="fedora" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8.6" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8" 2023-09-23T04:17:00.278 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T04:17:00.279 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T04:17:00.279 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-09-23T04:17:00.299 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2023-09-23T04:17:00.319 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2023-09-23T04:17:00.319 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-09-23T04:17:00.382 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Red Hat Enterprise Linux" 2023-09-23T04:17:00.382 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8.6 (Ootpa)" 2023-09-23T04:17:00.382 INFO:teuthology.orchestra.run.smithi118.stdout:ID="rhel" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="fedora" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8.6" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.redhat.com/" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-23T04:17:00.383 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-23T04:17:00.384 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-23T04:17:00.384 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-23T04:17:00.384 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-09-23T04:17:00.407 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-23T04:17:00.417 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-23T04:17:00.454 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-23T04:17:00.455 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2023-09-23T04:17:00.457 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2023-09-23T04:17:00.460 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2023-09-23T04:17:00.479 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-23T04:17:00.485 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-23T04:17:00.485 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2023-09-23T04:17:00.517 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2023-09-23T04:17:00.519 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2023-09-23T04:17:00.545 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-23T04:17:00.622 INFO:teuthology.run_tasks:Running task kernel... 2023-09-23T04:17:00.637 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-09-23T04:17:00.637 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-09-23T04:17:00.637 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-09-23T04:17:00.637 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T04:17:00.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:17:00.657 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-09-23T04:17:00.898 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T04:17:00.898 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-23T04:17:00.899 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-09-23T04:17:00.918 INFO:teuthology.orchestra.run.smithi005.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-23T04:17:00.919 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T04:17:00.919 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-23T04:17:00.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2023-09-23T04:17:02.254 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-09-23T04:17:02.254 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2023-09-23T04:17:03.719 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-09-23T04:17:04.444 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-09-23T04:17:08.145 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-09-23T04:17:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-09-23T04:17:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2023-09-23T04:17:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-09-23T04:17:09.018 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-23T04:17:09.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T04:17:09.238 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-09-23T04:17:09.238 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-09-23T04:17:09.241 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2023-09-23T04:17:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23 04:17:22 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-23T04:17:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2023-09-23 04:17:22-- 2023-09-23T04:17:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 13s 2023-09-23T04:17:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 363M in 13s (28.6 MB/s) 2023-09-23T04:17:22.271 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-23T04:17:22.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-23T04:17:23.923 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2023-09-23T04:17:23.979 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2023-09-23T04:17:24.066 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2023-09-23T04:18:04.613 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-23T04:18:34.314 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-23T04:18:34.314 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-23T04:18:34.373 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.373 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.373 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.373 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.373 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-23T04:18:34.374 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-23T04:18:34.375 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-23T04:18:34.376 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-23T04:18:34.377 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-23T04:18:34.378 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-23T04:18:34.379 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-23T04:18:34.380 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-23T04:18:34.381 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-23T04:18:34.382 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-23T04:18:34.383 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-23T04:18:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-23T04:18:34.385 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-23T04:18:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-23T04:18:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-23T04:18:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-23T04:18:34.389 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-23T04:18:34.390 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-23T04:18:34.391 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-23T04:18:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-23T04:18:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-23T04:18:34.394 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-23T04:18:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-23T04:18:34.396 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-23T04:18:34.397 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-23T04:18:34.398 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-23T04:18:34.399 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-23T04:18:34.400 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-23T04:18:34.401 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-23T04:18:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-23T04:18:34.403 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-23T04:18:34.404 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-23T04:18:34.405 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-23T04:18:34.406 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-23T04:18:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-23T04:18:34.408 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-23T04:18:34.409 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-23T04:18:34.410 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-23T04:18:34.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-23T04:18:34.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-23T04:18:34.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-23T04:18:34.414 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-23T04:18:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-23T04:18:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-23T04:18:34.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-23T04:18:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-23T04:18:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-23T04:18:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-23T04:18:34.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-23T04:18:34.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-23T04:18:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-23T04:18:34.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-23T04:18:34.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-23T04:18:34.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-23T04:18:34.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-23T04:18:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-23T04:18:34.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-23T04:18:34.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-23T04:18:34.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-23T04:18:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-23T04:18:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-23T04:18:34.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-23T04:18:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-23T04:18:34.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-23T04:18:34.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-23T04:18:34.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-23T04:18:34.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-23T04:18:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-23T04:18:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-23T04:18:34.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-23T04:18:34.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-23T04:18:34.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-23T04:18:34.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-23T04:18:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-23T04:18:34.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-23T04:18:34.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-23T04:18:34.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-23T04:18:34.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-23T04:18:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-23T04:18:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-23T04:18:34.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-23T04:18:34.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-23T04:18:34.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-23T04:18:34.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-23T04:18:34.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-23T04:18:34.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-23T04:18:34.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-23T04:18:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-23T04:18:34.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-23T04:18:34.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-23T04:18:34.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-23T04:18:34.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-23T04:18:34.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-23T04:18:34.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-23T04:18:34.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-23T04:18:34.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-23T04:18:34.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-23T04:18:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-23T04:18:34.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-23T04:18:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-23T04:18:34.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-23T04:18:34.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-23T04:18:34.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-23T04:18:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-23T04:18:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-23T04:18:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-23T04:18:34.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-23T04:18:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-23T04:18:34.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-23T04:18:34.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-23T04:18:34.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-23T04:18:34.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-23T04:18:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-23T04:18:34.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-23T04:18:34.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-23T04:18:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-23T04:18:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-23T04:18:34.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-23T04:18:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-23T04:18:34.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-23T04:18:34.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-23T04:18:34.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-23T04:18:34.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-23T04:18:34.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-23T04:18:34.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-23T04:18:34.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-23T04:18:34.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-23T04:18:34.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-23T04:18:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-23T04:18:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-23T04:18:34.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-23T04:18:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-23T04:18:34.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-23T04:18:34.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-23T04:18:34.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-23T04:18:34.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-23T04:18:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-23T04:18:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-23T04:18:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-23T04:18:34.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-23T04:18:34.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-23T04:18:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-23T04:18:34.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-23T04:18:34.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-23T04:18:34.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-23T04:18:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-23T04:18:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-23T04:18:34.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-23T04:18:34.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-23T04:18:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-23T04:18:34.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-23T04:18:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-23T04:18:34.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-23T04:18:34.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-23T04:18:34.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-23T04:18:34.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-23T04:18:34.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-23T04:18:34.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-23T04:18:34.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-23T04:18:34.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-23T04:18:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-23T04:18:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-23T04:18:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-23T04:18:34.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-23T04:18:34.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-23T04:18:34.537 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.537 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-23T04:18:34.565 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-23T04:18:34.566 INFO:teuthology.orchestra.run.smithi005.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.567 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-09-23T04:18:34.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2023-09-23T04:18:34.647 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.02 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 123.el8 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Group : System Environment/Base 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 9265450 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-09-23T04:18:34.648 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Relocations : (not relocatable) 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : Red Hat, Inc. 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : Red Hat, Inc. 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-23T04:18:34.649 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2023-09-23T04:18:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:18:34.650 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2023-09-23T04:18:34.650 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-23T04:18:34.651 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:34.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-23T04:18:35.818 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2023-09-23T04:18:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:done 2023-09-23T04:18:37.957 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-09-23T04:18:37.979 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.t3tjHN8iBn 2023-09-23T04:18:37.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.t3tjHN8iBn 2023-09-23T04:18:38.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.t3tjHN8iBn 2023-09-23T04:18:38.199 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.t3tjHN8iBn is 6KB 2023-09-23T04:18:38.247 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.t3tjHN8iBn 2023-09-23T04:18:38.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2023-09-23T04:18:38.334 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-09-23T04:18:38.334 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-09-23T04:18:38.334 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-09-23T04:18:38.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-09-23T04:18:38.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2023-09-23T04:19:08.497 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-23T04:19:08.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-09-23T04:19:08.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:20:08.500 DEBUG:teuthology.orchestra.remote:timed out 2023-09-23T04:20:17.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-09-23T04:20:17.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:20:18.864 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-09-23T04:20:19.332 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-09-23T04:20:19.332 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-23T04:20:19.332 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-09-23T04:20:19.373 INFO:teuthology.orchestra.run.smithi005.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-23T04:20:19.374 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-23T04:20:19.374 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-23T04:20:19.374 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-09-23T04:20:20.375 DEBUG:teuthology.parallel:result is None 2023-09-23T04:20:20.376 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-23T04:20:20.383 INFO:teuthology.task.internal:Creating test directory... 2023-09-23T04:20:20.383 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T04:20:20.389 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T04:20:20.392 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T04:20:20.428 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-23T04:20:20.435 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-23T04:20:20.485 INFO:teuthology.task.internal:Creating archive directory... 2023-09-23T04:20:20.485 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T04:20:20.489 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T04:20:20.492 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T04:20:20.618 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-23T04:20:20.624 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-23T04:20:20.625 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T04:20:20.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:20:20.643 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T04:20:20.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:20:20.662 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T04:20:20.667 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T04:20:20.686 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T04:20:20.752 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.766 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.775 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.782 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.843 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.888 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T04:20:20.892 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-23T04:20:20.899 INFO:teuthology.task.internal:Configuring sudo... 2023-09-23T04:20:20.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T04:20:20.905 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T04:20:20.908 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T04:20:20.972 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-23T04:20:20.988 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-23T04:20:20.988 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T04:20:20.992 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T04:20:20.994 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T04:20:21.031 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.250 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:21.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:21.392 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:20:21.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T04:20:21.472 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.577 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:21.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:21.707 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:20:21.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T04:20:21.776 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.821 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T04:20:21.906 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:21.953 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T04:20:22.000 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:20:22.000 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T04:20:22.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2023-09-23T04:20:22.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2023-09-23T04:20:22.092 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2023-09-23T04:20:22.158 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T04:20:22.206 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T04:20:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T04:20:22.611 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-23T04:20:22.619 INFO:teuthology.task.internal:Starting timer... 2023-09-23T04:20:22.619 INFO:teuthology.run_tasks:Running task pcp... 2023-09-23T04:20:22.680 INFO:teuthology.run_tasks:Running task selinux... 2023-09-23T04:20:22.728 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-23T04:20:22.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2023-09-23T04:20:22.846 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2023-09-23T04:20:22.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2023-09-23T04:20:22.907 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2023-09-23T04:20:22.909 DEBUG:teuthology.orchestra.run.smithi118:> sudo service auditd rotate 2023-09-23T04:20:22.987 INFO:teuthology.orchestra.run.smithi118.stdout:Rotating logs: 2023-09-23T04:20:22.989 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-23T04:20:22.989 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2023-09-23T04:20:23.040 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2023-09-23T04:20:23.040 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2023-09-23T04:20:23.065 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2023-09-23T04:20:23.066 DEBUG:teuthology.orchestra.run.smithi118:> /usr/sbin/getenforce 2023-09-23T04:20:23.099 INFO:teuthology.orchestra.run.smithi118.stdout:Permissive 2023-09-23T04:20:23.099 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi118.front.sepia.ceph.com': 'permissive'} 2023-09-23T04:20:23.099 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T04:20:23.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:20:23.176 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T04:20:23.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:20:23.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T04:20:23.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T04:20:23.233 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-23T04:20:23.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2023-09-23T04:20:23.303 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2023-09-23T04:20:23.329 DEBUG:teuthology.orchestra.run.smithi118:> sudo /usr/sbin/setenforce permissive 2023-09-23T04:20:23.357 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-23T04:20:23.367 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-23T04:20:23.372 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-23T04:20:23.526 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-23T04:20:23.545 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-23T04:20:23.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi069.front.sepia.ceph.com,smithi118.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-23T04:32:32.135 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi118.front.sepia.ceph.com')] 2023-09-23T04:32:32.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-09-23T04:32:32.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:32:32.226 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-09-23T04:32:32.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-09-23T04:32:32.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-23T04:32:32.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:32:32.418 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-09-23T04:32:32.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-09-23T04:32:32.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-23T04:32:32.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T04:32:32.557 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-09-23T04:32:32.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-23T04:32:32.625 INFO:teuthology.run_tasks:Running task clock... 2023-09-23T04:32:32.635 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-23T04:32:32.635 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T04:32:32.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T04:32:32.640 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T04:32:32.640 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T04:32:32.643 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T04:32:32.643 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T04:32:32.675 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T04:32:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T04:32:32.698 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T04:32:32.713 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T04:32:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T04:32:32.752 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T04:32:32.754 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2023-09-23T04:32:32.766 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2023-09-23T04:32:32.781 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T04:32:32.785 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: ntpd: command not found 2023-09-23T04:32:32.795 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T04:32:32.797 INFO:teuthology.orchestra.run.smithi118.stdout:506 Cannot talk to daemon 2023-09-23T04:32:32.812 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T04:32:32.826 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T04:32:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2023-09-23T04:32:32.874 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-09-23T04:32:32.876 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.878 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.899 INFO:teuthology.orchestra.run.smithi118.stderr:bash: ntpq: command not found 2023-09-23T04:32:32.902 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T04:32:32.902 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2023-09-23T04:32:32.902 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.902 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.903 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.903 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:32.911 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T04:32:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T04:32:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-09-23T04:32:33.047 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T04:32:33.047 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-09-23T04:32:33.047 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:33.048 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:33.048 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:33.048 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T04:32:33.049 INFO:teuthology.run_tasks:Running task pexec... 2023-09-23T04:32:33.058 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-23T04:32:33.058 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:32:33.058 DEBUG:teuthology.orchestra.run.smithi005:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T04:32:33.059 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi069.front.sepia.ceph.com 2023-09-23T04:32:33.059 DEBUG:teuthology.orchestra.run.smithi069:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T04:32:33.059 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi118.front.sepia.ceph.com 2023-09-23T04:32:33.059 DEBUG:teuthology.orchestra.run.smithi118:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-23T04:32:33.428 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:33.430 INFO:teuthology.orchestra.run.smithi118.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:33.574 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:34.261 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:04:54 ago on Sat 23 Sep 2023 04:27:39 AM UTC. 2023-09-23T04:32:34.386 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:04:48 ago on Sat 23 Sep 2023 04:27:45 AM UTC. 2023-09-23T04:32:34.696 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:46 ago on Sat 23 Sep 2023 04:30:48 AM UTC. 2023-09-23T04:32:35.988 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout:Resetting modules: 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:35.989 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-09-23T04:32:35.990 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2023-09-23T04:32:35.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:36.261 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2023-09-23T04:32:36.262 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-09-23T04:32:36.262 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2023-09-23T04:32:36.262 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-09-23T04:32:36.262 INFO:teuthology.orchestra.run.smithi118.stdout:Resetting modules: 2023-09-23T04:32:36.262 INFO:teuthology.orchestra.run.smithi118.stdout: container-tools 2023-09-23T04:32:36.263 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:36.263 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2023-09-23T04:32:36.263 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2023-09-23T04:32:36.263 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:36.764 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout:Resetting modules: 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-09-23T04:32:36.766 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-09-23T04:32:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:36.923 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:04:57 ago on Sat 23 Sep 2023 04:27:39 AM UTC. 2023-09-23T04:32:37.206 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:04:51 ago on Sat 23 Sep 2023 04:27:45 AM UTC. 2023-09-23T04:32:38.060 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:49 ago on Sat 23 Sep 2023 04:30:48 AM UTC. 2023-09-23T04:32:39.286 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-09-23T04:32:39.627 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2023-09-23T04:32:40.034 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:40.107 INFO:teuthology.orchestra.run.smithi118.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:40.574 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-09-23T04:32:40.857 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:05:01 ago on Sat 23 Sep 2023 04:27:39 AM UTC. 2023-09-23T04:32:40.924 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:04:55 ago on Sat 23 Sep 2023 04:27:45 AM UTC. 2023-09-23T04:32:41.208 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:42.211 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:53 ago on Sat 23 Sep 2023 04:30:48 AM UTC. 2023-09-23T04:32:42.575 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-09-23T04:32:42.621 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2023-09-23T04:32:42.667 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-09-23T04:32:42.667 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout:Installing group/module packages: 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T04:32:42.668 INFO:teuthology.orchestra.run.smithi069.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout:Installing module profiles: 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools/common 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling module streams: 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout: container-tools rhel8 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout:============================================================================================================ 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout:Install 10 Packages 2023-09-23T04:32:42.669 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:42.670 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 18 M 2023-09-23T04:32:42.670 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 65 M 2023-09-23T04:32:42.670 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-09-23T04:32:42.715 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================================ 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================================ 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout:Installing group/module packages: 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T04:32:42.716 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:Installing module profiles: 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: container-tools/common 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling module streams: 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: container-tools rhel8 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:============================================================================================================ 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout:Install 10 Packages 2023-09-23T04:32:42.717 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:42.718 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 18 M 2023-09-23T04:32:42.718 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 65 M 2023-09-23T04:32:42.718 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2023-09-23T04:32:42.772 INFO:teuthology.orchestra.run.smithi069.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 242 kB/s | 25 kB 00:00 2023-09-23T04:32:42.789 INFO:teuthology.orchestra.run.smithi069.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 341 kB/s | 41 kB 00:00 2023-09-23T04:32:42.814 INFO:teuthology.orchestra.run.smithi069.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 649 kB/s | 94 kB 00:00 2023-09-23T04:32:42.822 INFO:teuthology.orchestra.run.smithi118.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 905 kB/s | 94 kB 00:00 2023-09-23T04:32:42.831 INFO:teuthology.orchestra.run.smithi069.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 841 kB/s | 49 kB 00:00 2023-09-23T04:32:42.847 INFO:teuthology.orchestra.run.smithi118.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 193 kB/s | 25 kB 00:00 2023-09-23T04:32:42.865 INFO:teuthology.orchestra.run.smithi069.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-09-23T04:32:42.872 INFO:teuthology.orchestra.run.smithi118.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 265 kB/s | 41 kB 00:00 2023-09-23T04:32:42.890 INFO:teuthology.orchestra.run.smithi069.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-23T04:32:42.897 INFO:teuthology.orchestra.run.smithi118.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-09-23T04:32:42.907 INFO:teuthology.orchestra.run.smithi069.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-09-23T04:32:42.931 INFO:teuthology.orchestra.run.smithi118.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-23T04:32:42.987 INFO:teuthology.orchestra.run.smithi118.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.6 MB/s | 149 kB 00:00 2023-09-23T04:32:43.015 INFO:teuthology.orchestra.run.smithi118.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.5 MB/s | 213 kB 00:00 2023-09-23T04:32:43.024 INFO:teuthology.orchestra.run.smithi069.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-09-23T04:32:43.124 INFO:teuthology.orchestra.run.smithi069.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-09-23T04:32:43.132 INFO:teuthology.orchestra.run.smithi118.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-09-23T04:32:43.216 INFO:teuthology.orchestra.run.smithi069.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 22 MB/s | 7.0 MB 00:00 2023-09-23T04:32:43.217 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:32:43.217 INFO:teuthology.orchestra.run.smithi069.stdout:Total 33 MB/s | 18 MB 00:00 2023-09-23T04:32:43.240 INFO:teuthology.orchestra.run.smithi118.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-09-23T04:32:43.324 INFO:teuthology.orchestra.run.smithi118.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-23T04:32:43.324 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:32:43.324 INFO:teuthology.orchestra.run.smithi118.stdout:Total 30 MB/s | 18 MB 00:00 2023-09-23T04:32:43.416 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-09-23T04:32:43.435 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-09-23T04:32:43.435 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-09-23T04:32:43.525 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2023-09-23T04:32:43.543 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2023-09-23T04:32:43.543 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2023-09-23T04:32:43.617 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-09-23T04:32:43.620 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-09-23T04:32:43.727 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2023-09-23T04:32:43.730 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2023-09-23T04:32:44.108 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-09-23T04:32:44.295 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T04:32:44.296 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2023-09-23T04:32:44.302 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-09-23T04:32:44.409 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout:Installing group/module packages: 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-23T04:32:44.410 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Installing module profiles: 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools/common 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling module streams: 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools rhel8 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Install 10 Packages 2023-09-23T04:32:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 18 M 2023-09-23T04:32:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 65 M 2023-09-23T04:32:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-09-23T04:32:44.466 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T04:32:44.467 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:44.488 INFO:teuthology.orchestra.run.smithi005.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 541 kB/s | 41 kB 00:00 2023-09-23T04:32:44.514 INFO:teuthology.orchestra.run.smithi005.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 247 kB/s | 25 kB 00:00 2023-09-23T04:32:44.539 INFO:teuthology.orchestra.run.smithi005.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 987 kB/s | 49 kB 00:00 2023-09-23T04:32:44.564 INFO:teuthology.orchestra.run.smithi005.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 620 kB/s | 94 kB 00:00 2023-09-23T04:32:44.606 INFO:teuthology.orchestra.run.smithi005.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-09-23T04:32:44.640 INFO:teuthology.orchestra.run.smithi005.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-09-23T04:32:44.653 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:44.653 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T04:32:44.674 INFO:teuthology.orchestra.run.smithi005.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-09-23T04:32:44.799 INFO:teuthology.orchestra.run.smithi005.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-09-23T04:32:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T04:32:44.854 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T04:32:44.982 INFO:teuthology.orchestra.run.smithi005.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-09-23T04:32:45.074 INFO:teuthology.orchestra.run.smithi005.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-09-23T04:32:45.075 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:32:45.075 INFO:teuthology.orchestra.run.smithi005.stdout:Total 27 MB/s | 18 MB 00:00 2023-09-23T04:32:45.079 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T04:32:45.198 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T04:32:45.417 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-09-23T04:32:45.421 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T04:32:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-09-23T04:32:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-09-23T04:32:45.931 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T04:32:45.956 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-09-23T04:32:45.961 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-09-23T04:32:46.216 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T04:32:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-09-23T04:32:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-23T04:32:46.814 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T04:32:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:47.040 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T04:32:47.082 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T04:32:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-23T04:32:47.241 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T04:32:47.274 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T04:32:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-23T04:32:47.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:47.433 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T04:32:47.619 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T04:32:47.685 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T04:32:47.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T04:32:47.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T04:32:47.686 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T04:32:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-23T04:32:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T04:32:47.912 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T04:32:47.913 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T04:32:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-23T04:32:48.913 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T04:32:48.913 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:05:09 ago on Sat 23 Sep 2023 04:27:39 AM UTC. 2023-09-23T04:32:49.094 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T04:32:49.094 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:05:03 ago on Sat 23 Sep 2023 04:27:45 AM UTC. 2023-09-23T04:32:49.559 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-23T04:32:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-23T04:32:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-23T04:32:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-23T04:32:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-23T04:32:51.897 INFO:teuthology.orchestra.run.smithi069.stdout:Installed products updated. 2023-09-23T04:32:52.057 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-23T04:32:52.057 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:02:03 ago on Sat 23 Sep 2023 04:30:48 AM UTC. 2023-09-23T04:32:52.102 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:52.102 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2023-09-23T04:32:52.102 INFO:teuthology.orchestra.run.smithi069.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.102 INFO:teuthology.orchestra.run.smithi069.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:32:52.103 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-09-23T04:32:52.213 INFO:teuthology.orchestra.run.smithi118.stdout:Installed products updated. 2023-09-23T04:32:52.250 DEBUG:teuthology.parallel:result is None 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T04:32:52.434 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:32:52.435 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2023-09-23T04:32:52.574 DEBUG:teuthology.parallel:result is None 2023-09-23T04:32:54.879 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2023-09-23T04:32:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:55.097 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-09-23T04:32:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:32:55.098 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-09-23T04:32:55.321 DEBUG:teuthology.parallel:result is None 2023-09-23T04:32:55.322 INFO:teuthology.run_tasks:Running task install... 2023-09-23T04:32:55.333 DEBUG:teuthology.task.install:project ceph 2023-09-23T04:32:55.333 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}} 2023-09-23T04:32:55.333 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'} 2023-09-23T04:32:55.334 INFO:teuthology.task.install:Using flavor: default 2023-09-23T04:32:55.339 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-23T04:32:55.339 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-23T04:32:55.340 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T04:32:55.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:32:55.342 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T04:32:55.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:32:55.343 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T04:32:55.343 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:32:55.504 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T04:32:55.504 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T04:32:55.524 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T04:32:55.525 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T04:32:55.530 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/ 2023-09-23T04:32:55.530 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T04:32:55.619 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T04:32:55.620 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:32:55.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T04:32:55.633 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T04:32:55.633 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:32:55.633 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T04:32:55.660 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T04:32:55.660 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T04:32:55.689 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T04:32:55.689 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T04:32:55.727 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T04:32:55.781 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T04:32:55.781 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:32:55.781 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T04:32:55.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T04:32:55.803 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T04:32:55.808 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T04:32:55.808 DEBUG:teuthology.orchestra.run.smithi118:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T04:32:55.870 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2023-09-23T04:32:55.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2023-09-23T04:32:55.874 DEBUG:teuthology.orchestra.run.smithi118:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T04:32:55.896 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T04:32:55.951 DEBUG:teuthology.orchestra.run.smithi118:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T04:32:55.964 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2023-09-23T04:32:55.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2023-09-23T04:32:56.017 INFO:teuthology.orchestra.run.smithi118.stdout:check_obsoletes = 1 2023-09-23T04:32:56.019 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum clean all 2023-09-23T04:32:56.262 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:56.424 INFO:teuthology.orchestra.run.smithi118.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:56.446 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:56.557 INFO:teuthology.orchestra.run.smithi069.stdout:44 files removed 2023-09-23T04:32:56.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T04:32:56.740 INFO:teuthology.orchestra.run.smithi118.stdout:44 files removed 2023-09-23T04:32:56.794 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T04:32:56.821 INFO:teuthology.orchestra.run.smithi005.stdout:44 files removed 2023-09-23T04:32:56.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T04:32:56.957 INFO:teuthology.orchestra.run.smithi069.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:57.141 INFO:teuthology.orchestra.run.smithi118.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:57.362 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-09-23T04:32:57.634 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 323 kB/s | 81 kB 00:00 2023-09-23T04:32:58.043 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-09-23T04:32:58.124 INFO:teuthology.orchestra.run.smithi118.stdout:ceph packages for x86_64 160 kB/s | 81 kB 00:00 2023-09-23T04:32:58.311 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 6.1 kB/s | 1.5 kB 00:00 2023-09-23T04:32:58.587 INFO:teuthology.orchestra.run.smithi069.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-09-23T04:32:58.667 INFO:teuthology.orchestra.run.smithi118.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-09-23T04:32:58.728 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 101 kB/s | 81 kB 00:00 2023-09-23T04:32:59.120 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-09-23T04:32:59.185 INFO:teuthology.orchestra.run.smithi118.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2023-09-23T04:32:59.444 INFO:teuthology.orchestra.run.smithi118.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-09-23T04:32:59.512 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-09-23T04:32:59.789 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-09-23T04:33:00.628 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 16 MB 00:02 2023-09-23T04:33:00.976 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-09-23T04:33:01.880 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2023-09-23T04:33:04.263 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 601 kB/s | 24 kB 00:00 2023-09-23T04:33:04.618 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-09-23T04:33:05.198 INFO:teuthology.orchestra.run.smithi069.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-09-23T04:33:05.529 INFO:teuthology.orchestra.run.smithi118.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-09-23T04:33:06.222 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 336 kB/s | 24 kB 00:00 2023-09-23T04:33:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-09-23T04:33:13.420 INFO:teuthology.orchestra.run.smithi118.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-09-23T04:33:13.482 INFO:teuthology.orchestra.run.smithi069.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-09-23T04:33:16.708 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 47 MB 00:00 2023-09-23T04:33:24.498 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:24.499 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:24.508 INFO:teuthology.orchestra.run.smithi118.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:24.509 INFO:teuthology.orchestra.run.smithi118.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:24.513 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.514 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.515 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T04:33:24.517 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T04:33:24.519 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T04:33:24.522 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.523 INFO:teuthology.orchestra.run.smithi118.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.523 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T04:33:24.524 INFO:teuthology.orchestra.run.smithi118.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.525 INFO:teuthology.orchestra.run.smithi118.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T04:33:24.526 INFO:teuthology.orchestra.run.smithi118.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T04:33:24.528 INFO:teuthology.orchestra.run.smithi118.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T04:33:24.531 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T04:33:24.533 INFO:teuthology.orchestra.run.smithi118.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T04:33:24.653 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2023-09-23T04:33:24.665 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-09-23T04:33:24.665 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T04:33:24.666 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T04:33:24.667 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T04:33:24.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2023-09-23T04:33:24.669 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T04:33:24.670 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T04:33:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T04:33:24.672 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T04:33:24.673 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T04:33:24.674 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T04:33:24.675 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T04:33:24.676 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling module streams: 2023-09-23T04:33:24.677 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql 10 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================================================== 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout:Install 128 Packages 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 3 Packages 2023-09-23T04:33:24.678 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 209 M 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================================================== 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2023-09-23T04:33:24.679 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================================================== 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T04:33:24.680 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T04:33:24.681 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T04:33:24.682 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T04:33:24.683 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T04:33:24.684 INFO:teuthology.orchestra.run.smithi118.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T04:33:24.685 INFO:teuthology.orchestra.run.smithi118.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T04:33:24.686 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T04:33:24.687 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T04:33:24.688 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T04:33:24.689 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T04:33:24.690 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling module streams: 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql 10 2023-09-23T04:33:24.691 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================================================== 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:Install 128 Packages 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 3 Packages 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 209 M 2023-09-23T04:33:24.692 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2023-09-23T04:33:24.971 INFO:teuthology.orchestra.run.smithi118.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2023-09-23T04:33:25.238 INFO:teuthology.orchestra.run.smithi118.stdout:(2/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 3.3 MB/s | 912 kB 00:00 2023-09-23T04:33:25.322 INFO:teuthology.orchestra.run.smithi118.stdout:(3/131): ceph-immutable-object-cache-18.0.0-633 1.9 MB/s | 167 kB 00:00 2023-09-23T04:33:25.483 INFO:teuthology.orchestra.run.smithi069.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-09-23T04:33:25.930 INFO:teuthology.orchestra.run.smithi118.stdout:(4/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 3.8 MB/s | 2.3 MB 00:00 2023-09-23T04:33:26.222 INFO:teuthology.orchestra.run.smithi118.stdout:(5/131): ceph-common-18.0.0-6336.g86266391.el8. 16 MB/s | 23 MB 00:01 2023-09-23T04:33:26.264 INFO:teuthology.orchestra.run.smithi118.stdout:(6/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 4.9 MB/s | 1.6 MB 00:00 2023-09-23T04:33:26.565 INFO:teuthology.orchestra.run.smithi118.stdout:(7/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 14 MB/s | 4.6 MB 00:00 2023-09-23T04:33:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:(2/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 521 kB/s | 912 kB 00:01 2023-09-23T04:33:27.523 INFO:teuthology.orchestra.run.smithi118.stdout:(8/131): ceph-radosgw-18.0.0-6336.g86266391.el8 15 MB/s | 14 MB 00:00 2023-09-23T04:33:27.566 INFO:teuthology.orchestra.run.smithi069.stdout:(3/131): ceph-immutable-object-cache-18.0.0-633 501 kB/s | 167 kB 00:00 2023-09-23T04:33:27.598 INFO:teuthology.orchestra.run.smithi118.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 325 kB/s | 24 kB 00:00 2023-09-23T04:33:28.440 INFO:teuthology.orchestra.run.smithi118.stdout:(10/131): ceph-osd-18.0.0-6336.g86266391.el8.x8 8.2 MB/s | 18 MB 00:02 2023-09-23T04:33:28.490 INFO:teuthology.orchestra.run.smithi118.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 5.1 MB/s | 260 kB 00:00 2023-09-23T04:33:28.557 INFO:teuthology.orchestra.run.smithi118.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 467 kB/s | 31 kB 00:00 2023-09-23T04:33:28.566 INFO:teuthology.orchestra.run.smithi069.stdout:(4/131): ceph-base-18.0.0-6336.g86266391.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-09-23T04:33:28.724 INFO:teuthology.orchestra.run.smithi118.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 4.9 MB/s | 832 kB 00:00 2023-09-23T04:33:28.833 INFO:teuthology.orchestra.run.smithi118.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 1.7 MB/s | 184 kB 00:00 2023-09-23T04:33:28.908 INFO:teuthology.orchestra.run.smithi118.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 1.8 MB/s | 139 kB 00:00 2023-09-23T04:33:29.000 INFO:teuthology.orchestra.run.smithi118.stdout:(16/131): libradosstriper1-18.0.0-6336.g8626639 5.6 MB/s | 519 kB 00:00 2023-09-23T04:33:29.067 INFO:teuthology.orchestra.run.smithi069.stdout:(5/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 3.2 MB/s | 1.6 MB 00:00 2023-09-23T04:33:29.085 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:29.087 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-23T04:33:29.102 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:29.104 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-23T04:33:29.105 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-23T04:33:29.106 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-23T04:33:29.108 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-23T04:33:29.112 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-23T04:33:29.113 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-23T04:33:29.266 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-23T04:33:29.280 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.4 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el8 ceph 5.4 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 912 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el8 ceph 167 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el8 ceph 1.6 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 132 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 1.7 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 7.4 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 47 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 14 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el8 ceph 49 M 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 261 k 2023-09-23T04:33:29.281 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 181 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 260 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 18 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 31 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 832 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el8 ceph 139 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-23T04:33:29.282 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el8 ceph 197 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el8 ceph 371 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 364 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el8 ceph 107 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el8 ceph 94 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.6 M 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el8 ceph 180 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2023-09-23T04:33:29.283 INFO:teuthology.orchestra.run.smithi005.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 3.7 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 23 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 23 k 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el8 ceph 2.3 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 242 k 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el8 ceph 4.6 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el8 ceph 18 M 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el8 ceph-noarch 15 k 2023-09-23T04:33:29.284 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el8 ceph 24 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el8 ceph 184 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-23T04:33:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el8 ceph 519 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el8 ceph 6.9 M 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-23T04:33:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el8 ceph 45 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el8 ceph 125 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-23T04:33:29.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-23T04:33:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-23T04:33:29.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-23T04:33:29.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-23T04:33:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling module streams: 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql 10 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:Install 128 Packages 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 3 Packages 2023-09-23T04:33:29.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:33:29.293 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 209 M 2023-09-23T04:33:29.293 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-09-23T04:33:29.750 INFO:teuthology.orchestra.run.smithi005.stdout:(1/131): ceph-18.0.0-6336.g86266391.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2023-09-23T04:33:30.085 INFO:teuthology.orchestra.run.smithi118.stdout:(17/131): librgw2-18.0.0-6336.g86266391.el8.x86 6.4 MB/s | 6.9 MB 00:01 2023-09-23T04:33:30.150 INFO:teuthology.orchestra.run.smithi069.stdout:(6/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 4.3 MB/s | 4.6 MB 00:01 2023-09-23T04:33:30.151 INFO:teuthology.orchestra.run.smithi118.stdout:(18/131): python3-ceph-argparse-18.0.0-6336.g86 681 kB/s | 45 kB 00:00 2023-09-23T04:33:30.559 INFO:teuthology.orchestra.run.smithi005.stdout:(2/131): ceph-fuse-18.0.0-6336.g86266391.el8.x8 1.1 MB/s | 912 kB 00:00 2023-09-23T04:33:30.684 INFO:teuthology.orchestra.run.smithi005.stdout:(3/131): ceph-immutable-object-cache-18.0.0-633 1.3 MB/s | 167 kB 00:00 2023-09-23T04:33:30.768 INFO:teuthology.orchestra.run.smithi118.stdout:(19/131): python3-ceph-common-18.0.0-6336.g8626 204 kB/s | 125 kB 00:00 2023-09-23T04:33:30.868 INFO:teuthology.orchestra.run.smithi118.stdout:(20/131): python3-cephfs-18.0.0-6336.g86266391. 1.9 MB/s | 197 kB 00:00 2023-09-23T04:33:31.043 INFO:teuthology.orchestra.run.smithi118.stdout:(21/131): python3-rados-18.0.0-6336.g86266391.e 2.1 MB/s | 371 kB 00:00 2023-09-23T04:33:31.210 INFO:teuthology.orchestra.run.smithi118.stdout:(22/131): python3-rbd-18.0.0-6336.g86266391.el8 2.1 MB/s | 364 kB 00:00 2023-09-23T04:33:31.269 INFO:teuthology.orchestra.run.smithi118.stdout:(23/131): python3-rgw-18.0.0-6336.g86266391.el8 1.8 MB/s | 107 kB 00:00 2023-09-23T04:33:31.468 INFO:teuthology.orchestra.run.smithi069.stdout:(7/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 607 kB/s | 2.3 MB 00:03 2023-09-23T04:33:31.551 INFO:teuthology.orchestra.run.smithi005.stdout:(4/131): ceph-base-18.0.0-6336.g86266391.el8.x8 2.4 MB/s | 5.4 MB 00:02 2023-09-23T04:33:31.677 INFO:teuthology.orchestra.run.smithi118.stdout:(24/131): ceph-test-18.0.0-6336.g86266391.el8.x 12 MB/s | 49 MB 00:04 2023-09-23T04:33:31.702 INFO:teuthology.orchestra.run.smithi118.stdout:(25/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 216 kB/s | 94 kB 00:00 2023-09-23T04:33:31.759 INFO:teuthology.orchestra.run.smithi005.stdout:(5/131): ceph-mds-18.0.0-6336.g86266391.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-09-23T04:33:31.803 INFO:teuthology.orchestra.run.smithi118.stdout:(26/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 1.8 MB/s | 180 kB 00:00 2023-09-23T04:33:31.911 INFO:teuthology.orchestra.run.smithi118.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6336.g 213 kB/s | 23 kB 00:00 2023-09-23T04:33:32.004 INFO:teuthology.orchestra.run.smithi118.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.4 MB/s | 132 kB 00:00 2023-09-23T04:33:32.069 INFO:teuthology.orchestra.run.smithi005.stdout:(6/131): ceph-mgr-18.0.0-6336.g86266391.el8.x86 3.1 MB/s | 1.6 MB 00:00 2023-09-23T04:33:32.262 INFO:teuthology.orchestra.run.smithi118.stdout:(29/131): rbd-mirror-18.0.0-6336.g86266391.el8. 6.1 MB/s | 3.6 MB 00:00 2023-09-23T04:33:32.355 INFO:teuthology.orchestra.run.smithi118.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6336.g86266 5.0 MB/s | 1.7 MB 00:00 2023-09-23T04:33:32.418 INFO:teuthology.orchestra.run.smithi069.stdout:(8/131): ceph-osd-18.0.0-6336.g86266391.el8.x86 7.9 MB/s | 18 MB 00:02 2023-09-23T04:33:32.421 INFO:teuthology.orchestra.run.smithi118.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 3.6 MB/s | 242 kB 00:00 2023-09-23T04:33:32.488 INFO:teuthology.orchestra.run.smithi118.stdout:(32/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 714 kB/s | 47 kB 00:00 2023-09-23T04:33:32.510 INFO:teuthology.orchestra.run.smithi069.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 266 kB/s | 24 kB 00:00 2023-09-23T04:33:32.555 INFO:teuthology.orchestra.run.smithi118.stdout:(33/131): ceph-base-18.0.0-6336.g86266391.el8.x 718 kB/s | 5.4 MB 00:07 2023-09-23T04:33:32.580 INFO:teuthology.orchestra.run.smithi118.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6336.g8 158 kB/s | 15 kB 00:00 2023-09-23T04:33:32.639 INFO:teuthology.orchestra.run.smithi118.stdout:(35/131): ceph-volume-18.0.0-6336.g86266391.el8 3.1 MB/s | 261 kB 00:00 2023-09-23T04:33:32.723 INFO:teuthology.orchestra.run.smithi118.stdout:(36/131): cephfs-top-18.0.0-6336.g86266391.el8. 217 kB/s | 18 kB 00:00 2023-09-23T04:33:32.881 INFO:teuthology.orchestra.run.smithi118.stdout:(37/131): cephadm-18.0.0-6336.g86266391.el8.noa 602 kB/s | 181 kB 00:00 2023-09-23T04:33:33.098 INFO:teuthology.orchestra.run.smithi118.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 816 kB/s | 306 kB 00:00 2023-09-23T04:33:33.124 INFO:teuthology.orchestra.run.smithi118.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-09-23T04:33:33.199 INFO:teuthology.orchestra.run.smithi118.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 749 kB/s | 75 kB 00:00 2023-09-23T04:33:33.291 INFO:teuthology.orchestra.run.smithi118.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 7.2 MB/s | 7.4 MB 00:01 2023-09-23T04:33:33.316 INFO:teuthology.orchestra.run.smithi118.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2023-09-23T04:33:33.408 INFO:teuthology.orchestra.run.smithi118.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2023-09-23T04:33:33.450 INFO:teuthology.orchestra.run.smithi118.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-09-23T04:33:33.475 INFO:teuthology.orchestra.run.smithi118.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2023-09-23T04:33:33.551 INFO:teuthology.orchestra.run.smithi118.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-09-23T04:33:33.552 INFO:teuthology.orchestra.run.smithi005.stdout:(7/131): ceph-mon-18.0.0-6336.g86266391.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-09-23T04:33:33.634 INFO:teuthology.orchestra.run.smithi118.stdout:(47/131): python3-beautifulsoup4-4.6.3-2.el8.1. 540 kB/s | 185 kB 00:00 2023-09-23T04:33:33.659 INFO:teuthology.orchestra.run.smithi118.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2023-09-23T04:33:33.685 INFO:teuthology.orchestra.run.smithi118.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-09-23T04:33:33.701 INFO:teuthology.orchestra.run.smithi118.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 165 kB/s | 11 kB 00:00 2023-09-23T04:33:33.752 INFO:teuthology.orchestra.run.smithi118.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-09-23T04:33:33.785 INFO:teuthology.orchestra.run.smithi118.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 581 kB/s | 49 kB 00:00 2023-09-23T04:33:33.836 INFO:teuthology.orchestra.run.smithi118.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 703 kB/s | 59 kB 00:00 2023-09-23T04:33:33.869 INFO:teuthology.orchestra.run.smithi118.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 721 kB/s | 60 kB 00:00 2023-09-23T04:33:33.936 INFO:teuthology.orchestra.run.smithi118.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2023-09-23T04:33:33.962 INFO:teuthology.orchestra.run.smithi118.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-09-23T04:33:33.987 INFO:teuthology.orchestra.run.smithi118.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-09-23T04:33:34.012 INFO:teuthology.orchestra.run.smithi118.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-09-23T04:33:34.046 INFO:teuthology.orchestra.run.smithi118.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-09-23T04:33:34.071 INFO:teuthology.orchestra.run.smithi118.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 763 kB/s | 64 kB 00:00 2023-09-23T04:33:34.096 INFO:teuthology.orchestra.run.smithi118.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 222 kB/s | 19 kB 00:00 2023-09-23T04:33:34.122 INFO:teuthology.orchestra.run.smithi118.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-09-23T04:33:34.147 INFO:teuthology.orchestra.run.smithi118.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-09-23T04:33:34.172 INFO:teuthology.orchestra.run.smithi118.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-09-23T04:33:34.206 INFO:teuthology.orchestra.run.smithi118.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-09-23T04:33:34.248 INFO:teuthology.orchestra.run.smithi118.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-09-23T04:33:34.273 INFO:teuthology.orchestra.run.smithi118.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 609 kB/s | 61 kB 00:00 2023-09-23T04:33:34.298 INFO:teuthology.orchestra.run.smithi118.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 925 kB/s | 85 kB 00:00 2023-09-23T04:33:34.323 INFO:teuthology.orchestra.run.smithi118.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-09-23T04:33:34.349 INFO:teuthology.orchestra.run.smithi118.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-09-23T04:33:34.416 INFO:teuthology.orchestra.run.smithi118.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-09-23T04:33:34.458 INFO:teuthology.orchestra.run.smithi118.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-23T04:33:34.508 INFO:teuthology.orchestra.run.smithi118.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-09-23T04:33:34.541 INFO:teuthology.orchestra.run.smithi118.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-09-23T04:33:34.575 INFO:teuthology.orchestra.run.smithi118.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-09-23T04:33:34.625 INFO:teuthology.orchestra.run.smithi118.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 465 kB/s | 39 kB 00:00 2023-09-23T04:33:34.659 INFO:teuthology.orchestra.run.smithi118.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 8.6 MB/s | 688 kB 00:00 2023-09-23T04:33:34.692 INFO:teuthology.orchestra.run.smithi118.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-09-23T04:33:34.718 INFO:teuthology.orchestra.run.smithi118.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-23T04:33:34.743 INFO:teuthology.orchestra.run.smithi118.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-23T04:33:34.777 INFO:teuthology.orchestra.run.smithi118.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-23T04:33:34.818 INFO:teuthology.orchestra.run.smithi118.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 640 kB/s | 27 kB 00:00 2023-09-23T04:33:34.860 INFO:teuthology.orchestra.run.smithi118.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-23T04:33:34.902 INFO:teuthology.orchestra.run.smithi118.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-09-23T04:33:34.919 INFO:teuthology.orchestra.run.smithi005.stdout:(8/131): ceph-common-18.0.0-6336.g86266391.el8. 4.0 MB/s | 23 MB 00:05 2023-09-23T04:33:34.986 INFO:teuthology.orchestra.run.smithi005.stdout:(9/131): ceph-selinux-18.0.0-6336.g86266391.el8 366 kB/s | 24 kB 00:00 2023-09-23T04:33:35.036 INFO:teuthology.orchestra.run.smithi118.stdout:(85/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-09-23T04:33:35.112 INFO:teuthology.orchestra.run.smithi118.stdout:(86/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-09-23T04:33:35.411 INFO:teuthology.orchestra.run.smithi118.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.2 MB/s | 457 kB 00:00 2023-09-23T04:33:35.568 INFO:teuthology.orchestra.run.smithi118.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 494 kB/s | 225 kB 00:00 2023-09-23T04:33:35.588 INFO:teuthology.orchestra.run.smithi118.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 362 kB/s | 64 kB 00:00 2023-09-23T04:33:35.613 INFO:teuthology.orchestra.run.smithi118.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.8 MB/s | 126 kB 00:00 2023-09-23T04:33:35.646 INFO:teuthology.orchestra.run.smithi118.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 343 kB/s | 20 kB 00:00 2023-09-23T04:33:35.671 INFO:teuthology.orchestra.run.smithi118.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-09-23T04:33:35.705 INFO:teuthology.orchestra.run.smithi118.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-23T04:33:35.789 INFO:teuthology.orchestra.run.smithi118.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 4.2 MB/s | 3.7 MB 00:00 2023-09-23T04:33:35.814 INFO:teuthology.orchestra.run.smithi118.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2023-09-23T04:33:35.856 INFO:teuthology.orchestra.run.smithi118.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-23T04:33:35.906 INFO:teuthology.orchestra.run.smithi118.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-09-23T04:33:35.918 INFO:teuthology.orchestra.run.smithi069.stdout:(10/131): ceph-common-18.0.0-6336.g86266391.el8 2.1 MB/s | 23 MB 00:10 2023-09-23T04:33:35.990 INFO:teuthology.orchestra.run.smithi118.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-09-23T04:33:36.068 INFO:teuthology.orchestra.run.smithi069.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 1.7 MB/s | 260 kB 00:00 2023-09-23T04:33:36.073 INFO:teuthology.orchestra.run.smithi118.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-09-23T04:33:36.115 INFO:teuthology.orchestra.run.smithi118.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-09-23T04:33:36.144 INFO:teuthology.orchestra.run.smithi069.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 415 kB/s | 31 kB 00:00 2023-09-23T04:33:36.157 INFO:teuthology.orchestra.run.smithi118.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-09-23T04:33:36.191 INFO:teuthology.orchestra.run.smithi118.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-09-23T04:33:36.216 INFO:teuthology.orchestra.run.smithi118.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-09-23T04:33:36.241 INFO:teuthology.orchestra.run.smithi118.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-09-23T04:33:36.267 INFO:teuthology.orchestra.run.smithi118.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-23T04:33:36.292 INFO:teuthology.orchestra.run.smithi118.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-09-23T04:33:36.317 INFO:teuthology.orchestra.run.smithi118.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-23T04:33:36.342 INFO:teuthology.orchestra.run.smithi118.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-09-23T04:33:36.368 INFO:teuthology.orchestra.run.smithi118.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-23T04:33:36.393 INFO:teuthology.orchestra.run.smithi118.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-23T04:33:36.427 INFO:teuthology.orchestra.run.smithi118.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-09-23T04:33:36.485 INFO:teuthology.orchestra.run.smithi069.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 2.4 MB/s | 832 kB 00:00 2023-09-23T04:33:36.493 INFO:teuthology.orchestra.run.smithi118.stdout:(112/131): postgresql-server-10.21-2.module+el8 8.6 MB/s | 5.1 MB 00:00 2023-09-23T04:33:36.519 INFO:teuthology.orchestra.run.smithi118.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-09-23T04:33:36.544 INFO:teuthology.orchestra.run.smithi118.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.3 MB/s | 155 kB 00:00 2023-09-23T04:33:36.569 INFO:teuthology.orchestra.run.smithi118.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 355 kB/s | 18 kB 00:00 2023-09-23T04:33:36.594 INFO:teuthology.orchestra.run.smithi069.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 1.7 MB/s | 184 kB 00:00 2023-09-23T04:33:36.603 INFO:teuthology.orchestra.run.smithi118.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-09-23T04:33:36.636 INFO:teuthology.orchestra.run.smithi118.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-09-23T04:33:36.661 INFO:teuthology.orchestra.run.smithi069.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 2.0 MB/s | 139 kB 00:00 2023-09-23T04:33:36.661 INFO:teuthology.orchestra.run.smithi118.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-09-23T04:33:36.687 INFO:teuthology.orchestra.run.smithi118.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-23T04:33:36.712 INFO:teuthology.orchestra.run.smithi118.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-23T04:33:36.737 INFO:teuthology.orchestra.run.smithi118.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-23T04:33:36.763 INFO:teuthology.orchestra.run.smithi118.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-23T04:33:36.788 INFO:teuthology.orchestra.run.smithi118.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-23T04:33:36.802 INFO:teuthology.orchestra.run.smithi005.stdout:(10/131): ceph-osd-18.0.0-6336.g86266391.el8.x8 3.8 MB/s | 18 MB 00:04 2023-09-23T04:33:36.813 INFO:teuthology.orchestra.run.smithi118.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-09-23T04:33:36.878 INFO:teuthology.orchestra.run.smithi118.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 7.0 MB/s | 644 kB 00:00 2023-09-23T04:33:36.894 INFO:teuthology.orchestra.run.smithi005.stdout:(11/131): cephfs-mirror-18.0.0-6336.g86266391.e 2.8 MB/s | 260 kB 00:00 2023-09-23T04:33:36.914 INFO:teuthology.orchestra.run.smithi118.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-09-23T04:33:36.939 INFO:teuthology.orchestra.run.smithi118.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.2 MB/s | 138 kB 00:00 2023-09-23T04:33:36.986 INFO:teuthology.orchestra.run.smithi005.stdout:(12/131): libcephfs-devel-18.0.0-6336.g86266391 339 kB/s | 31 kB 00:00 2023-09-23T04:33:36.989 INFO:teuthology.orchestra.run.smithi118.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-09-23T04:33:37.040 INFO:teuthology.orchestra.run.smithi118.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-23T04:33:37.145 INFO:teuthology.orchestra.run.smithi069.stdout:(16/131): ceph-test-18.0.0-6336.g86266391.el8.x 11 MB/s | 49 MB 00:04 2023-09-23T04:33:37.169 INFO:teuthology.orchestra.run.smithi069.stdout:(17/131): libradosstriper1-18.0.0-6336.g8626639 1.0 MB/s | 519 kB 00:00 2023-09-23T04:33:37.220 INFO:teuthology.orchestra.run.smithi005.stdout:(13/131): libcephfs2-18.0.0-6336.g86266391.el8. 3.5 MB/s | 832 kB 00:00 2023-09-23T04:33:37.245 INFO:teuthology.orchestra.run.smithi069.stdout:(18/131): python3-ceph-argparse-18.0.0-6336.g86 596 kB/s | 45 kB 00:00 2023-09-23T04:33:37.353 INFO:teuthology.orchestra.run.smithi005.stdout:(14/131): libcephsqlite-18.0.0-6336.g86266391.e 1.3 MB/s | 184 kB 00:00 2023-09-23T04:33:37.395 INFO:teuthology.orchestra.run.smithi069.stdout:(19/131): python3-ceph-common-18.0.0-6336.g8626 838 kB/s | 125 kB 00:00 2023-09-23T04:33:37.479 INFO:teuthology.orchestra.run.smithi005.stdout:(15/131): librados-devel-18.0.0-6336.g86266391. 1.1 MB/s | 139 kB 00:00 2023-09-23T04:33:37.529 INFO:teuthology.orchestra.run.smithi069.stdout:(20/131): python3-cephfs-18.0.0-6336.g86266391. 1.4 MB/s | 197 kB 00:00 2023-09-23T04:33:37.623 INFO:teuthology.orchestra.run.smithi118.stdout:(130/131): librados2-18.0.0-6336.g86266391.el8. 5.3 MB/s | 3.7 MB 00:00 2023-09-23T04:33:37.696 INFO:teuthology.orchestra.run.smithi069.stdout:(21/131): python3-rados-18.0.0-6336.g86266391.e 2.2 MB/s | 371 kB 00:00 2023-09-23T04:33:37.704 INFO:teuthology.orchestra.run.smithi005.stdout:(16/131): libradosstriper1-18.0.0-6336.g8626639 2.3 MB/s | 519 kB 00:00 2023-09-23T04:33:37.846 INFO:teuthology.orchestra.run.smithi069.stdout:(22/131): python3-rbd-18.0.0-6336.g86266391.el8 2.4 MB/s | 364 kB 00:00 2023-09-23T04:33:37.930 INFO:teuthology.orchestra.run.smithi069.stdout:(23/131): librgw2-18.0.0-6336.g86266391.el8.x86 8.8 MB/s | 6.9 MB 00:00 2023-09-23T04:33:37.955 INFO:teuthology.orchestra.run.smithi069.stdout:(24/131): python3-rgw-18.0.0-6336.g86266391.el8 983 kB/s | 107 kB 00:00 2023-09-23T04:33:38.022 INFO:teuthology.orchestra.run.smithi069.stdout:(25/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 1.0 MB/s | 94 kB 00:00 2023-09-23T04:33:38.112 INFO:teuthology.orchestra.run.smithi005.stdout:(17/131): ceph-radosgw-18.0.0-6336.g86266391.el 3.0 MB/s | 14 MB 00:04 2023-09-23T04:33:38.122 INFO:teuthology.orchestra.run.smithi069.stdout:(26/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 1.8 MB/s | 180 kB 00:00 2023-09-23T04:33:38.204 INFO:teuthology.orchestra.run.smithi005.stdout:(18/131): python3-ceph-argparse-18.0.0-6336.g86 490 kB/s | 45 kB 00:00 2023-09-23T04:33:38.222 INFO:teuthology.orchestra.run.smithi069.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6336.g 230 kB/s | 23 kB 00:00 2023-09-23T04:33:38.288 INFO:teuthology.orchestra.run.smithi005.stdout:(19/131): python3-ceph-common-18.0.0-6336.g8626 1.5 MB/s | 125 kB 00:00 2023-09-23T04:33:38.314 INFO:teuthology.orchestra.run.smithi069.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.4 MB/s | 132 kB 00:00 2023-09-23T04:33:38.389 INFO:teuthology.orchestra.run.smithi005.stdout:(20/131): python3-cephfs-18.0.0-6336.g86266391. 1.9 MB/s | 197 kB 00:00 2023-09-23T04:33:38.548 INFO:teuthology.orchestra.run.smithi069.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6336.g86266 7.5 MB/s | 1.7 MB 00:00 2023-09-23T04:33:38.580 INFO:teuthology.orchestra.run.smithi005.stdout:(21/131): python3-rados-18.0.0-6336.g86266391.e 1.9 MB/s | 371 kB 00:00 2023-09-23T04:33:38.798 INFO:teuthology.orchestra.run.smithi005.stdout:(22/131): python3-rbd-18.0.0-6336.g86266391.el8 1.6 MB/s | 364 kB 00:00 2023-09-23T04:33:38.881 INFO:teuthology.orchestra.run.smithi069.stdout:(30/131): rbd-mirror-18.0.0-6336.g86266391.el8. 3.9 MB/s | 3.6 MB 00:00 2023-09-23T04:33:38.906 INFO:teuthology.orchestra.run.smithi005.stdout:(23/131): python3-rgw-18.0.0-6336.g86266391.el8 988 kB/s | 107 kB 00:00 2023-09-23T04:33:39.015 INFO:teuthology.orchestra.run.smithi005.stdout:(24/131): rbd-fuse-18.0.0-6336.g86266391.el8.x8 864 kB/s | 94 kB 00:00 2023-09-23T04:33:39.023 INFO:teuthology.orchestra.run.smithi069.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 1.7 MB/s | 242 kB 00:00 2023-09-23T04:33:39.165 INFO:teuthology.orchestra.run.smithi069.stdout:(32/131): ceph-radosgw-18.0.0-6336.g86266391.el 1.8 MB/s | 14 MB 00:07 2023-09-23T04:33:39.182 INFO:teuthology.orchestra.run.smithi069.stdout:(33/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 299 kB/s | 47 kB 00:00 2023-09-23T04:33:39.324 INFO:teuthology.orchestra.run.smithi069.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 9.5 MB/s | 7.4 MB 00:00 2023-09-23T04:33:39.349 INFO:teuthology.orchestra.run.smithi069.stdout:(35/131): ceph-prometheus-alerts-18.0.0-6336.g8 79 kB/s | 15 kB 00:00 2023-09-23T04:33:39.375 INFO:teuthology.orchestra.run.smithi069.stdout:(36/131): ceph-volume-18.0.0-6336.g86266391.el8 1.3 MB/s | 261 kB 00:00 2023-09-23T04:33:39.400 INFO:teuthology.orchestra.run.smithi069.stdout:(37/131): cephadm-18.0.0-6336.g86266391.el8.noa 2.3 MB/s | 181 kB 00:00 2023-09-23T04:33:39.426 INFO:teuthology.orchestra.run.smithi069.stdout:(38/131): cephfs-top-18.0.0-6336.g86266391.el8. 239 kB/s | 18 kB 00:00 2023-09-23T04:33:39.465 INFO:teuthology.orchestra.run.smithi005.stdout:(25/131): librgw2-18.0.0-6336.g86266391.el8.x86 3.9 MB/s | 6.9 MB 00:01 2023-09-23T04:33:39.534 INFO:teuthology.orchestra.run.smithi069.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2023-09-23T04:33:39.540 INFO:teuthology.orchestra.run.smithi005.stdout:(26/131): rbd-nbd-18.0.0-6336.g86266391.el8.x86 2.4 MB/s | 180 kB 00:00 2023-09-23T04:33:39.576 INFO:teuthology.orchestra.run.smithi069.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2023-09-23T04:33:39.618 INFO:teuthology.orchestra.run.smithi069.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-09-23T04:33:39.624 INFO:teuthology.orchestra.run.smithi005.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6336.g 278 kB/s | 23 kB 00:00 2023-09-23T04:33:39.643 INFO:teuthology.orchestra.run.smithi069.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 652 kB/s | 44 kB 00:00 2023-09-23T04:33:39.686 INFO:teuthology.orchestra.run.smithi069.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-09-23T04:33:39.708 INFO:teuthology.orchestra.run.smithi005.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.5 MB/s | 132 kB 00:00 2023-09-23T04:33:39.711 INFO:teuthology.orchestra.run.smithi069.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 491 kB/s | 33 kB 00:00 2023-09-23T04:33:39.727 INFO:teuthology.orchestra.run.smithi069.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 390 kB/s | 16 kB 00:00 2023-09-23T04:33:39.753 INFO:teuthology.orchestra.run.smithi069.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-09-23T04:33:39.803 INFO:teuthology.orchestra.run.smithi069.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-09-23T04:33:39.828 INFO:teuthology.orchestra.run.smithi069.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-09-23T04:33:39.840 INFO:teuthology.orchestra.run.smithi118.stdout:(131/131): librbd1-18.0.0-6336.g86266391.el8.x8 1.3 MB/s | 3.7 MB 00:02 2023-09-23T04:33:39.843 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:33:39.843 INFO:teuthology.orchestra.run.smithi118.stdout:Total 14 MB/s | 209 MB 00:15 2023-09-23T04:33:39.854 INFO:teuthology.orchestra.run.smithi069.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 809 kB/s | 81 kB 00:00 2023-09-23T04:33:39.870 INFO:teuthology.orchestra.run.smithi069.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 165 kB/s | 11 kB 00:00 2023-09-23T04:33:39.887 INFO:teuthology.orchestra.run.smithi069.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2023-09-23T04:33:39.913 INFO:teuthology.orchestra.run.smithi069.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-09-23T04:33:39.925 INFO:teuthology.orchestra.run.smithi005.stdout:(29/131): rbd-mirror-18.0.0-6336.g86266391.el8. 3.9 MB/s | 3.6 MB 00:00 2023-09-23T04:33:39.929 INFO:teuthology.orchestra.run.smithi069.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-09-23T04:33:39.972 INFO:teuthology.orchestra.run.smithi069.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-09-23T04:33:39.988 INFO:teuthology.orchestra.run.smithi069.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 799 kB/s | 60 kB 00:00 2023-09-23T04:33:40.014 INFO:teuthology.orchestra.run.smithi069.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-09-23T04:33:40.039 INFO:teuthology.orchestra.run.smithi069.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-09-23T04:33:40.042 INFO:teuthology.orchestra.run.smithi005.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6336.g86266 5.2 MB/s | 1.7 MB 00:00 2023-09-23T04:33:40.064 INFO:teuthology.orchestra.run.smithi069.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 445 kB/s | 34 kB 00:00 2023-09-23T04:33:40.090 INFO:teuthology.orchestra.run.smithi069.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-09-23T04:33:40.115 INFO:teuthology.orchestra.run.smithi069.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-09-23T04:33:40.125 INFO:teuthology.orchestra.run.smithi005.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6336.g86 2.8 MB/s | 242 kB 00:00 2023-09-23T04:33:40.140 INFO:teuthology.orchestra.run.smithi069.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-09-23T04:33:40.174 INFO:teuthology.orchestra.run.smithi069.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-09-23T04:33:40.199 INFO:teuthology.orchestra.run.smithi069.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 349 kB/s | 29 kB 00:00 2023-09-23T04:33:40.200 INFO:teuthology.orchestra.run.smithi005.stdout:(32/131): ceph-mgr-rook-18.0.0-6336.g86266391.e 634 kB/s | 47 kB 00:00 2023-09-23T04:33:40.224 INFO:teuthology.orchestra.run.smithi069.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 328 kB/s | 27 kB 00:00 2023-09-23T04:33:40.249 INFO:teuthology.orchestra.run.smithi069.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-09-23T04:33:40.277 INFO:teuthology.orchestra.run.smithi069.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-09-23T04:33:40.292 INFO:teuthology.orchestra.run.smithi005.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6336.g8 159 kB/s | 15 kB 00:00 2023-09-23T04:33:40.300 INFO:teuthology.orchestra.run.smithi069.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-09-23T04:33:40.325 INFO:teuthology.orchestra.run.smithi069.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.9 MB/s | 251 kB 00:00 2023-09-23T04:33:40.351 INFO:teuthology.orchestra.run.smithi069.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-09-23T04:33:40.368 INFO:teuthology.orchestra.run.smithi005.stdout:(34/131): ceph-volume-18.0.0-6336.g86266391.el8 3.4 MB/s | 261 kB 00:00 2023-09-23T04:33:40.376 INFO:teuthology.orchestra.run.smithi069.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-09-23T04:33:40.426 INFO:teuthology.orchestra.run.smithi069.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-09-23T04:33:40.460 INFO:teuthology.orchestra.run.smithi005.stdout:(35/131): cephadm-18.0.0-6336.g86266391.el8.noa 1.9 MB/s | 181 kB 00:00 2023-09-23T04:33:40.460 INFO:teuthology.orchestra.run.smithi069.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 432 kB/s | 47 kB 00:00 2023-09-23T04:33:40.493 INFO:teuthology.orchestra.run.smithi069.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-09-23T04:33:40.519 INFO:teuthology.orchestra.run.smithi069.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 492 kB/s | 43 kB 00:00 2023-09-23T04:33:40.535 INFO:teuthology.orchestra.run.smithi005.stdout:(36/131): cephfs-top-18.0.0-6336.g86266391.el8. 241 kB/s | 18 kB 00:00 2023-09-23T04:33:40.544 INFO:teuthology.orchestra.run.smithi069.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 956 kB/s | 48 kB 00:00 2023-09-23T04:33:40.561 INFO:teuthology.orchestra.run.smithi069.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-09-23T04:33:40.586 INFO:teuthology.orchestra.run.smithi069.stdout:(77/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-09-23T04:33:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 636 kB/s | 43 kB 00:00 2023-09-23T04:33:40.637 INFO:teuthology.orchestra.run.smithi069.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-23T04:33:40.662 INFO:teuthology.orchestra.run.smithi069.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-09-23T04:33:40.687 INFO:teuthology.orchestra.run.smithi069.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-09-23T04:33:40.694 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2023-09-23T04:33:40.721 INFO:teuthology.orchestra.run.smithi069.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-09-23T04:33:40.754 INFO:teuthology.orchestra.run.smithi069.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-09-23T04:33:40.771 INFO:teuthology.orchestra.run.smithi069.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 356 kB/s | 18 kB 00:00 2023-09-23T04:33:40.805 INFO:teuthology.orchestra.run.smithi069.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-09-23T04:33:40.812 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2023-09-23T04:33:40.812 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2023-09-23T04:33:40.827 INFO:teuthology.orchestra.run.smithi005.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-09-23T04:33:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-09-23T04:33:40.897 INFO:teuthology.orchestra.run.smithi069.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-09-23T04:33:40.930 INFO:teuthology.orchestra.run.smithi069.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.4 MB/s | 605 kB 00:00 2023-09-23T04:33:40.952 INFO:teuthology.orchestra.run.smithi005.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-09-23T04:33:40.956 INFO:teuthology.orchestra.run.smithi069.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-09-23T04:33:40.981 INFO:teuthology.orchestra.run.smithi069.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-23T04:33:40.998 INFO:teuthology.orchestra.run.smithi069.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-09-23T04:33:41.023 INFO:teuthology.orchestra.run.smithi069.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-09-23T04:33:41.048 INFO:teuthology.orchestra.run.smithi069.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-23T04:33:41.074 INFO:teuthology.orchestra.run.smithi069.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-09-23T04:33:41.116 INFO:teuthology.orchestra.run.smithi069.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-09-23T04:33:41.141 INFO:teuthology.orchestra.run.smithi069.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-09-23T04:33:41.158 INFO:teuthology.orchestra.run.smithi069.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-09-23T04:33:41.217 INFO:teuthology.orchestra.run.smithi069.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.6 MB/s | 2.3 MB 00:00 2023-09-23T04:33:41.242 INFO:teuthology.orchestra.run.smithi069.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-09-23T04:33:41.267 INFO:teuthology.orchestra.run.smithi069.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-23T04:33:41.293 INFO:teuthology.orchestra.run.smithi069.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-09-23T04:33:41.326 INFO:teuthology.orchestra.run.smithi069.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-09-23T04:33:41.352 INFO:teuthology.orchestra.run.smithi069.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-09-23T04:33:41.385 INFO:teuthology.orchestra.run.smithi069.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-09-23T04:33:41.419 INFO:teuthology.orchestra.run.smithi069.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-09-23T04:33:41.444 INFO:teuthology.orchestra.run.smithi069.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-23T04:33:41.469 INFO:teuthology.orchestra.run.smithi069.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-23T04:33:41.470 INFO:teuthology.orchestra.run.smithi005.stdout:(40/131): ceph-test-18.0.0-6336.g86266391.el8.x 7.5 MB/s | 49 MB 00:06 2023-09-23T04:33:41.495 INFO:teuthology.orchestra.run.smithi005.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 878 kB/s | 476 kB 00:00 2023-09-23T04:33:41.503 INFO:teuthology.orchestra.run.smithi069.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-23T04:33:41.545 INFO:teuthology.orchestra.run.smithi069.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 638 kB/s | 27 kB 00:00 2023-09-23T04:33:41.587 INFO:teuthology.orchestra.run.smithi069.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-23T04:33:41.595 INFO:teuthology.orchestra.run.smithi005.stdout:(42/131): ceph-mgr-diskprediction-local-18.0.0- 4.4 MB/s | 7.4 MB 00:01 2023-09-23T04:33:41.620 INFO:teuthology.orchestra.run.smithi005.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-09-23T04:33:41.628 INFO:teuthology.orchestra.run.smithi069.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-23T04:33:41.704 INFO:teuthology.orchestra.run.smithi005.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-09-23T04:33:41.729 INFO:teuthology.orchestra.run.smithi005.stdout:(45/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 168 kB/s | 44 kB 00:00 2023-09-23T04:33:41.738 INFO:teuthology.orchestra.run.smithi069.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-09-23T04:33:41.754 INFO:teuthology.orchestra.run.smithi005.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 208 kB/s | 33 kB 00:00 2023-09-23T04:33:41.788 INFO:teuthology.orchestra.run.smithi069.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.1 MB/s | 457 kB 00:00 2023-09-23T04:33:41.805 INFO:teuthology.orchestra.run.smithi005.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-09-23T04:33:41.830 INFO:teuthology.orchestra.run.smithi069.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-23T04:33:41.855 INFO:teuthology.orchestra.run.smithi005.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-09-23T04:33:41.889 INFO:teuthology.orchestra.run.smithi005.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 606 kB/s | 81 kB 00:00 2023-09-23T04:33:41.896 INFO:teuthology.orchestra.run.smithi069.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-09-23T04:33:41.906 INFO:teuthology.orchestra.run.smithi005.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2023-09-23T04:33:41.922 INFO:teuthology.orchestra.run.smithi069.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-09-23T04:33:41.946 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2023-09-23T04:33:41.947 INFO:teuthology.orchestra.run.smithi069.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-23T04:33:41.950 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2023-09-23T04:33:41.964 INFO:teuthology.orchestra.run.smithi005.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2023-09-23T04:33:41.972 INFO:teuthology.orchestra.run.smithi069.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-09-23T04:33:41.981 INFO:teuthology.orchestra.run.smithi005.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 649 kB/s | 49 kB 00:00 2023-09-23T04:33:41.997 INFO:teuthology.orchestra.run.smithi069.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-23T04:33:42.023 INFO:teuthology.orchestra.run.smithi005.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-09-23T04:33:42.031 INFO:teuthology.orchestra.run.smithi069.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-23T04:33:42.040 INFO:teuthology.orchestra.run.smithi005.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-09-23T04:33:42.107 INFO:teuthology.orchestra.run.smithi005.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2023-09-23T04:33:42.133 INFO:teuthology.orchestra.run.smithi005.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-09-23T04:33:42.173 INFO:teuthology.orchestra.run.smithi069.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-09-23T04:33:42.175 INFO:teuthology.orchestra.run.smithi005.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2023-09-23T04:33:42.198 INFO:teuthology.orchestra.run.smithi069.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-09-23T04:33:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-09-23T04:33:42.225 INFO:teuthology.orchestra.run.smithi005.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-09-23T04:33:42.241 INFO:teuthology.orchestra.run.smithi069.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-23T04:33:42.250 INFO:teuthology.orchestra.run.smithi005.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 853 kB/s | 64 kB 00:00 2023-09-23T04:33:42.275 INFO:teuthology.orchestra.run.smithi005.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-09-23T04:33:42.282 INFO:teuthology.orchestra.run.smithi069.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-23T04:33:42.301 INFO:teuthology.orchestra.run.smithi005.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-09-23T04:33:42.326 INFO:teuthology.orchestra.run.smithi005.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-09-23T04:33:42.351 INFO:teuthology.orchestra.run.smithi005.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-09-23T04:33:42.357 INFO:teuthology.orchestra.run.smithi069.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 14 MB/s | 4.9 MB 00:00 2023-09-23T04:33:42.385 INFO:teuthology.orchestra.run.smithi005.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-09-23T04:33:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-09-23T04:33:42.433 INFO:teuthology.orchestra.run.smithi069.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-09-23T04:33:42.436 INFO:teuthology.orchestra.run.smithi005.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-09-23T04:33:42.461 INFO:teuthology.orchestra.run.smithi005.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-09-23T04:33:42.466 INFO:teuthology.orchestra.run.smithi069.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-09-23T04:33:42.486 INFO:teuthology.orchestra.run.smithi005.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-09-23T04:33:42.512 INFO:teuthology.orchestra.run.smithi005.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-09-23T04:33:42.525 INFO:teuthology.orchestra.run.smithi069.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-09-23T04:33:42.553 INFO:teuthology.orchestra.run.smithi005.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 704 kB/s | 47 kB 00:00 2023-09-23T04:33:42.595 INFO:teuthology.orchestra.run.smithi005.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-09-23T04:33:42.600 INFO:teuthology.orchestra.run.smithi069.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-09-23T04:33:42.612 INFO:teuthology.orchestra.run.smithi005.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 729 kB/s | 43 kB 00:00 2023-09-23T04:33:42.646 INFO:teuthology.orchestra.run.smithi005.stdout:(74/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2023-09-23T04:33:42.650 INFO:teuthology.orchestra.run.smithi069.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-23T04:33:42.663 INFO:teuthology.orchestra.run.smithi005.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-09-23T04:33:42.697 INFO:teuthology.orchestra.run.smithi005.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-23T04:33:42.714 INFO:teuthology.orchestra.run.smithi005.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 847 kB/s | 43 kB 00:00 2023-09-23T04:33:42.739 INFO:teuthology.orchestra.run.smithi005.stdout:(78/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-09-23T04:33:42.764 INFO:teuthology.orchestra.run.smithi005.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-09-23T04:33:42.798 INFO:teuthology.orchestra.run.smithi005.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-09-23T04:33:42.823 INFO:teuthology.orchestra.run.smithi005.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-09-23T04:33:42.857 INFO:teuthology.orchestra.run.smithi005.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-09-23T04:33:42.899 INFO:teuthology.orchestra.run.smithi005.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-09-23T04:33:42.925 INFO:teuthology.orchestra.run.smithi005.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-09-23T04:33:42.958 INFO:teuthology.orchestra.run.smithi005.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-09-23T04:33:43.006 INFO:teuthology.orchestra.run.smithi069.stdout:(130/131): librbd1-18.0.0-6336.g86266391.el8.x8 7.8 MB/s | 3.7 MB 00:00 2023-09-23T04:33:43.007 INFO:teuthology.orchestra.run.smithi005.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-23T04:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-09-23T04:33:43.059 INFO:teuthology.orchestra.run.smithi005.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-09-23T04:33:43.109 INFO:teuthology.orchestra.run.smithi005.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-23T04:33:43.134 INFO:teuthology.orchestra.run.smithi005.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 798 kB/s | 60 kB 00:00 2023-09-23T04:33:43.160 INFO:teuthology.orchestra.run.smithi005.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-23T04:33:43.176 INFO:teuthology.orchestra.run.smithi069.stdout:(131/131): librados2-18.0.0-6336.g86266391.el8. 5.3 MB/s | 3.7 MB 00:00 2023-09-23T04:33:43.179 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:33:43.179 INFO:teuthology.orchestra.run.smithi069.stdout:Total 11 MB/s | 209 MB 00:18 2023-09-23T04:33:43.185 INFO:teuthology.orchestra.run.smithi005.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-23T04:33:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 773 kB/s | 39 kB 00:00 2023-09-23T04:33:43.252 INFO:teuthology.orchestra.run.smithi005.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2023-09-23T04:33:43.286 INFO:teuthology.orchestra.run.smithi005.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.3 MB/s | 644 kB 00:00 2023-09-23T04:33:43.311 INFO:teuthology.orchestra.run.smithi005.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-09-23T04:33:43.353 INFO:teuthology.orchestra.run.smithi005.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-09-23T04:33:43.395 INFO:teuthology.orchestra.run.smithi005.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-23T04:33:43.437 INFO:teuthology.orchestra.run.smithi005.stdout:(99/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2023-09-23T04:33:43.470 INFO:teuthology.orchestra.run.smithi005.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-09-23T04:33:43.504 INFO:teuthology.orchestra.run.smithi005.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.8 MB/s | 323 kB 00:00 2023-09-23T04:33:43.529 INFO:teuthology.orchestra.run.smithi005.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 666 kB/s | 39 kB 00:00 2023-09-23T04:33:43.580 INFO:teuthology.orchestra.run.smithi005.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-09-23T04:33:43.622 INFO:teuthology.orchestra.run.smithi005.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 5.7 MB/s | 538 kB 00:00 2023-09-23T04:33:43.655 INFO:teuthology.orchestra.run.smithi005.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-09-23T04:33:43.689 INFO:teuthology.orchestra.run.smithi005.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-09-23T04:33:43.739 INFO:teuthology.orchestra.run.smithi005.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-09-23T04:33:43.773 INFO:teuthology.orchestra.run.smithi005.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 800 kB/s | 27 kB 00:00 2023-09-23T04:33:43.823 INFO:teuthology.orchestra.run.smithi005.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-09-23T04:33:43.865 INFO:teuthology.orchestra.run.smithi005.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-23T04:33:44.016 INFO:teuthology.orchestra.run.smithi005.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-09-23T04:33:44.016 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2023-09-23T04:33:44.066 INFO:teuthology.orchestra.run.smithi005.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-09-23T04:33:44.135 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2023-09-23T04:33:44.136 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2023-09-23T04:33:44.141 INFO:teuthology.orchestra.run.smithi005.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-09-23T04:33:44.174 INFO:teuthology.orchestra.run.smithi005.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-09-23T04:33:44.199 INFO:teuthology.orchestra.run.smithi005.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-09-23T04:33:44.225 INFO:teuthology.orchestra.run.smithi005.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-23T04:33:44.251 INFO:teuthology.orchestra.run.smithi005.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 395 kB/s | 20 kB 00:00 2023-09-23T04:33:44.275 INFO:teuthology.orchestra.run.smithi005.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-23T04:33:44.318 INFO:teuthology.orchestra.run.smithi005.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-09-23T04:33:44.361 INFO:teuthology.orchestra.run.smithi005.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-09-23T04:33:44.760 INFO:teuthology.orchestra.run.smithi005.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 9.3 MB/s | 14 MB 00:01 2023-09-23T04:33:44.802 INFO:teuthology.orchestra.run.smithi005.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-23T04:33:44.852 INFO:teuthology.orchestra.run.smithi005.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-09-23T04:33:44.936 INFO:teuthology.orchestra.run.smithi005.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 7.4 MB/s | 4.9 MB 00:00 2023-09-23T04:33:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:(125/131): postgresql-contrib-10.21-2.module+el 11 MB/s | 808 kB 00:00 2023-09-23T04:33:45.078 INFO:teuthology.orchestra.run.smithi005.stdout:(126/131): openblas-0.3.15-3.el8.x86_64.rpm 6.6 MB/s | 4.7 MB 00:00 2023-09-23T04:33:45.153 INFO:teuthology.orchestra.run.smithi005.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-09-23T04:33:45.253 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2023-09-23T04:33:45.254 INFO:teuthology.orchestra.run.smithi005.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-09-23T04:33:45.257 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2023-09-23T04:33:45.304 INFO:teuthology.orchestra.run.smithi005.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-23T04:33:46.121 INFO:teuthology.orchestra.run.smithi005.stdout:(130/131): librados2-18.0.0-6336.g86266391.el8. 3.6 MB/s | 3.7 MB 00:01 2023-09-23T04:33:46.189 INFO:teuthology.orchestra.run.smithi005.stdout:(131/131): librbd1-18.0.0-6336.g86266391.el8.x8 3.6 MB/s | 3.7 MB 00:01 2023-09-23T04:33:46.191 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:33:46.191 INFO:teuthology.orchestra.run.smithi005.stdout:Total 12 MB/s | 209 MB 00:16 2023-09-23T04:33:46.789 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2023-09-23T04:33:46.968 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T04:33:47.133 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T04:33:47.344 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-23T04:33:47.495 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-23T04:33:47.603 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-23T04:33:47.829 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-23T04:33:47.879 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-09-23T04:33:48.023 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-09-23T04:33:48.023 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-09-23T04:33:48.234 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-23T04:33:48.420 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:48.827 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:49.029 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:49.160 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:49.353 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:33:49.452 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:33:49.627 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T04:33:49.641 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-09-23T04:33:49.651 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-09-23T04:33:49.856 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T04:33:50.056 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T04:33:50.426 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2023-09-23T04:33:50.453 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T04:33:50.601 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T04:33:50.718 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T04:33:50.784 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T04:33:50.878 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:50.936 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T04:33:50.969 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:51.041 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T04:33:51.111 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:33:51.180 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:33:51.252 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T04:33:51.311 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-23T04:33:51.449 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-23T04:33:51.720 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T04:33:51.886 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-23T04:33:52.011 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T04:33:52.407 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T04:33:52.497 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-23T04:33:52.561 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:52.670 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-23T04:33:52.696 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:52.894 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:52.988 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:53.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:33:53.253 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-23T04:33:53.293 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-23T04:33:53.419 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-23T04:33:53.522 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T04:33:53.730 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T04:33:54.104 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T04:33:54.109 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-23T04:33:54.309 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T04:33:54.401 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-23T04:33:54.444 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:54.547 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:54.621 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-23T04:33:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-09-23T04:33:54.685 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:33:54.834 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:33:54.840 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-23T04:33:55.034 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-23T04:33:55.055 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-23T04:33:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-23T04:33:55.232 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-23T04:33:55.233 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-23T04:33:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-23T04:33:55.391 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-23T04:33:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T04:33:55.496 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T04:33:55.518 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T04:33:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-23T04:33:55.661 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T04:33:55.667 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T04:33:55.780 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T04:33:55.810 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T04:33:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T04:33:56.378 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-23T04:33:56.409 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T04:33:56.571 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T04:33:56.576 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T04:33:56.674 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T04:33:57.041 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-23T04:33:57.071 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el8.x86_64 5/134 2023-09-23T04:33:57.161 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T04:33:57.279 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:57.312 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-23T04:33:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T04:33:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:33:57.568 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-23T04:33:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:33:57.785 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-23T04:33:57.894 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:33:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T04:33:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-23T04:33:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-23T04:33:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-23T04:33:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-23T04:33:58.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-23T04:33:58.678 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T04:33:58.858 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T04:33:58.987 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T04:33:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-23T04:33:59.197 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T04:33:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-23T04:33:59.376 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T04:33:59.445 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:59.530 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-23T04:33:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 14/134 2023-09-23T04:33:59.753 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:33:59.835 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-23T04:34:00.046 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T04:34:00.047 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 15/134 2023-09-23T04:34:00.220 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T04:34:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-23T04:34:00.407 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T04:34:00.556 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-23T04:34:00.572 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T04:34:00.717 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-23T04:34:00.764 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-23T04:34:00.882 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-23T04:34:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T04:34:01.062 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-23T04:34:01.080 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-23T04:34:01.119 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T04:34:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-23T04:34:01.245 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T04:34:01.434 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-23T04:34:01.649 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-23T04:34:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-23T04:34:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-23T04:34:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T04:34:02.212 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-23T04:34:02.304 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-23T04:34:02.511 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-23T04:34:02.516 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-23T04:34:02.703 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-23T04:34:02.802 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-23T04:34:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-23T04:34:02.930 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-23T04:34:03.012 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-23T04:34:03.103 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T04:34:03.204 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T04:34:03.266 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-23T04:34:03.386 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T04:34:03.501 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-23T04:34:03.548 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T04:34:03.687 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-23T04:34:03.723 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T04:34:03.866 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-23T04:34:03.905 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-23T04:34:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-23T04:34:04.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-23T04:34:04.079 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-23T04:34:04.150 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T04:34:04.218 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T04:34:04.221 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-23T04:34:04.286 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-23T04:34:04.448 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-23T04:34:04.499 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-23T04:34:04.501 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-23T04:34:04.645 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-23T04:34:04.768 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-23T04:34:04.772 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-23T04:34:04.792 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T04:34:04.871 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T04:34:04.952 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 26/134 2023-09-23T04:34:05.021 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-23T04:34:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-23T04:34:05.100 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/134 2023-09-23T04:34:05.220 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-23T04:34:05.266 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-23T04:34:05.351 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 28/134 2023-09-23T04:34:05.357 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-23T04:34:05.433 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-23T04:34:05.527 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-23T04:34:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T04:34:05.577 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-23T04:34:05.686 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-23T04:34:05.750 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-23T04:34:05.818 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-23T04:34:05.837 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-23T04:34:06.031 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-23T04:34:06.036 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-23T04:34:06.222 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-23T04:34:06.279 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-23T04:34:06.417 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-23T04:34:06.488 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-23T04:34:06.571 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-23T04:34:06.713 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-23T04:34:06.860 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-23T04:34:06.868 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-23T04:34:07.008 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T04:34:07.053 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-23T04:34:07.203 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T04:34:07.270 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-23T04:34:07.463 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-23T04:34:07.600 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T04:34:07.767 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-23T04:34:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-23T04:34:07.907 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T04:34:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-23T04:34:08.054 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T04:34:08.159 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-23T04:34:08.186 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T04:34:08.367 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T04:34:08.379 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-23T04:34:08.537 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-23T04:34:08.562 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T04:34:08.728 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-23T04:34:08.731 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T04:34:08.903 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T04:34:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-23T04:34:09.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-23T04:34:09.048 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T04:34:09.253 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-23T04:34:09.263 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T04:34:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-23T04:34:09.454 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-23T04:34:09.479 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T04:34:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-23T04:34:09.676 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-23T04:34:09.681 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T04:34:09.849 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T04:34:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-23T04:34:09.937 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T04:34:10.010 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T04:34:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-23T04:34:10.166 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T04:34:10.274 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T04:34:10.306 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T04:34:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-23T04:34:10.506 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T04:34:10.515 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T04:34:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 36/134 2023-09-23T04:34:10.637 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T04:34:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 37/134 2023-09-23T04:34:10.768 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T04:34:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-23T04:34:10.957 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T04:34:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-23T04:34:11.144 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T04:34:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T04:34:11.321 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T04:34:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-23T04:34:11.477 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T04:34:11.589 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T04:34:11.597 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T04:34:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-23T04:34:11.812 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T04:34:11.837 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T04:34:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-23T04:34:12.003 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T04:34:12.065 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T04:34:12.068 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T04:34:12.163 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T04:34:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-23T04:34:12.340 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T04:34:12.387 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T04:34:12.407 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-23T04:34:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-23T04:34:12.573 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T04:34:12.635 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T04:34:12.731 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T04:34:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-23T04:34:12.822 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T04:34:12.939 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T04:34:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-23T04:34:13.138 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-23T04:34:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-23T04:34:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-23T04:34:13.625 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T04:34:13.770 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-23T04:34:13.799 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:14.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T04:34:14.083 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T04:34:14.213 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-23T04:34:14.286 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T04:34:14.540 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T04:34:14.560 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-23T04:34:14.777 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-23T04:34:14.824 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T04:34:15.028 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-23T04:34:15.089 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T04:34:15.280 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T04:34:15.288 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-23T04:34:15.653 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-23T04:34:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-23T04:34:16.062 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T04:34:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-23T04:34:16.232 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:16.302 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:16.364 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-23T04:34:16.457 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T04:34:16.560 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-23T04:34:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-23T04:34:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-23T04:34:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-23T04:34:17.804 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-23T04:34:17.817 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:18.047 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-23T04:34:18.049 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:18.228 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-23T04:34:18.478 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-23T04:34:18.636 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:18.777 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-23T04:34:18.936 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-23T04:34:19.017 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:19.174 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:19.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-23T04:34:19.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-23T04:34:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-23T04:34:19.829 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-23T04:34:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-23T04:34:20.170 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-23T04:34:20.348 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:20.384 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-23T04:34:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-23T04:34:20.938 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:20.950 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-23T04:34:21.184 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-23T04:34:21.242 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:21.311 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:21.423 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-23T04:34:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-23T04:34:21.904 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-23T04:34:22.147 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-23T04:34:23.567 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-23T04:34:23.882 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-23T04:34:24.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-23T04:34:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-23T04:34:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-23T04:34:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-23T04:34:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-23T04:34:26.441 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 92/134 2023-09-23T04:34:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 93/134 2023-09-23T04:34:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 94/134 2023-09-23T04:34:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:33.280 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 95/134 2023-09-23T04:34:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:37.972 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /sys 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /proc 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /dev 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /run 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /mnt 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /var/tmp 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /home 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /tmp 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /dev 2023-09-23T04:34:37.973 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:34:38.313 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:39.911 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /run 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2023-09-23T04:34:39.971 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2023-09-23T04:34:39.972 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2023-09-23T04:34:39.972 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /dev 2023-09-23T04:34:39.972 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:34:40.058 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:40.278 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:40.295 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:40.462 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:40.943 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:41.096 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:41.112 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:42.267 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:42.453 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:42.914 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:43.016 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:34:43.053 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:43.185 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:34:43.293 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T04:34:43.469 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T04:34:43.636 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:34:43.857 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:34:44.141 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:34:44.716 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:34:44.908 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:34:45.017 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T04:34:45.098 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:34:45.193 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T04:34:45.327 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:34:45.360 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:34:45.419 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:34:45.580 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:34:45.774 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T04:34:45.846 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:34:45.984 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:34:46.019 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:34:46.161 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:34:46.172 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:34:46.808 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:34:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:34:47.060 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:34:47.388 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T04:34:47.514 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T04:34:47.596 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:34:47.631 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:34:47.676 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:34:47.709 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:34:47.785 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:34:47.797 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:34:47.845 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:34:47.886 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:34:48.053 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:34:48.410 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:34:48.602 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:34:49.117 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T04:34:49.304 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:34:49.337 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:34:49.477 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:34:49.519 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:34:49.677 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:34:50.036 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:34:50.193 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:34:53.205 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:34:53.488 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T04:34:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-23T04:34:53.837 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-23T04:34:54.727 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-23T04:34:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 96/134 2023-09-23T04:34:54.734 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /run 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-09-23T04:34:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:34:54.818 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:34:55.071 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-23T04:34:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:55.254 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 119/134 2023-09-23T04:34:55.430 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-23T04:34:55.597 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T04:34:55.672 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T04:34:55.752 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T04:34:55.978 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-23T04:34:56.170 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 117/134 2023-09-23T04:34:56.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-23T04:34:56.567 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T04:34:56.756 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T04:34:56.980 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-23T04:34:57.019 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 97/134 2023-09-23T04:34:57.146 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T04:34:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T04:34:57.505 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 98/134 2023-09-23T04:34:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:57.919 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:58.176 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:58.193 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 99/134 2023-09-23T04:34:58.351 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T04:34:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:58.503 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 100/134 2023-09-23T04:34:58.642 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T04:34:58.950 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T04:34:59.252 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T04:34:59.376 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T04:34:59.384 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T04:34:59.460 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:59.548 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-23T04:34:59.692 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:59.711 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:34:59.742 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-23T04:34:59.820 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-23T04:34:59.859 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T04:35:00.025 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-23T04:35:00.038 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-23T04:35:00.038 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:00.141 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T04:35:00.245 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:00.245 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:00.438 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T04:35:00.464 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:00.665 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:00.798 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T04:35:00.974 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T04:35:01.054 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T04:35:01.211 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:01.211 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T04:35:01.212 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T04:35:01.213 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T04:35:01.214 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T04:35:01.215 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T04:35:01.216 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T04:35:01.217 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-23T04:35:01.218 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-23T04:35:01.221 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-23T04:35:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-23T04:35:01.223 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-23T04:35:01.224 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:01.225 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T04:35:01.233 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T04:35:01.246 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T04:35:01.400 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T04:35:01.408 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T04:35:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:35:01.535 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-23T04:35:01.535 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el8.noarch 101/134 2023-09-23T04:35:01.719 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:01.719 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:01.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 102/134 2023-09-23T04:35:01.938 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 103/134 2023-09-23T04:35:02.132 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:35:02.414 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 104/134 2023-09-23T04:35:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:35:02.668 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T04:35:02.669 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T04:35:02.670 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T04:35:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T04:35:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T04:35:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-23T04:35:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-23T04:35:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-23T04:35:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-23T04:35:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-23T04:35:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T04:35:03.685 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:03.685 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:01:50 ago on Sat 23 Sep 2023 04:33:13 AM UTC. 2023-09-23T04:35:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 105/134 2023-09-23T04:35:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:35:04.120 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 106/134 2023-09-23T04:35:04.518 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 107/134 2023-09-23T04:35:04.732 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:35:04.800 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 108/134 2023-09-23T04:35:05.002 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:35:05.032 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 109/134 2023-09-23T04:35:05.130 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:05.130 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:53 ago on Sat 23 Sep 2023 04:33:12 AM UTC. 2023-09-23T04:35:06.222 INFO:teuthology.orchestra.run.smithi118.stdout:Installed products updated. 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T04:35:06.435 INFO:teuthology.orchestra.run.smithi118.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.436 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.437 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: jq-1.6-3.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.438 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.439 INFO:teuthology.orchestra.run.smithi118.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T04:35:06.440 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.441 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T04:35:06.442 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T04:35:06.443 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T04:35:06.444 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T04:35:06.445 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T04:35:06.446 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T04:35:06.447 INFO:teuthology.orchestra.run.smithi118.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T04:35:06.447 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:35:06.447 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2023-09-23T04:35:06.570 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el8.x86_64 110/134 2023-09-23T04:35:06.635 DEBUG:teuthology.parallel:result is None 2023-09-23T04:35:06.771 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:35:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 111/134 2023-09-23T04:35:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:35:07.101 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 112/134 2023-09-23T04:35:07.278 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:35:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 113/134 2023-09-23T04:35:07.771 INFO:teuthology.orchestra.run.smithi069.stdout:Installed products updated. 2023-09-23T04:35:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:35:07.983 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:35:07.983 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2023-09-23T04:35:07.983 INFO:teuthology.orchestra.run.smithi069.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T04:35:07.983 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.984 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.985 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: jq-1.6-3.el8.x86_64 2023-09-23T04:35:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T04:35:07.987 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T04:35:07.988 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T04:35:07.989 INFO:teuthology.orchestra.run.smithi069.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T04:35:07.990 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:07.991 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T04:35:07.992 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T04:35:07.993 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T04:35:07.994 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T04:35:07.995 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T04:35:07.995 INFO:teuthology.orchestra.run.smithi069.stdout: 2023-09-23T04:35:07.995 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2023-09-23T04:35:08.185 DEBUG:teuthology.parallel:result is None 2023-09-23T04:35:13.349 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 114/134 2023-09-23T04:35:14.662 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 115/134 2023-09-23T04:35:15.024 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-23T04:35:15.208 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 117/134 2023-09-23T04:35:15.494 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-23T04:35:15.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T04:35:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-23T04:35:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-23T04:35:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 121/134 2023-09-23T04:35:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-23T04:35:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:35:20.339 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:35:20.377 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-23T04:35:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 124/134 2023-09-23T04:35:20.778 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 125/134 2023-09-23T04:35:21.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 126/134 2023-09-23T04:35:21.544 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-23T04:35:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T04:35:21.842 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-23T04:35:22.050 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T04:35:22.078 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-23T04:35:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T04:35:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-23T04:35:22.495 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-23T04:35:22.496 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:22.723 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:22.724 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:23.010 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-23T04:35:23.208 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:23.487 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:23.487 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-23T04:35:23.487 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el8.x86_64 1/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 3/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 4/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 6/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 7/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 8/134 2023-09-23T04:35:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 9/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 10/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 11/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 12/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 13/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_ 14/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 15/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 16/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el8.x86_6 17/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el8.x86 18/134 2023-09-23T04:35:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el8.x86_64 19/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el8. 21/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_6 22/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 23/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 24/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 25/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 26/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 27/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 28/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/134 2023-09-23T04:35:24.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noa 30/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.n 31/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 34/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 36/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el8.noarch 37/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 38/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-23T04:35:24.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-23T04:35:24.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-23T04:35:24.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-23T04:35:24.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-23T04:35:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-23T04:35:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-23T04:35:24.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-23T04:35:24.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el8.x86_64 129/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el8.x86_64 131/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-23T04:35:24.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-23T04:35:26.850 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-23T04:35:26.850 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:02:10 ago on Sat 23 Sep 2023 04:33:16 AM UTC. 2023-09-23T04:35:29.577 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2023-09-23T04:35:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:35:29.818 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2023-09-23T04:35:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-23T04:35:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.1-7.el8.noarch 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.0.4-10.el8.x86_64 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:18.0.0-6336.g86266391.el8.noarch 2023-09-23T04:35:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.1-9.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: jq-1.6-3.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-23T04:35:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-23T04:35:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-23T04:35:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-23T04:35:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-23T04:35:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: libpq-13.5-1.el8.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-23T04:35:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-23T04:35:29.828 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-23T04:35:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-23T04:35:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-23T04:35:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-23T04:35:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-23T04:35:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-2.6-1.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-23T04:35:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el8.x86_64 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-23T04:35:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:35:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-09-23T04:35:30.160 DEBUG:teuthology.parallel:result is None 2023-09-23T04:35:30.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:30.430 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T04:35:30.486 INFO:teuthology.orchestra.run.smithi005.stdout:18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.486 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.487 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T04:35:30.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:30.733 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T04:35:30.761 INFO:teuthology.orchestra.run.smithi069.stdout:18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.761 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.761 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T04:35:30.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:30.909 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T04:35:30.939 INFO:teuthology.orchestra.run.smithi118.stdout:18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.939 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el8 2023-09-23T04:35:30.939 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T04:35:30.939 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-23T04:35:30.940 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:35:30.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T04:35:31.032 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:35:31.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T04:35:31.079 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:35:31.079 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T04:35:31.127 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-23T04:35:31.127 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:35:31.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T04:35:31.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T04:35:31.309 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:35:31.310 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T04:35:31.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T04:35:31.412 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:35:31.413 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T04:35:31.445 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T04:35:31.517 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-23T04:35:31.518 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:35:31.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T04:35:31.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T04:35:31.700 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:35:31.700 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T04:35:31.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T04:35:31.802 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:35:31.803 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T04:35:31.835 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T04:35:31.908 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-23T04:35:31.908 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:35:31.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T04:35:31.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T04:35:32.091 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:35:32.091 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T04:35:32.123 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T04:35:32.192 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:35:32.193 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T04:35:32.224 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T04:35:32.296 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-23T04:35:32.372 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-23T04:35:32.372 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:32.372 INFO:tasks.cephadm:Cluster fsid is a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:35:32.372 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-23T04:35:32.373 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.69', 'mon.c': '172.21.15.118'} 2023-09-23T04:35:32.373 INFO:tasks.cephadm:First mon is mon.a on smithi005 2023-09-23T04:35:32.373 INFO:tasks.cephadm:First mgr is x 2023-09-23T04:35:32.373 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-23T04:35:32.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2023-09-23T04:35:32.444 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2023-09-23T04:35:32.477 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname $(hostname -s) 2023-09-23T04:35:32.512 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-23T04:35:32.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:32.761 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72754/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6336-g86266391', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6336.g86266391'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-22 19:32:56.769053', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20230922.144532', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72751/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6336-g86266391', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6336.g86266391'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-22 16:40:43.037820', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20230922.144532', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-23T04:35:32.907 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20230922.144532, sha1 862663913a74a99e1c52d50e6a7e59dd6705749a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:35:32.907 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm 2023-09-23T04:35:32.907 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:33.590 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 04:35 /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:33.591 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:34.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 04:35 /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:34.458 DEBUG:teuthology.orchestra.run.smithi118:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:35.281 INFO:teuthology.orchestra.run.smithi118.stdout:-rw-rw-r--. 1 ubuntu ubuntu 741164 Sep 23 04:35 /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:35.282 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:35.333 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:35.354 DEBUG:teuthology.orchestra.run.smithi118:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-23T04:35:35.463 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a on all hosts... 2023-09-23T04:35:35.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T04:35:35.468 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T04:35:35.470 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a pull 2023-09-23T04:35:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T04:35:36.832 INFO:teuthology.orchestra.run.smithi118.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T04:35:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T04:36:08.019 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout: "repo_digests": [ 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2023-09-23T04:36:08.020 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-09-23T04:36:13.659 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:36:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T04:36:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T04:36:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2023-09-23T04:36:13.659 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T04:36:13.660 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T04:36:13.660 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:36:14.632 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c", 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6882747db0c539a4e5112458cfa4d3819bf677e6f6528ea4b6d97351de2a66fa" 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2023-09-23T04:36:14.633 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-23T04:36:14.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2023-09-23T04:36:14.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2023-09-23T04:36:14.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph 2023-09-23T04:36:14.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2023-09-23T04:36:14.870 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2023-09-23T04:36:14.897 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /etc/ceph 2023-09-23T04:36:14.928 INFO:tasks.cephadm:Writing seed config... 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [client] debug client = 20 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-09-23T04:36:14.930 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-09-23T04:36:14.931 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-23T04:36:14.932 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-23T04:36:14.933 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-09-23T04:36:14.934 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:36:14.934 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-23T04:36:14.977 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a18d5e54-59ca-11ee-8db4-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-09-23T04:36:14.977 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service 2023-09-23T04:36:15.021 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x.service 2023-09-23T04:36:15.066 INFO:tasks.cephadm:Bootstrapping... 2023-09-23T04:36:15.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a -v bootstrap --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-23T04:36:15.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:36:15.135 INFO:journalctl@ceph.mgr.x.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:36:15.301 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-09-23T04:36:15.301 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a', '-v', 'bootstrap', '--fsid', 'a18d5e54-59ca-11ee-8db4-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2023-09-23T04:36:15.474 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-09-23T04:36:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-23T04:36:15.475 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2023-09-23T04:36:15.646 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-09-23T04:36:15.647 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2023-09-23T04:36:15.647 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2023-09-23T04:36:15.661 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-23T04:36:15.661 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-23T04:36:15.674 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-23T04:36:15.674 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:36:15.688 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2023-09-23T04:36:15.702 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2023-09-23T04:36:15.702 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2023-09-23T04:36:15.702 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2023-09-23T04:36:15.873 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-09-23T04:36:15.873 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-23T04:36:15.873 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2023-09-23T04:36:15.873 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2023-09-23T04:36:15.886 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-23T04:36:15.886 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-23T04:36:15.899 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-23T04:36:15.900 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:36:15.913 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2023-09-23T04:36:15.926 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2023-09-23T04:36:15.926 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2023-09-23T04:36:15.926 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2023-09-23T04:36:15.926 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:15.927 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140535435995624 on /run/cephadm/a18d5e54-59ca-11ee-8db4-212e2dc638e7.lock 2023-09-23T04:36:15.927 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140535435995624 acquired on /run/cephadm/a18d5e54-59ca-11ee-8db4-212e2dc638e7.lock 2023-09-23T04:36:15.927 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2023-09-23T04:36:15.928 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2023-09-23T04:36:15.928 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2023-09-23T04:36:15.936 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-23T04:36:15.936 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2023-09-23T04:36:15.943 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-23T04:36:15.943 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-23T04:36:15.943 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2023-09-23T04:36:15.949 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-23T04:36:15.949 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-23T04:36:15.949 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-23T04:36:15.949 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-09-23T04:36:15.950 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2023-09-23T04:36:15.950 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-23T04:36:15.950 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2023-09-23T04:36:15.951 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2023-09-23T04:36:15.951 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-23T04:36:15.952 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-23T04:36:15.953 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T04:36:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a... 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:451a6a10e70d6de11ee942f3f27d2d4d1cdb7f417bfc327107b96bbd8c6f6af1 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:3f4fd5c3c800c6da9ef94691df203a05a5bbbf32c483d09d620ec06eef42ea2b 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:9b6b9c6d0d34b39b370c1621d94f476493fc3d9e4cee418b6a840a66fb1b0b9c 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-23T04:36:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Storing signatures 2023-09-23T04:36:17.302 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T04:36:17.302 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev) 2023-09-23T04:36:17.302 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2023-09-23T04:36:18.076 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2023-09-23T04:36:18.076 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2023-09-23T04:36:18.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBCaw5lo/UyHBAAsf6532aoHV8xFIzxaAaqzA== 2023-09-23T04:36:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBDaw5lc08+EBAA52x9T3kRO342qSNE6t27Cw== 2023-09-23T04:36:20.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBEaw5loDfKABAAKGkh57P8Ygg/sNWY8w9z+w== 2023-09-23T04:36:20.412 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = pacific 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-23T04:36:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:36:22.394 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2023-09-23T04:36:23.169 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2023-09-23T04:36:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-23T04:36:23.630 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-23T04:36:23.885 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target → /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target. 2023-09-23T04:36:23.886 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target → /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target. 2023-09-23T04:36:24.234 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a 2023-09-23T04:36:24.234 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service: Unit ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service not loaded. 2023-09-23T04:36:24.444 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target.wants/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@.service. 2023-09-23T04:36:24.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:24 smithi005 systemd[1]: Starting Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:36:25.206 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:24 smithi005 podman[97093]: 2023-09-23T04:36:25.206 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:25 smithi005 bash[97093]: 92f505b678a3668c8cebf34e2ba5ef3a91e7437a0c986c8689937afc69d67f4e 2023-09-23T04:36:25.260 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T04:36:25.260 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-09-23T04:36:25.276 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T04:36:25.276 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:36:25.276 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-09-23T04:36:25.277 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2023-09-23T04:36:25.277 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2023-09-23T04:36:25.277 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2023-09-23T04:36:25.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:25 smithi005 ceph-mon[97133]: mkfs a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:25.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:25 smithi005 ceph-mon[97133]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T04:36:25.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:25 smithi005 systemd[1]: Started Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:36:26.359 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2023-09-23T04:36:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-23T04:36:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2023-09-23T04:36:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.689985s) 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-23T04:36:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.804+0000 7f46c2f7e700 1 Processor -- start 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.804+0000 7f46c2f7e700 1 -- start start 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.805+0000 7f46c2f7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.805+0000 7f46c2f7e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc07a8b0 con 0x7f46bc07be80 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.805+0000 7f46c0d1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.805+0000 7f46c0d1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37708/0 (socket says 172.21.15.5:37708) 2023-09-23T04:36:26.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.805+0000 7f46c0d1a700 1 -- 172.21.15.5:0/2991937096 learned_addr learned my addr 172.21.15.5:0/2991937096 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.806+0000 7f46c0d1a700 1 -- 172.21.15.5:0/2991937096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc07a9f0 con 0x7f46bc07be80 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.806+0000 7f46c0d1a700 1 --2- 172.21.15.5:0/2991937096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46b00098a0 tx=0x7f46b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9075c79a08c74f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.806+0000 7f46bb7fe700 1 -- 172.21.15.5:0/2991937096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b0004160 con 0x7f46bc07be80 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.806+0000 7f46bb7fe700 1 -- 172.21.15.5:0/2991937096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b00042c0 con 0x7f46bc07be80 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.806+0000 7f46bb7fe700 1 -- 172.21.15.5:0/2991937096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b00045a0 con 0x7f46bc07be80 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 -- 172.21.15.5:0/2991937096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 msgr2=0x7f46bc07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 --2- 172.21.15.5:0/2991937096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46b00098a0 tx=0x7f46b0009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 -- 172.21.15.5:0/2991937096 shutdown_connections 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 --2- 172.21.15.5:0/2991937096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc07a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 -- 172.21.15.5:0/2991937096 >> 172.21.15.5:0/2991937096 conn(0x7f46bc101c40 msgr2=0x7f46bc104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 -- 172.21.15.5:0/2991937096 shutdown_connections 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.807+0000 7f46c2f7e700 1 -- 172.21.15.5:0/2991937096 wait complete. 2023-09-23T04:36:26.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.808+0000 7f46c2f7e700 1 Processor -- start 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.808+0000 7f46c2f7e700 1 -- start start 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c2f7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c2f7e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc19c0e0 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c0d1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c0d1a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37712/0 (socket says 172.21.15.5:37712) 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c0d1a700 1 -- 172.21.15.5:0/260092067 learned_addr learned my addr 172.21.15.5:0/260092067 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.809+0000 7f46c0d1a700 1 -- 172.21.15.5:0/260092067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0009550 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46c0d1a700 1 --2- 172.21.15.5:0/260092067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f46bc07b350 tx=0x7f46b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b0011630 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0011790 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46b0011a30 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc19c2e0 con 0x7f46bc07be80 2023-09-23T04:36:26.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc19c700 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f46b0022070 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.810+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f46b001c450 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.811+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46bc195c40 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.814+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f46b001b450 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.890+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f46bc07ad60 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.891+0000 7f46b9ffb700 1 -- 172.21.15.5:0/260092067 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f46b001b5f0 con 0x7f46bc07be80 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 msgr2=0x7f46bc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 --2- 172.21.15.5:0/260092067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f46bc07b350 tx=0x7f46b0004750 comp rx=0 tx=0).stop 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 shutdown_connections 2023-09-23T04:36:26.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 --2- 172.21.15.5:0/260092067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46bc07be80 0x7f46bc19bba0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:26.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 >> 172.21.15.5:0/260092067 conn(0x7f46bc101c40 msgr2=0x7f46bc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:26.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 shutdown_connections 2023-09-23T04:36:26.375 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:25.892+0000 7f46c2f7e700 1 -- 172.21.15.5:0/260092067 wait complete. 2023-09-23T04:36:26.375 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2023-09-23T04:36:26.375 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2023-09-23T04:36:26.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T04:36:26.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: monmap e1: 1 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0]} removed_ranks: {} 2023-09-23T04:36:26.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: fsmap 2023-09-23T04:36:26.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: osdmap e1: 0 total, 0 up, 0 in 2023-09-23T04:36:26.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: mgrmap e1: no daemons active 2023-09-23T04:36:26.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:26 smithi005 ceph-mon[97133]: from='client.? 172.21.15.5:0/260092067' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-23T04:36:27.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:27 smithi005 ceph-mon[97133]: from='client.? 172.21.15.5:0/2048820741' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-23T04:36:27.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:27 smithi005 ceph-mon[97133]: from='client.? 172.21.15.5:0/2048820741' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-23T04:36:27.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-23T04:36:27.346 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.868+0000 7fc446817700 1 Processor -- start 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.868+0000 7fc446817700 1 -- start start 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.869+0000 7fc446817700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.869+0000 7fc446817700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440107050 con 0x7fc440106660 2023-09-23T04:36:27.347 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.870+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.870+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37720/0 (socket says 172.21.15.5:37720) 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.870+0000 7fc43ffff700 1 -- 172.21.15.5:0/1873288458 learned_addr learned my addr 172.21.15.5:0/1873288458 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.870+0000 7fc43ffff700 1 -- 172.21.15.5:0/1873288458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440107860 con 0x7fc440106660 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.870+0000 7fc43ffff700 1 --2- 172.21.15.5:0/1873288458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc4300098a0 tx=0x7fc430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96daf87ba875ae21 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc43effd700 1 -- 172.21.15.5:0/1873288458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc430004160 con 0x7fc440106660 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc43effd700 1 -- 172.21.15.5:0/1873288458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300042c0 con 0x7fc440106660 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc43effd700 1 -- 172.21.15.5:0/1873288458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4300045a0 con 0x7fc440106660 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 -- 172.21.15.5:0/1873288458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 msgr2=0x7fc440106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 --2- 172.21.15.5:0/1873288458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc4300098a0 tx=0x7fc430009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 -- 172.21.15.5:0/1873288458 shutdown_connections 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 --2- 172.21.15.5:0/1873288458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc440106a80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:27.348 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 -- 172.21.15.5:0/1873288458 >> 172.21.15.5:0/1873288458 conn(0x7fc440101c40 msgr2=0x7fc440104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 -- 172.21.15.5:0/1873288458 shutdown_connections 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.871+0000 7fc446817700 1 -- 172.21.15.5:0/1873288458 wait complete. 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.873+0000 7fc446817700 1 Processor -- start 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.873+0000 7fc446817700 1 -- start start 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc446817700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc446817700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44019c150 con 0x7fc440106660 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:27.349 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37726/0 (socket says 172.21.15.5:37726) 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc43ffff700 1 -- 172.21.15.5:0/2048820741 learned_addr learned my addr 172.21.15.5:0/2048820741 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc43ffff700 1 -- 172.21.15.5:0/2048820741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430009550 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.874+0000 7fc43ffff700 1 --2- 172.21.15.5:0/2048820741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc430004000 tx=0x7fc430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc430011630 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc430011790 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44019c350 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc430011a30 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44019c770 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc430022070 con 0x7fc440106660 2023-09-23T04:36:27.350 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.875+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc43001c450 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.876+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc440062450 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.879+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc43001b450 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.952+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc440107bd0 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.956+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fc43001b5f0 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.957+0000 7fc43d7fa700 1 -- 172.21.15.5:0/2048820741 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc43001b790 con 0x7fc440106660 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.957+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 msgr2=0x7fc44019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.957+0000 7fc446817700 1 --2- 172.21.15.5:0/2048820741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc430004000 tx=0x7fc430004750 comp rx=0 tx=0).stop 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.958+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 shutdown_connections 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.958+0000 7fc446817700 1 --2- 172.21.15.5:0/2048820741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc440106660 0x7fc44019bc10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.958+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 >> 172.21.15.5:0/2048820741 conn(0x7fc440101c40 msgr2=0x7fc44007a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:27.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.958+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 shutdown_connections 2023-09-23T04:36:27.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:26.958+0000 7fc446817700 1 -- 172.21.15.5:0/2048820741 wait complete. 2023-09-23T04:36:27.352 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2023-09-23T04:36:28.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:28 smithi005 ceph-mon[97133]: from='client.? 172.21.15.5:0/3004966217' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.886+0000 7f2a2a4ee700 1 Processor -- start 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.886+0000 7f2a2a4ee700 1 -- start start 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a2a4ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a2a4ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24107050 con 0x7f2a24106660 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37736/0 (socket says 172.21.15.5:37736) 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a23fff700 1 -- 172.21.15.5:0/2216241609 learned_addr learned my addr 172.21.15.5:0/2216241609 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:28.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.887+0000 7f2a23fff700 1 -- 172.21.15.5:0/2216241609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24107860 con 0x7f2a24106660 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a23fff700 1 --2- 172.21.15.5:0/2216241609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a140098a0 tx=0x7f2a14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906c7d13435f81d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a22ffd700 1 -- 172.21.15.5:0/2216241609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a14004160 con 0x7f2a24106660 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a22ffd700 1 -- 172.21.15.5:0/2216241609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a140042c0 con 0x7f2a24106660 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a22ffd700 1 -- 172.21.15.5:0/2216241609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a14011a00 con 0x7f2a24106660 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/2216241609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 msgr2=0x7f2a24106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.888+0000 7f2a2a4ee700 1 --2- 172.21.15.5:0/2216241609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a140098a0 tx=0x7f2a14009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.889+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/2216241609 shutdown_connections 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.889+0000 7f2a2a4ee700 1 --2- 172.21.15.5:0/2216241609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a24106a80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.889+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/2216241609 >> 172.21.15.5:0/2216241609 conn(0x7f2a24101c40 msgr2=0x7f2a24104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.889+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/2216241609 shutdown_connections 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.889+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/2216241609 wait complete. 2023-09-23T04:36:28.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a2a4ee700 1 Processor -- start 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a2a4ee700 1 -- start start 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a2a4ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a2a4ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2419c940 con 0x7f2a24106660 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37748/0 (socket says 172.21.15.5:37748) 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.890+0000 7f2a23fff700 1 -- 172.21.15.5:0/3004966217 learned_addr learned my addr 172.21.15.5:0/3004966217 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a23fff700 1 -- 172.21.15.5:0/3004966217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14009550 con 0x7f2a24106660 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a23fff700 1 --2- 172.21.15.5:0/3004966217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009fb0 tx=0x7f2a14011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a14003930 con 0x7f2a24106660 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a14003a90 con 0x7f2a24106660 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2419cb40 con 0x7f2a24106660 2023-09-23T04:36:28.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a140195e0 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.891+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2419cf60 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.892+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2a14019b70 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.892+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2a140243f0 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.892+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a24062450 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.895+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2a1402de30 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.968+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2a2410aa60 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.969+0000 7f2a217fa700 1 -- 172.21.15.5:0/3004966217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f2a14003c00 con 0x7f2a24106660 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.969+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 msgr2=0x7f2a2419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.969+0000 7f2a2a4ee700 1 --2- 172.21.15.5:0/3004966217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009fb0 tx=0x7f2a14011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.970+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 shutdown_connections 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.970+0000 7f2a2a4ee700 1 --2- 172.21.15.5:0/3004966217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a24106660 0x7f2a2419c400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:28.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.970+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 >> 172.21.15.5:0/3004966217 conn(0x7f2a24101c40 msgr2=0x7f2a241028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:28.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.970+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 shutdown_connections 2023-09-23T04:36:28.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:27.970+0000 7f2a2a4ee700 1 -- 172.21.15.5:0/3004966217 wait complete. 2023-09-23T04:36:28.426 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2023-09-23T04:36:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:28 smithi005 systemd[1]: Stopping Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:36:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:28 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a[97109]: 2023-09-23T04:36:28.879+0000 7f98b3c5f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T04:36:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:28 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a[97109]: 2023-09-23T04:36:28.879+0000 7f98b3c5f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-23T04:36:29.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:29 smithi005 bash[97472]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a 2023-09-23T04:36:29.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:29 smithi005 systemd[1]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service: Succeeded. 2023-09-23T04:36:29.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:29 smithi005 systemd[1]: Stopped Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:36:29.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:29 smithi005 systemd[1]: Starting Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:36:30.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 podman[97598]: 2023-09-23T04:36:30.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T04:36:30.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T04:36:30.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: pidfile_write: ignore empty --pid-file 2023-09-23T04:36:30.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: load: jerasure load: lrc 2023-09-23T04:36:30.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: RocksDB version: 7.9.2 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Git sha 0 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: DB SUMMARY 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: DB Session ID: J0V3H9VSR1GIGIRRTBVE 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: CURRENT file: CURRENT 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T04:36:30.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98749 ; 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.error_if_exists: 0 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.create_if_missing: 0 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.paranoid_checks: 1 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T04:36:30.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.env: 0x5631195b9da0 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.info_log: 0x56311b03f5a0 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.statistics: (nil) 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.use_fsync: 0 2023-09-23T04:36:30.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_log_file_size: 0 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_fallocate: 1 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T04:36:30.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.use_direct_reads: 0 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.db_log_dir: 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.wal_dir: 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T04:36:30.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.write_buffer_manager: 0x56311aff74a0 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T04:36:30.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.unordered_write: 0 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T04:36:30.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.row_cache: None 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.wal_filter: None 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.two_write_queues: 0 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T04:36:30.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.wal_compression: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.atomic_flush: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.log_readahead_size: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T04:36:30.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_background_jobs: 2 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_background_compactions: -1 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_subcompactions: 1 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T04:36:30.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_open_files: -1 2023-09-23T04:36:30.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_background_flushes: -1 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Compression algorithms supported: 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T04:36:30.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kZSTD supported: 0 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kXpressCompression supported: 0 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kZlibCompression supported: 1 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kSnappyCompression supported: 1 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kLZ4Compression supported: 1 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: kBZip2Compression supported: 0 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T04:36:30.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.merge_operator: 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_filter: None 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T04:36:30.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56311b03f6a0) 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2023-09-23T04:36:30.397 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x56311b0951f0 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2023-09-23T04:36:30.398 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2023-09-23T04:36:30.399 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2023-09-23T04:36:30.400 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2023-09-23T04:36:30.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2023-09-23T04:36:30.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2023-09-23T04:36:30.402 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T04:36:30.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T04:36:30.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression: NoCompression 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.num_levels: 7 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T04:36:30.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T04:36:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:36:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:36:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T04:36:30.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T04:36:30.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T04:36:30.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.table_properties_collectors: 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.inplace_update_support: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.bloom_locality: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.max_successive_merges: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T04:36:30.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.ttl: 2592000 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enable_blob_files: false 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.min_blob_size: 0 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 bash[97598]: a2c7341a82a44ca13e29430c900f1189110409f0769742ccba9e393fcd7d97f1 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T04:36:30.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 93a6fea7-14a5-4004-b220-29db5928b97b 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443790319606, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443790321563, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93990, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91995, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84560, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695443790, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "93a6fea7-14a5-4004-b220-29db5928b97b", "db_session_id": "J0V3H9VSR1GIGIRRTBVE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443790321728, "job": 1, "event": "recovery_finished"} 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T04:36:30.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56311b132000 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: DB pointer 0x56311b11e000 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:36:30.412 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 93.62 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 93.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:36:30.413 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 13.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 13.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T04:36:30.414 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x56311b0951f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???) e1 preinit fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).mds e1 new map 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).mds e1 print_map 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2023-09-23T04:36:30.415 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expand map: {default=false} 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta from 'false' to 'false' 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expanded map: {default=false} 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expand map: {default=info} 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta from 'info' to 'info' 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expanded map: {default=info} 2023-09-23T04:36:30.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expand map: {default=daemon} 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expand map: {default=debug} 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: expand_channel_meta expanded map: {default=debug} 2023-09-23T04:36:30.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 systemd[1]: Started Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:36:30.417 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-23T04:36:30.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-23T04:36:30.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: monmap e1: 1 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0]} removed_ranks: {} 2023-09-23T04:36:30.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: fsmap 2023-09-23T04:36:30.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: osdmap e1: 0 total, 0 up, 0 in 2023-09-23T04:36:30.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:30 smithi005 ceph-mon[97638]: mgrmap e1: no daemons active 2023-09-23T04:36:31.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.891+0000 7f8ec290e700 1 Processor -- start 2023-09-23T04:36:31.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.891+0000 7f8ec290e700 1 -- start start 2023-09-23T04:36:31.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.891+0000 7f8ec290e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:31.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.891+0000 7f8ec290e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc107050 con 0x7f8ebc106660 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.892+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.892+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42706/0 (socket says 172.21.15.5:42706) 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.892+0000 7f8ebbfff700 1 -- 172.21.15.5:0/3633046519 learned_addr learned my addr 172.21.15.5:0/3633046519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.892+0000 7f8ebbfff700 1 -- 172.21.15.5:0/3633046519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc107860 con 0x7f8ebc106660 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ebbfff700 1 --2- 172.21.15.5:0/3633046519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0098a0 tx=0x7f8eac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9a05612947da52 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ebaffd700 1 -- 172.21.15.5:0/3633046519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eac004160 con 0x7f8ebc106660 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ebaffd700 1 -- 172.21.15.5:0/3633046519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8eac0042c0 con 0x7f8ebc106660 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ebaffd700 1 -- 172.21.15.5:0/3633046519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eac011a00 con 0x7f8ebc106660 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ec290e700 1 -- 172.21.15.5:0/3633046519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 msgr2=0x7f8ebc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.893+0000 7f8ec290e700 1 --2- 172.21.15.5:0/3633046519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0098a0 tx=0x7f8eac009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.894+0000 7f8ec290e700 1 -- 172.21.15.5:0/3633046519 shutdown_connections 2023-09-23T04:36:31.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.894+0000 7f8ec290e700 1 --2- 172.21.15.5:0/3633046519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc106a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.894+0000 7f8ec290e700 1 -- 172.21.15.5:0/3633046519 >> 172.21.15.5:0/3633046519 conn(0x7f8ebc101c40 msgr2=0x7f8ebc104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.894+0000 7f8ec290e700 1 -- 172.21.15.5:0/3633046519 shutdown_connections 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.894+0000 7f8ec290e700 1 -- 172.21.15.5:0/3633046519 wait complete. 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.895+0000 7f8ec290e700 1 Processor -- start 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ec290e700 1 -- start start 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ec290e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ec290e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc07c190 con 0x7f8ebc106660 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ebbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42720/0 (socket says 172.21.15.5:42720) 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.896+0000 7f8ebbfff700 1 -- 172.21.15.5:0/3692010254 learned_addr learned my addr 172.21.15.5:0/3692010254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8ebbfff700 1 -- 172.21.15.5:0/3692010254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac009550 con 0x7f8ebc106660 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8ebbfff700 1 --2- 172.21.15.5:0/3692010254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006af0 tx=0x7f8eac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:31.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eac003a40 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8eac003ba0 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eac01a700 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc07a2a0 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.897+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc07a6c0 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.898+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8eac01ac30 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.898+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8eac019700 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.898+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc062450 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.901+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f8eac024ad0 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.974+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8ebc10aa60 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.976+0000 7f8eb97fa700 1 -- 172.21.15.5:0/3692010254 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8eac024c70 con 0x7f8ebc106660 2023-09-23T04:36:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.977+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 msgr2=0x7f8ebc07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.977+0000 7f8ec290e700 1 --2- 172.21.15.5:0/3692010254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006af0 tx=0x7f8eac011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.977+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 shutdown_connections 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.977+0000 7f8ec290e700 1 --2- 172.21.15.5:0/3692010254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ebc106660 0x7f8ebc07bc50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.977+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 >> 172.21.15.5:0/3692010254 conn(0x7f8ebc101c40 msgr2=0x7f8ebc1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.978+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 shutdown_connections 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:30.978+0000 7f8ec290e700 1 -- 172.21.15.5:0/3692010254 wait complete. 2023-09-23T04:36:31.474 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-23T04:36:31.495 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-23T04:36:31.495 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2023-09-23T04:36:31.496 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-23T04:36:31.496 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-23T04:36:31.901 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x 2023-09-23T04:36:31.901 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x.service: Unit ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x.service not loaded. 2023-09-23T04:36:32.116 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7.target.wants/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@.service. 2023-09-23T04:36:32.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:31 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3692010254' entity='client.admin' 2023-09-23T04:36:32.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:32 smithi005 systemd[1]: Starting Ceph mgr.x for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:36:32.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:32 smithi005 podman[97872]: 2023-09-23T04:36:32.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:32 smithi005 bash[97872]: 7b302ee8c293703d76f48ce21a80e1c161abe6ca069b6ebd54396d467885847e 2023-09-23T04:36:32.941 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T04:36:32.941 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-09-23T04:36:32.960 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T04:36:32.960 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:36:32.960 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-09-23T04:36:32.960 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2023-09-23T04:36:32.980 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T04:36:32.980 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2023-09-23T04:36:32.997 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2023-09-23T04:36:33.266 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:32 smithi005 systemd[1]: Started Ceph mgr.x for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:36:33.266 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:33.031+0000 7fcf937b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:36:33.266 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:33.144+0000 7fcf937b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:36:33.266 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:33.265+0000 7fcf937b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:36:33.647 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:33.396+0000 7fcf937b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:36:33.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:33.646+0000 7fcf937b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:36:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:33 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1647557908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T04:36:34.083 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:34.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:36:34.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:36:34.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T04:36:34.085 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T04:36:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T04:36:34.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T04:36:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T04:36:25.204168+0000", 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.581+0000 7ffbad569700 1 Processor -- start 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.581+0000 7ffbad569700 1 -- start start 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.581+0000 7ffbad569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.581+0000 7ffbad569700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba807a8b0 con 0x7ffba807be80 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.582+0000 7ffba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.582+0000 7ffba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42750/0 (socket says 172.21.15.5:42750) 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.582+0000 7ffba6ffd700 1 -- 172.21.15.5:0/3978090834 learned_addr learned my addr 172.21.15.5:0/3978090834 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.582+0000 7ffba6ffd700 1 -- 172.21.15.5:0/3978090834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba807a9f0 con 0x7ffba807be80 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.583+0000 7ffba6ffd700 1 --2- 172.21.15.5:0/3978090834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098a0 tx=0x7ffb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669e06f0873cc486 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:34.098 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.583+0000 7ffba5ffb700 1 -- 172.21.15.5:0/3978090834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb98004160 con 0x7ffba807be80 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.583+0000 7ffba5ffb700 1 -- 172.21.15.5:0/3978090834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb980042c0 con 0x7ffba807be80 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.583+0000 7ffba5ffb700 1 -- 172.21.15.5:0/3978090834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb98011a00 con 0x7ffba807be80 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 -- 172.21.15.5:0/3978090834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 msgr2=0x7ffba807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 --2- 172.21.15.5:0/3978090834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098a0 tx=0x7ffb98009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 -- 172.21.15.5:0/3978090834 shutdown_connections 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 --2- 172.21.15.5:0/3978090834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba807a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 -- 172.21.15.5:0/3978090834 >> 172.21.15.5:0/3978090834 conn(0x7ffba8101c40 msgr2=0x7ffba8104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 -- 172.21.15.5:0/3978090834 shutdown_connections 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.584+0000 7ffbad569700 1 -- 172.21.15.5:0/3978090834 wait complete. 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.585+0000 7ffbad569700 1 Processor -- start 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffbad569700 1 -- start start 2023-09-23T04:36:34.099 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffbad569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffbad569700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba819c8d0 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42756/0 (socket says 172.21.15.5:42756) 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.586+0000 7ffba6ffd700 1 -- 172.21.15.5:0/1647557908 learned_addr learned my addr 172.21.15.5:0/1647557908 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffba6ffd700 1 -- 172.21.15.5:0/1647557908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb98009550 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffba6ffd700 1 --2- 172.21.15.5:0/1647557908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006af0 tx=0x7ffb98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb98003a40 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb98003ba0 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb9801a6e0 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba819cad0 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.587+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba819cef0 con 0x7ffba807be80 2023-09-23T04:36:34.100 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.588+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffb98024430 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.588+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb9802d730 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.589+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba8062450 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.592+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7ffb98024870 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.679+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffba810cc70 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.680+0000 7ffb8ffff700 1 -- 172.21.15.5:0/1647557908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ffb9801a840 con 0x7ffba807be80 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 msgr2=0x7ffba819c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 --2- 172.21.15.5:0/1647557908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb98006af0 tx=0x7ffb98011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 shutdown_connections 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 --2- 172.21.15.5:0/1647557908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba807be80 0x7ffba819c390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 >> 172.21.15.5:0/1647557908 conn(0x7ffba8101c40 msgr2=0x7ffba81028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 shutdown_connections 2023-09-23T04:36:34.101 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:33.681+0000 7ffbad569700 1 -- 172.21.15.5:0/1647557908 wait complete. 2023-09-23T04:36:34.102 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2023-09-23T04:36:34.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:34.145+0000 7fcf937b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:36:34.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:34.313+0000 7fcf937b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:36:34.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:34.432+0000 7fcf937b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:36:34.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:34.560+0000 7fcf937b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:36:35.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:35.122+0000 7fcf937b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:36:36.340 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:36.051+0000 7fcf937b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:36:36.340 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:36.164+0000 7fcf937b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:36:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2862992465' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T04:36:37.572 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:37.206+0000 7fcf937b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:36:37.573 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:37.337+0000 7fcf937b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:36:37.573 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:37.572+0000 7fcf937b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:36:38.246 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:37.955+0000 7fcf937b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:36:38.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:38.245+0000 7fcf937b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:36:38.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:38.451+0000 7fcf937b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:36:39.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:38.809+0000 7fcf937b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:36:39.176 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-09-23T04:36:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T04:36:39.179 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T04:36:39.179 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T04:36:39.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T04:36:39.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T04:36:25.204168+0000", 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:39.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.611+0000 7fbc880a5700 1 Processor -- start 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.611+0000 7fbc880a5700 1 -- start start 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.611+0000 7fbc880a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.611+0000 7fbc880a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc80109240 con 0x7fbc80108850 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc85e41700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc85e41700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42760/0 (socket says 172.21.15.5:42760) 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc85e41700 1 -- 172.21.15.5:0/558355200 learned_addr learned my addr 172.21.15.5:0/558355200 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc85e41700 1 -- 172.21.15.5:0/558355200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80109a50 con 0x7fbc80108850 2023-09-23T04:36:39.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc85e41700 1 --2- 172.21.15.5:0/558355200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098a0 tx=0x7fbc7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedad2a9a510b41a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc84e3f700 1 -- 172.21.15.5:0/558355200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c004160 con 0x7fbc80108850 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc84e3f700 1 -- 172.21.15.5:0/558355200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc7c0042c0 con 0x7fbc80108850 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.612+0000 7fbc84e3f700 1 -- 172.21.15.5:0/558355200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c011a00 con 0x7fbc80108850 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.613+0000 7fbc880a5700 1 -- 172.21.15.5:0/558355200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 msgr2=0x7fbc80108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.613+0000 7fbc880a5700 1 --2- 172.21.15.5:0/558355200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098a0 tx=0x7fbc7c009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.613+0000 7fbc880a5700 1 -- 172.21.15.5:0/558355200 shutdown_connections 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.613+0000 7fbc880a5700 1 --2- 172.21.15.5:0/558355200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc80108c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.613+0000 7fbc880a5700 1 -- 172.21.15.5:0/558355200 >> 172.21.15.5:0/558355200 conn(0x7fbc8007bd80 msgr2=0x7fbc801064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.614+0000 7fbc880a5700 1 -- 172.21.15.5:0/558355200 shutdown_connections 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.614+0000 7fbc880a5700 1 -- 172.21.15.5:0/558355200 wait complete. 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc880a5700 1 Processor -- start 2023-09-23T04:36:39.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc880a5700 1 -- start start 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc880a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc880a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8019c940 con 0x7fbc80108850 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc85e41700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc85e41700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42770/0 (socket says 172.21.15.5:42770) 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.615+0000 7fbc85e41700 1 -- 172.21.15.5:0/2862992465 learned_addr learned my addr 172.21.15.5:0/2862992465 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc85e41700 1 -- 172.21.15.5:0/2862992465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c009550 con 0x7fbc80108850 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc85e41700 1 --2- 172.21.15.5:0/2862992465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c006af0 tx=0x7fbc7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c003a40 con 0x7fbc80108850 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc7c003ba0 con 0x7fbc80108850 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c01a700 con 0x7fbc80108850 2023-09-23T04:36:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.616+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8019cb40 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.617+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8019cf60 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.617+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbc7c01ac80 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.617+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbc7c019700 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.618+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc80062450 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.620+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fbc7c024ad0 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.697+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbc8010cc80 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.698+0000 7fbc76ffd700 1 -- 172.21.15.5:0/2862992465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbc7c024c70 con 0x7fbc80108850 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 msgr2=0x7fbc8019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 --2- 172.21.15.5:0/2862992465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c006af0 tx=0x7fbc7c011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 shutdown_connections 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 --2- 172.21.15.5:0/2862992465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc80108850 0x7fbc8019c400 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 >> 172.21.15.5:0/2862992465 conn(0x7fbc8007bd80 msgr2=0x7fbc80105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 shutdown_connections 2023-09-23T04:36:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:36.699+0000 7fbc880a5700 1 -- 172.21.15.5:0/2862992465 wait complete. 2023-09-23T04:36:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2023-09-23T04:36:40.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:39 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:39.809+0000 7fcf937b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:36:40.155 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:40.003+0000 7fcf937b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:36:40.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:40.272+0000 7fcf937b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:36:40.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:40.431+0000 7fcf937b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:36:41.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:40.725+0000 7fcf937b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:41.882+0000 7fcf937b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: Activating manager daemon x 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: mgrmap e2: x(active, starting, since 0.00411529s) 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:36:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:41 smithi005 ceph-mon[97638]: Manager daemon x is now available 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: from='mgr.14100 172.21.15.5:0/710175946' entity='mgr.x' 2023-09-23T04:36:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:42 smithi005 ceph-mon[97638]: mgrmap e3: x(active, since 1.01011s) 2023-09-23T04:36:45.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:44 smithi005 ceph-mon[97638]: mgrmap e4: x(active, since 2s) 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-23T04:36:45.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-09-23T04:36:45.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-23T04:36:45.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-23T04:36:45.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-23T04:36:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-23T04:36:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-09-23T04:36:25.204168+0000", 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.962+0000 7f4d6e969700 1 Processor -- start 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.962+0000 7f4d6e969700 1 -- start start 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.962+0000 7f4d6e969700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.962+0000 7f4d6e969700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6807a8b0 con 0x7f4d6807be80 2023-09-23T04:36:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.963+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.963+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52160/0 (socket says 172.21.15.5:52160) 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.963+0000 7f4d67fff700 1 -- 172.21.15.5:0/49533089 learned_addr learned my addr 172.21.15.5:0/49533089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.963+0000 7f4d67fff700 1 -- 172.21.15.5:0/49533089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6807a9f0 con 0x7f4d6807be80 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.963+0000 7f4d67fff700 1 --2- 172.21.15.5:0/49533089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098a0 tx=0x7f4d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80063380ae27f382 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d66ffd700 1 -- 172.21.15.5:0/49533089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d58004160 con 0x7f4d6807be80 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d66ffd700 1 -- 172.21.15.5:0/49533089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d580042c0 con 0x7f4d6807be80 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d66ffd700 1 -- 172.21.15.5:0/49533089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d58011a00 con 0x7f4d6807be80 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 -- 172.21.15.5:0/49533089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 msgr2=0x7f4d6807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 --2- 172.21.15.5:0/49533089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098a0 tx=0x7f4d58009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 -- 172.21.15.5:0/49533089 shutdown_connections 2023-09-23T04:36:45.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 --2- 172.21.15.5:0/49533089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6807a2e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 -- 172.21.15.5:0/49533089 >> 172.21.15.5:0/49533089 conn(0x7f4d68101c40 msgr2=0x7f4d68104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.964+0000 7f4d6e969700 1 -- 172.21.15.5:0/49533089 shutdown_connections 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.965+0000 7f4d6e969700 1 -- 172.21.15.5:0/49533089 wait complete. 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d6e969700 1 Processor -- start 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d6e969700 1 -- start start 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d6e969700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d6e969700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6819c8d0 con 0x7f4d6807be80 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52172/0 (socket says 172.21.15.5:52172) 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.966+0000 7f4d67fff700 1 -- 172.21.15.5:0/872617732 learned_addr learned my addr 172.21.15.5:0/872617732 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:45.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d67fff700 1 -- 172.21.15.5:0/872617732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d58009550 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d67fff700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006af0 tx=0x7f4d58011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d58003a40 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d58003ba0 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6819cad0 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d5801a6e0 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.967+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6819cef0 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.968+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68062450 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.969+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f4d58024a60 con 0x7f4d6807be80 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.969+0000 7f4d657fa700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 0x7f4d5003ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:45.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.969+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4d5804c860 con 0x7f4d6807be80 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.969+0000 7f4d677fe700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 0x7f4d5003ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.970+0000 7f4d677fe700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 0x7f4d5003ae00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c006fd0 tx=0x7f4d5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:44.975+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4d58024430 con 0x7f4d6807be80 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.265+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4d6819fc50 con 0x7f4d6807be80 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.265+0000 7f4d657fa700 1 -- 172.21.15.5:0/872617732 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f4d580474f0 con 0x7f4d6807be80 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 msgr2=0x7f4d5003ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 0x7f4d5003ae00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c006fd0 tx=0x7f4d5c006e40 comp rx=0 tx=0).stop 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 msgr2=0x7f4d6819c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006af0 tx=0x7f4d58011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:45.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 shutdown_connections 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f4d50038940 0x7f4d5003ae00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 --2- 172.21.15.5:0/872617732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6807be80 0x7f4d6819c390 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.267+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 >> 172.21.15.5:0/872617732 conn(0x7f4d68101c40 msgr2=0x7f4d681028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.268+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 shutdown_connections 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:45.268+0000 7f4d6e969700 1 -- 172.21.15.5:0/872617732 wait complete. 2023-09-23T04:36:45.703 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2023-09-23T04:36:45.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:45 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/872617732' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-23T04:36:46.873 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:46.873 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2023-09-23T04:36:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7feaa58ae700 1 Processor -- start 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7feaa58ae700 1 -- start start 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7feaa58ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7feaa58ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa0079e50 con 0x7feaa0107120 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7fea9effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7fea9effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52174/0 (socket says 172.21.15.5:52174) 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.217+0000 7fea9effd700 1 -- 172.21.15.5:0/2428638987 learned_addr learned my addr 172.21.15.5:0/2428638987 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.218+0000 7fea9effd700 1 -- 172.21.15.5:0/2428638987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0109a50 con 0x7feaa0107120 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.218+0000 7fea9effd700 1 --2- 172.21.15.5:0/2428638987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea900098a0 tx=0x7fea90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d1ed65712aed71 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.218+0000 7fea9dffb700 1 -- 172.21.15.5:0/2428638987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea90004160 con 0x7feaa0107120 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.218+0000 7fea9dffb700 1 -- 172.21.15.5:0/2428638987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea900042c0 con 0x7feaa0107120 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.218+0000 7fea9dffb700 1 -- 172.21.15.5:0/2428638987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea90011a00 con 0x7feaa0107120 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 -- 172.21.15.5:0/2428638987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 msgr2=0x7feaa0109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2428638987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea900098a0 tx=0x7fea90009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 -- 172.21.15.5:0/2428638987 shutdown_connections 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2428638987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa0109510 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 -- 172.21.15.5:0/2428638987 >> 172.21.15.5:0/2428638987 conn(0x7feaa0100b10 msgr2=0x7feaa0102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 -- 172.21.15.5:0/2428638987 shutdown_connections 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.219+0000 7feaa58ae700 1 -- 172.21.15.5:0/2428638987 wait complete. 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.220+0000 7feaa58ae700 1 Processor -- start 2023-09-23T04:36:46.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.220+0000 7feaa58ae700 1 -- start start 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7feaa58ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7feaa58ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa019a6b0 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7fea9effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7fea9effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52186/0 (socket says 172.21.15.5:52186) 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7fea9effd700 1 -- 172.21.15.5:0/2100919460 learned_addr learned my addr 172.21.15.5:0/2100919460 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7fea9effd700 1 -- 172.21.15.5:0/2100919460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90009550 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.221+0000 7fea9effd700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fea90006af0 tx=0x7fea90011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.222+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea90003a40 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.222+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea90003ba0 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.222+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa019a8b0 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.222+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea9001a6e0 con 0x7feaa0107120 2023-09-23T04:36:46.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.222+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa019acd0 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.223+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7fea90024430 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.223+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa0062450 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.223+0000 7feaa48ac700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 0x7fea8803ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.223+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fea9004c860 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.224+0000 7fea9e7fc700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 0x7fea8803ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.224+0000 7fea9e7fc700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 0x7fea8803ae00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea94006fd0 tx=0x7fea94006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.228+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fea900246e0 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.434+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7feaa010a7f0 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.435+0000 7feaa48ac700 1 -- 172.21.15.5:0/2100919460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fea90047e40 con 0x7feaa0107120 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 msgr2=0x7fea8803ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:46.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 0x7fea8803ae00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea94006fd0 tx=0x7fea94006e40 comp rx=0 tx=0).stop 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 msgr2=0x7feaa019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fea90006af0 tx=0x7fea90011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 shutdown_connections 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7fea88038940 0x7fea8803ae00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 --2- 172.21.15.5:0/2100919460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaa0107120 0x7feaa019a170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 >> 172.21.15.5:0/2100919460 conn(0x7feaa0100b10 msgr2=0x7feaa0101790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 shutdown_connections 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:46.438+0000 7feaa58ae700 1 -- 172.21.15.5:0/2100919460 wait complete. 2023-09-23T04:36:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2023-09-23T04:36:47.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2100919460' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-23T04:36:48.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:47 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/311647546' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-23T04:36:48.224 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-09-23T04:36:48.224 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-09-23T04:36:48.224 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:48.223+0000 7f3afd0c4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff3730e0700 1 Processor -- start 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff3730e0700 1 -- start start 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff3730e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff3730e0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c07a8b0 con 0x7ff36c07be80 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff370e7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff370e7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52190/0 (socket says 172.21.15.5:52190) 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.385+0000 7ff370e7c700 1 -- 172.21.15.5:0/2454478481 learned_addr learned my addr 172.21.15.5:0/2454478481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.386+0000 7ff370e7c700 1 -- 172.21.15.5:0/2454478481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c07a9f0 con 0x7ff36c07be80 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.386+0000 7ff370e7c700 1 --2- 172.21.15.5:0/2454478481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3600098a0 tx=0x7ff360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a4de08077bd86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.386+0000 7ff36b7fe700 1 -- 172.21.15.5:0/2454478481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff360004160 con 0x7ff36c07be80 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.386+0000 7ff36b7fe700 1 -- 172.21.15.5:0/2454478481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3600042c0 con 0x7ff36c07be80 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.386+0000 7ff36b7fe700 1 -- 172.21.15.5:0/2454478481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff360011a00 con 0x7ff36c07be80 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 -- 172.21.15.5:0/2454478481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 msgr2=0x7ff36c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 --2- 172.21.15.5:0/2454478481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3600098a0 tx=0x7ff360009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 -- 172.21.15.5:0/2454478481 shutdown_connections 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 --2- 172.21.15.5:0/2454478481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c07a2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:48.352 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 -- 172.21.15.5:0/2454478481 >> 172.21.15.5:0/2454478481 conn(0x7ff36c101c40 msgr2=0x7ff36c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 -- 172.21.15.5:0/2454478481 shutdown_connections 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.387+0000 7ff3730e0700 1 -- 172.21.15.5:0/2454478481 wait complete. 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.388+0000 7ff3730e0700 1 Processor -- start 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.388+0000 7ff3730e0700 1 -- start start 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff3730e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff3730e0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c19c8d0 con 0x7ff36c07be80 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff370e7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff370e7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52200/0 (socket says 172.21.15.5:52200) 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff370e7c700 1 -- 172.21.15.5:0/311647546 learned_addr learned my addr 172.21.15.5:0/311647546 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.389+0000 7ff370e7c700 1 -- 172.21.15.5:0/311647546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360009550 con 0x7ff36c07be80 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff370e7c700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff360006af0 tx=0x7ff360011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff360003a40 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff360003ba0 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c19cad0 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff36001a6e0 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.390+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c19cef0 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.391+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c062450 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.391+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 45631+0+0 (secure 0 0 0) 0x7ff360024a60 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.391+0000 7ff369ffb700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 0x7ff35c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.392+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff36004c860 con 0x7ff36c07be80 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.392+0000 7ff36bfff700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 0x7ff35c03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.392+0000 7ff36bfff700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 0x7ff35c03ae00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff358006fd0 tx=0x7ff358006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:48.354 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.397+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff360024430 con 0x7ff36c07be80 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.659+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ff36c19fc50 con 0x7ff36c07be80 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.943+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7ff360018350 con 0x7ff36c07be80 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.945+0000 7ff369ffb700 1 -- 172.21.15.5:0/311647546 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7ff36002d910 con 0x7ff36c07be80 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 msgr2=0x7ff35c03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 0x7ff35c03ae00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff358006fd0 tx=0x7ff358006e40 comp rx=0 tx=0).stop 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 msgr2=0x7ff36c19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff360006af0 tx=0x7ff360011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 shutdown_connections 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7ff35c038940 0x7ff35c03ae00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 --2- 172.21.15.5:0/311647546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff36c07be80 0x7ff36c19c390 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 >> 172.21.15.5:0/311647546 conn(0x7ff36c101c40 msgr2=0x7ff36c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:48.355 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 shutdown_connections 2023-09-23T04:36:48.356 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:47.947+0000 7ff3730e0700 1 -- 172.21.15.5:0/311647546 wait complete. 2023-09-23T04:36:48.591 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:48.331+0000 7f3afd0c4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:36:48.591 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:48.466+0000 7f3afd0c4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:36:48.591 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:48.591+0000 7f3afd0c4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:36:48.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:48.835+0000 7f3afd0c4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:36:49.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/311647546' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-23T04:36:49.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:48 smithi005 ceph-mon[97638]: mgrmap e5: x(active, since 6s) 2023-09-23T04:36:49.501 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:49.331+0000 7f3afd0c4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:36:49.501 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:49.500+0000 7f3afd0c4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:36:49.753 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:49.620+0000 7f3afd0c4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:36:49.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:36:49.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.935+0000 7f10679ce700 1 Processor -- start 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f10679ce700 1 -- start start 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f10679ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f10679ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1060107050 con 0x7f1060106660 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f106576a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:49.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f106576a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52218/0 (socket says 172.21.15.5:52218) 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.936+0000 7f106576a700 1 -- 172.21.15.5:0/3310092632 learned_addr learned my addr 172.21.15.5:0/3310092632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.937+0000 7f106576a700 1 -- 172.21.15.5:0/3310092632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060107860 con 0x7f1060106660 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.937+0000 7f106576a700 1 --2- 172.21.15.5:0/3310092632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f105c0098a0 tx=0x7f105c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e015d82f73f44293 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.937+0000 7f1057fff700 1 -- 172.21.15.5:0/3310092632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f105c004160 con 0x7f1060106660 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.937+0000 7f1057fff700 1 -- 172.21.15.5:0/3310092632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f105c0042c0 con 0x7f1060106660 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.937+0000 7f1057fff700 1 -- 172.21.15.5:0/3310092632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f105c011a00 con 0x7f1060106660 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 -- 172.21.15.5:0/3310092632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 msgr2=0x7f1060106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 --2- 172.21.15.5:0/3310092632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f105c0098a0 tx=0x7f105c009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 -- 172.21.15.5:0/3310092632 shutdown_connections 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 --2- 172.21.15.5:0/3310092632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f1060106a80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 -- 172.21.15.5:0/3310092632 >> 172.21.15.5:0/3310092632 conn(0x7f1060101c40 msgr2=0x7f1060104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:49.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 -- 172.21.15.5:0/3310092632 shutdown_connections 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.938+0000 7f10679ce700 1 -- 172.21.15.5:0/3310092632 wait complete. 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.939+0000 7f10679ce700 1 Processor -- start 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f10679ce700 1 -- start start 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f10679ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f10679ce700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106019c920 con 0x7f1060106660 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f106576a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f106576a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52234/0 (socket says 172.21.15.5:52234) 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.940+0000 7f106576a700 1 -- 172.21.15.5:0/3072940847 learned_addr learned my addr 172.21.15.5:0/3072940847 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f106576a700 1 -- 172.21.15.5:0/3072940847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009550 con 0x7f1060106660 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f106576a700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f105c006af0 tx=0x7f105c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f105c003a40 con 0x7f1060106660 2023-09-23T04:36:49.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f105c003ba0 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106019cb20 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.941+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f105c01a700 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.942+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106019cf40 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.943+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f105c024ad0 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.943+0000 7f10567fc700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 0x7f104c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.943+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1060062450 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.943+0000 7f1064f69700 1 -- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 msgr2=0x7f104c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.944+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f105c04c950 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.944+0000 7f1064f69700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 0x7f104c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:48.949+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f105c018ba0 con 0x7f1060106660 2023-09-23T04:36:49.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.145+0000 7f1064f69700 1 -- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 msgr2=0x7f104c03ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.145+0000 7f1064f69700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 0x7f104c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.214+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f106019fca0 con 0x7f1060106660 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.215+0000 7f10567fc700 1 -- 172.21.15.5:0/3072940847 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f105c0187c0 con 0x7f1060106660 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 msgr2=0x7f104c03ae00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 0x7f104c03ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 msgr2=0x7f106019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f105c006af0 tx=0x7f105c011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 shutdown_connections 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f104c038940 0x7f104c03ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 --2- 172.21.15.5:0/3072940847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1060106660 0x7f106019c3e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:49.946 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 >> 172.21.15.5:0/3072940847 conn(0x7f1060101c40 msgr2=0x7f10601028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:49.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 shutdown_connections 2023-09-23T04:36:49.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:49.217+0000 7f10679ce700 1 -- 172.21.15.5:0/3072940847 wait complete. 2023-09-23T04:36:49.947 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2023-09-23T04:36:49.947 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2023-09-23T04:36:50.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:49 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3072940847' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-23T04:36:50.155 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:49.753+0000 7f3afd0c4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:36:50.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:50.326+0000 7f3afd0c4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:36:51.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:51 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:51.206+0000 7f3afd0c4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:36:51.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:51 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:51.316+0000 7f3afd0c4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:36:52.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:52 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:52.307+0000 7f3afd0c4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:36:52.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:52 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:52.433+0000 7f3afd0c4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:36:53.037 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:52 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:52.663+0000 7f3afd0c4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:36:53.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:53 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:53.037+0000 7f3afd0c4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:36:53.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:53 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:53.319+0000 7f3afd0c4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:36:53.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:53 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:53.520+0000 7f3afd0c4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:36:54.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:53 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:53.873+0000 7f3afd0c4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:36:55.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:54 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:54.835+0000 7f3afd0c4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:36:55.155 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:55 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:55.016+0000 7f3afd0c4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:36:55.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:55 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:55.273+0000 7f3afd0c4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:36:55.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:55 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:55.424+0000 7f3afd0c4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:36:56.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:55 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:55.709+0000 7f3afd0c4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:36:57.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:36:56.814+0000 7f3afd0c4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: Active manager daemon x restarted 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: Activating manager daemon x 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: osdmap e2: 0 total, 0 up, 0 in 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: mgrmap e6: x(active, starting, since 0.00408202s) 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:36:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:36:57.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: Manager daemon x is now available 2023-09-23T04:36:57.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:36:57.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:36:57.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:36:57.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:56 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:36:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: Found migration_current of "None". Setting to last migration. 2023-09-23T04:36:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:36:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:36:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:36:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:36:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:36:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:57 smithi005 ceph-mon[97638]: mgrmap e7: x(active, since 1.00701s) 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.455+0000 7f31d1c12700 1 Processor -- start 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.455+0000 7f31d1c12700 1 -- start start 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.455+0000 7f31d1c12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.455+0000 7f31d1c12700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc107050 con 0x7f31cc106660 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.455+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:58.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.456+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52250/0 (socket says 172.21.15.5:52250) 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.456+0000 7f31cb7fe700 1 -- 172.21.15.5:0/1061227517 learned_addr learned my addr 172.21.15.5:0/1061227517 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.456+0000 7f31cb7fe700 1 -- 172.21.15.5:0/1061227517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc107860 con 0x7f31cc106660 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.456+0000 7f31cb7fe700 1 --2- 172.21.15.5:0/1061227517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0098a0 tx=0x7f31bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb881a9f5afa1e28 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.456+0000 7f31ca7fc700 1 -- 172.21.15.5:0/1061227517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31bc004160 con 0x7f31cc106660 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31ca7fc700 1 -- 172.21.15.5:0/1061227517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31bc0042c0 con 0x7f31cc106660 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31ca7fc700 1 -- 172.21.15.5:0/1061227517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31bc011a00 con 0x7f31cc106660 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 -- 172.21.15.5:0/1061227517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 msgr2=0x7f31cc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 --2- 172.21.15.5:0/1061227517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0098a0 tx=0x7f31bc009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 -- 172.21.15.5:0/1061227517 shutdown_connections 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 --2- 172.21.15.5:0/1061227517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc106a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 -- 172.21.15.5:0/1061227517 >> 172.21.15.5:0/1061227517 conn(0x7f31cc101c40 msgr2=0x7f31cc104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 -- 172.21.15.5:0/1061227517 shutdown_connections 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.457+0000 7f31d1c12700 1 -- 172.21.15.5:0/1061227517 wait complete. 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.459+0000 7f31d1c12700 1 Processor -- start 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.459+0000 7f31d1c12700 1 -- start start 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31d1c12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31d1c12700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc19a720 con 0x7f31cc106660 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52264/0 (socket says 172.21.15.5:52264) 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31cb7fe700 1 -- 172.21.15.5:0/2468349457 learned_addr learned my addr 172.21.15.5:0/2468349457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31cb7fe700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc009550 con 0x7f31cc106660 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.460+0000 7f31cb7fe700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006af0 tx=0x7f31bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:58.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.461+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31bc003a40 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.461+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31bc003ba0 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.461+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc19a920 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.461+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31bc01a700 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.461+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc19ad40 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.462+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f31bc024ad0 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.462+0000 7f31c8ff9700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.462+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31b403b510 con 0x7f31b4038940 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.462+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f31bc04c8d0 con 0x7f31cc106660 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.463+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.463+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:36:58.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.664+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:50.664+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:51.065+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:51.065+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:51.866+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:51.866+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:53.467+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:53.467+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:56.671+0000 7f31caffd700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/774964976 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:56.671+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:56.819+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) v1 ==== 45409+0+0 (secure 0 0 0) 0x7f31bc02d910 con 0x7f31cc106660 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:56.819+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 msgr2=0x7f31b403ae00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:36:58.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:56.819+0000 7f31c8ff9700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.820+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) v1 ==== 45536+0+0 (secure 0 0 0) 0x7f31bc02d3f0 con 0x7f31cc106660 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.820+0000 7f31c8ff9700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 0x7f31b403e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.820+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31b403b510 con 0x7f31b403bfe0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.821+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 0x7f31b403e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.823+0000 7f31caffd700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 0x7f31b403e3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c0003a10 tx=0x7f31c0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.824+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f31b403b510 con 0x7f31b403bfe0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.830+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f31cc19b520 con 0x7f31b403bfe0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31c8ff9700 1 -- 172.21.15.5:0/2468349457 <== mgr.14118 v2:172.21.15.5:6800/1973270664 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f31cc19b520 con 0x7f31b403bfe0 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 msgr2=0x7f31b403e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 0x7f31b403e3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c0003a10 tx=0x7f31c0009290 comp rx=0 tx=0).stop 2023-09-23T04:36:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 msgr2=0x7f31cc19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006af0 tx=0x7f31bc011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 shutdown_connections 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f31b403bfe0 0x7f31b403e3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31cc106660 0x7f31cc19a1e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 --2- 172.21.15.5:0/2468349457 >> [v2:172.21.15.5:6800/774964976,v1:172.21.15.5:6801/774964976] conn(0x7f31b4038940 0x7f31b403ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 >> 172.21.15.5:0/2468349457 conn(0x7f31cc101c40 msgr2=0x7f31cc1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 shutdown_connections 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:57.831+0000 7f31d1c12700 1 -- 172.21.15.5:0/2468349457 wait complete. 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2023-09-23T04:36:58.251 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2023-09-23T04:36:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:58 smithi005 ceph-mon[97638]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-23T04:36:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:58 smithi005 ceph-mon[97638]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-23T04:36:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:58 smithi005 ceph-mon[97638]: [23/Sep/2023:04:36:57] ENGINE Bus STARTING 2023-09-23T04:36:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:58 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:36:59.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.751+0000 7f4425dc6700 1 Processor -- start 2023-09-23T04:36:59.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.751+0000 7f4425dc6700 1 -- start start 2023-09-23T04:36:59.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f4425dc6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:59.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f4425dc6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4420107050 con 0x7f4420106660 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52446/0 (socket says 172.21.15.5:52446) 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f441f7fe700 1 -- 172.21.15.5:0/3089888912 learned_addr learned my addr 172.21.15.5:0/3089888912 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.752+0000 7f441f7fe700 1 -- 172.21.15.5:0/3089888912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420107860 con 0x7f4420106660 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f441f7fe700 1 --2- 172.21.15.5:0/3089888912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f44100098a0 tx=0x7f4410009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e47f97795abfab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:59.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f441e7fc700 1 -- 172.21.15.5:0/3089888912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4410004160 con 0x7f4420106660 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f441e7fc700 1 -- 172.21.15.5:0/3089888912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44100042c0 con 0x7f4420106660 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f441e7fc700 1 -- 172.21.15.5:0/3089888912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4410011a00 con 0x7f4420106660 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f4425dc6700 1 -- 172.21.15.5:0/3089888912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 msgr2=0x7f4420106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.753+0000 7f4425dc6700 1 --2- 172.21.15.5:0/3089888912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f44100098a0 tx=0x7f4410009bb0 comp rx=0 tx=0).stop 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.754+0000 7f4425dc6700 1 -- 172.21.15.5:0/3089888912 shutdown_connections 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.754+0000 7f4425dc6700 1 --2- 172.21.15.5:0/3089888912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f4420106a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:59.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.754+0000 7f4425dc6700 1 -- 172.21.15.5:0/3089888912 >> 172.21.15.5:0/3089888912 conn(0x7f4420101c40 msgr2=0x7f4420104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.754+0000 7f4425dc6700 1 -- 172.21.15.5:0/3089888912 shutdown_connections 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.754+0000 7f4425dc6700 1 -- 172.21.15.5:0/3089888912 wait complete. 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.755+0000 7f4425dc6700 1 Processor -- start 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f4425dc6700 1 -- start start 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f4425dc6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f4425dc6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442019a720 con 0x7f4420106660 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52454/0 (socket says 172.21.15.5:52454) 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.756+0000 7f441f7fe700 1 -- 172.21.15.5:0/1737175801 learned_addr learned my addr 172.21.15.5:0/1737175801 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f441f7fe700 1 -- 172.21.15.5:0/1737175801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4410009550 con 0x7f4420106660 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f441f7fe700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4410006af0 tx=0x7f4410011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:59.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4410003a40 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4410003ba0 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442019a920 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f441001a700 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.757+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442019ad40 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.758+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) v1 ==== 45536+0+0 (secure 0 0 0) 0x7f4410024ad0 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.758+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420062450 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.759+0000 7f441cff9700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 0x7f440803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.759+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f441004c710 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.759+0000 7f441effd700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 0x7f440803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.760+0000 7f441effd700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 0x7f440803ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4414006fd0 tx=0x7f4414006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.765+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4410018b80 con 0x7f4420106660 2023-09-23T04:36:59.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.873+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f441002d910 con 0x7f4420106660 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:58.993+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4420102ff0 con 0x7f4408038860 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.000+0000 7f441cff9700 1 -- 172.21.15.5:0/1737175801 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4420102ff0 con 0x7f4408038860 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.002+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 msgr2=0x7f440803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.002+0000 7f4425dc6700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 0x7f440803ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4414006fd0 tx=0x7f4414006e40 comp rx=0 tx=0).stop 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.002+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 msgr2=0x7f442019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.002+0000 7f4425dc6700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4410006af0 tx=0x7f4410011ad0 comp rx=0 tx=0).stop 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 shutdown_connections 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4408038860 0x7f440803ad20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 --2- 172.21.15.5:0/1737175801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4420106660 0x7f442019a1e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 >> 172.21.15.5:0/1737175801 conn(0x7f4420101c40 msgr2=0x7f44201028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:36:59.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 shutdown_connections 2023-09-23T04:36:59.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.003+0000 7f4425dc6700 1 -- 172.21.15.5:0/1737175801 wait complete. 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: [23/Sep/2023:04:36:58] ENGINE Serving on https://172.21.15.5:7150 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: [23/Sep/2023:04:36:58] ENGINE Serving on http://172.21.15.5:8765 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: [23/Sep/2023:04:36:58] ENGINE Bus STARTED 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: mgrmap e8: x(active, since 2s) 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:00.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:36:59 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:00.613 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.959+0000 7f954413e700 1 Processor -- start 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.959+0000 7f954413e700 1 -- start start 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.959+0000 7f954413e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.959+0000 7f954413e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c107050 con 0x7f953c106660 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9541eda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9541eda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52466/0 (socket says 172.21.15.5:52466) 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9541eda700 1 -- 172.21.15.5:0/625313056 learned_addr learned my addr 172.21.15.5:0/625313056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9541eda700 1 -- 172.21.15.5:0/625313056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c107860 con 0x7f953c106660 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9541eda700 1 --2- 172.21.15.5:0/625313056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95380098a0 tx=0x7f9538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7b3a19565d9a87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:00.614 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9540ed8700 1 -- 172.21.15.5:0/625313056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9538004160 con 0x7f953c106660 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9540ed8700 1 -- 172.21.15.5:0/625313056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95380042c0 con 0x7f953c106660 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.960+0000 7f9540ed8700 1 -- 172.21.15.5:0/625313056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9538011a00 con 0x7f953c106660 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 -- 172.21.15.5:0/625313056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 msgr2=0x7f953c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 --2- 172.21.15.5:0/625313056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95380098a0 tx=0x7f9538009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 -- 172.21.15.5:0/625313056 shutdown_connections 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 --2- 172.21.15.5:0/625313056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c106a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 -- 172.21.15.5:0/625313056 >> 172.21.15.5:0/625313056 conn(0x7f953c101c40 msgr2=0x7f953c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 -- 172.21.15.5:0/625313056 shutdown_connections 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.961+0000 7f954413e700 1 -- 172.21.15.5:0/625313056 wait complete. 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.962+0000 7f954413e700 1 Processor -- start 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.962+0000 7f954413e700 1 -- start start 2023-09-23T04:37:00.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f954413e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f954413e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c19c920 con 0x7f953c106660 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f9541eda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f9541eda700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52482/0 (socket says 172.21.15.5:52482) 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f9541eda700 1 -- 172.21.15.5:0/3367202675 learned_addr learned my addr 172.21.15.5:0/3367202675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.963+0000 7f9541eda700 1 -- 172.21.15.5:0/3367202675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009550 con 0x7f953c106660 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f9541eda700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9538006af0 tx=0x7f9538011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9538003a40 con 0x7f953c106660 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c19cb20 con 0x7f953c106660 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9538003ba0 con 0x7f953c106660 2023-09-23T04:37:00.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f953801a700 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.964+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c19cf40 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.965+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f9538024430 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.965+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c062450 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.965+0000 7f9532ffd700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 0x7f952803ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.965+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f953804c930 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.966+0000 7f95416d9700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 0x7f952803ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.966+0000 7f95416d9700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 0x7f952803ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f952c006fd0 tx=0x7f952c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:36:59.971+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f95380189a0 con 0x7f953c106660 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.181+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f953c102ff0 con 0x7f9528038990 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.182+0000 7f9532ffd700 1 -- 172.21.15.5:0/3367202675 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f953c102ff0 con 0x7f9528038990 2023-09-23T04:37:00.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.184+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 msgr2=0x7f952803ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.184+0000 7f954413e700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 0x7f952803ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f952c006fd0 tx=0x7f952c006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.184+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 msgr2=0x7f953c19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.184+0000 7f954413e700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9538006af0 tx=0x7f9538011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 shutdown_connections 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f9528038990 0x7f952803ae50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 --2- 172.21.15.5:0/3367202675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f953c106660 0x7f953c19c3e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 >> 172.21.15.5:0/3367202675 conn(0x7f953c101c40 msgr2=0x7f953c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 shutdown_connections 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:00.185+0000 7f954413e700 1 -- 172.21.15.5:0/3367202675 wait complete. 2023-09-23T04:37:00.618 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2023-09-23T04:37:01.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-mon[97638]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: Generating public/private rsa key pair. 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: Your identification has been saved in /tmp/tmp17xlsxd6/key. 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: Your public key has been saved in /tmp/tmp17xlsxd6/key.pub. 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: The key fingerprint is: 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: SHA256:ah2mIssyIY9V0BqoPP5tlHibFqD/gsAIpNMftpBteEE ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: The key's randomart image is: 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: +---[RSA 3072]----+ 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: | . oE | 2023-09-23T04:37:01.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |..o o | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |=. B . | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |=o*.B | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |=.oBoo. S | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |=+..o= = . | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |o== = B . | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: |+o.* O | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: | oo =. | 2023-09-23T04:37:01.656 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: +----[SHA256]-----+ 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.114+0000 7fe926b3c700 1 Processor -- start 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.114+0000 7fe926b3c700 1 -- start start 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.114+0000 7fe926b3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.114+0000 7fe926b3c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920107050 con 0x7fe920106660 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.115+0000 7fe9248d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.115+0000 7fe9248d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44132/0 (socket says 172.21.15.5:44132) 2023-09-23T04:37:01.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.115+0000 7fe9248d8700 1 -- 172.21.15.5:0/288645069 learned_addr learned my addr 172.21.15.5:0/288645069 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.115+0000 7fe9248d8700 1 -- 172.21.15.5:0/288645069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920107860 con 0x7fe920106660 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.115+0000 7fe9248d8700 1 --2- 172.21.15.5:0/288645069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9140098a0 tx=0x7fe914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba15f2a085830ff6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe91f7fe700 1 -- 172.21.15.5:0/288645069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe914004160 con 0x7fe920106660 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe91f7fe700 1 -- 172.21.15.5:0/288645069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9140042c0 con 0x7fe920106660 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe91f7fe700 1 -- 172.21.15.5:0/288645069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe914011a00 con 0x7fe920106660 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 -- 172.21.15.5:0/288645069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 msgr2=0x7fe920106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 --2- 172.21.15.5:0/288645069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9140098a0 tx=0x7fe914009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:01.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 -- 172.21.15.5:0/288645069 shutdown_connections 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 --2- 172.21.15.5:0/288645069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe920106a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 -- 172.21.15.5:0/288645069 >> 172.21.15.5:0/288645069 conn(0x7fe920101c40 msgr2=0x7fe920104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.116+0000 7fe926b3c700 1 -- 172.21.15.5:0/288645069 shutdown_connections 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.117+0000 7fe926b3c700 1 -- 172.21.15.5:0/288645069 wait complete. 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe926b3c700 1 Processor -- start 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe926b3c700 1 -- start start 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe926b3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe926b3c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92007c190 con 0x7fe920106660 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe9248d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe9248d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44142/0 (socket says 172.21.15.5:44142) 2023-09-23T04:37:01.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.118+0000 7fe9248d8700 1 -- 172.21.15.5:0/107576166 learned_addr learned my addr 172.21.15.5:0/107576166 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe9248d8700 1 -- 172.21.15.5:0/107576166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009550 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe9248d8700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe914006af0 tx=0x7fe914011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe914003a40 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe914003ba0 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92007a2a0 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe91401a700 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.119+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92007a6c0 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.120+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7fe914024430 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.120+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920062450 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.121+0000 7fe91dffb700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 0x7fe91003ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.121+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe91404c930 con 0x7fe920106660 2023-09-23T04:37:01.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.121+0000 7fe91ffff700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 0x7fe91003ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.122+0000 7fe91ffff700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 0x7fe91003ae50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006fd0 tx=0x7fe90c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.126+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe9140189a0 con 0x7fe920106660 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.335+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe92010aa60 con 0x7fe910038990 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.530+0000 7fe91dffb700 1 -- 172.21.15.5:0/107576166 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe92010aa60 con 0x7fe910038990 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.532+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 msgr2=0x7fe91003ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.532+0000 7fe926b3c700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 0x7fe91003ae50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006fd0 tx=0x7fe90c006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.532+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 msgr2=0x7fe92007bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.532+0000 7fe926b3c700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe914006af0 tx=0x7fe914011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 shutdown_connections 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fe910038990 0x7fe91003ae50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 --2- 172.21.15.5:0/107576166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe920106660 0x7fe92007bc50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 >> 172.21.15.5:0/107576166 conn(0x7fe920101c40 msgr2=0x7fe9201028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:01.993 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 shutdown_connections 2023-09-23T04:37:01.993 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:01.533+0000 7fe926b3c700 1 -- 172.21.15.5:0/107576166 wait complete. 2023-09-23T04:37:02.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:02 smithi005 ceph-mon[97638]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:02.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:02 smithi005 ceph-mon[97638]: Generating ssh key... 2023-09-23T04:37:02.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:02 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:02.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:02 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:03.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:37:03.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.482+0000 7fa2924a1700 1 Processor -- start 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.482+0000 7fa2924a1700 1 -- start start 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.482+0000 7fa2924a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.482+0000 7fa2924a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c107050 con 0x7fa28c106660 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.483+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.483+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44158/0 (socket says 172.21.15.5:44158) 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.483+0000 7fa28bfff700 1 -- 172.21.15.5:0/3791771169 learned_addr learned my addr 172.21.15.5:0/3791771169 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.483+0000 7fa28bfff700 1 -- 172.21.15.5:0/3791771169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c107860 con 0x7fa28c106660 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.483+0000 7fa28bfff700 1 --2- 172.21.15.5:0/3791771169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0098a0 tx=0x7fa27c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bd7a7896726ac6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa28affd700 1 -- 172.21.15.5:0/3791771169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27c004160 con 0x7fa28c106660 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa28affd700 1 -- 172.21.15.5:0/3791771169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa27c0042c0 con 0x7fa28c106660 2023-09-23T04:37:03.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa28affd700 1 -- 172.21.15.5:0/3791771169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27c011a00 con 0x7fa28c106660 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 -- 172.21.15.5:0/3791771169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 msgr2=0x7fa28c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3791771169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0098a0 tx=0x7fa27c009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 -- 172.21.15.5:0/3791771169 shutdown_connections 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3791771169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c106a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 -- 172.21.15.5:0/3791771169 >> 172.21.15.5:0/3791771169 conn(0x7fa28c101c40 msgr2=0x7fa28c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 -- 172.21.15.5:0/3791771169 shutdown_connections 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.484+0000 7fa2924a1700 1 -- 172.21.15.5:0/3791771169 wait complete. 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.485+0000 7fa2924a1700 1 Processor -- start 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa2924a1700 1 -- start start 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa2924a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa2924a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19c920 con 0x7fa28c106660 2023-09-23T04:37:03.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44160/0 (socket says 172.21.15.5:44160) 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa28bfff700 1 -- 172.21.15.5:0/3811394617 learned_addr learned my addr 172.21.15.5:0/3811394617 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.486+0000 7fa28bfff700 1 -- 172.21.15.5:0/3811394617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c009550 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa28bfff700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa27c006af0 tx=0x7fa27c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27c003a40 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa27c003ba0 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c19cb20 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27c01a700 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.487+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c19cf40 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.488+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7fa27c024ad0 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.488+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c062450 con 0x7fa28c106660 2023-09-23T04:37:03.169 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.488+0000 7fa2897fa700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 0x7fa27403ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.488+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa27c04c9a0 con 0x7fa28c106660 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.489+0000 7fa28b7fe700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 0x7fa27403ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.489+0000 7fa28b7fe700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 0x7fa27403ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa280006fd0 tx=0x7fa280006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.494+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa27c018b80 con 0x7fa28c106660 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.704+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28c102ff0 con 0x7fa274038940 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.705+0000 7fa2897fa700 1 -- 172.21.15.5:0/3811394617 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa28c102ff0 con 0x7fa274038940 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.707+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 msgr2=0x7fa27403ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.707+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 0x7fa27403ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa280006fd0 tx=0x7fa280006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.707+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 msgr2=0x7fa28c19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.707+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa27c006af0 tx=0x7fa27c011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:03.170 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 shutdown_connections 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa274038940 0x7fa27403ae00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 --2- 172.21.15.5:0/3811394617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa28c106660 0x7fa28c19c3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 >> 172.21.15.5:0/3811394617 conn(0x7fa28c101c40 msgr2=0x7fa28c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 shutdown_connections 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:02.708+0000 7fa2924a1700 1 -- 172.21.15.5:0/3811394617 wait complete. 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2023-09-23T04:37:03.171 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2023-09-23T04:37:03.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:03 smithi005 ceph-mon[97638]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:04.603 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:04 smithi005 ceph-mon[97638]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:05.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:05 smithi005 ceph-mon[97638]: Deploying cephadm binary to smithi005 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.684+0000 7f16f6e3b700 1 Processor -- start 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.684+0000 7f16f6e3b700 1 -- start start 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f6e3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f6e3b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0107050 con 0x7f16f0106660 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f4bd7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f4bd7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44172/0 (socket says 172.21.15.5:44172) 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f4bd7700 1 -- 172.21.15.5:0/1269135105 learned_addr learned my addr 172.21.15.5:0/1269135105 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.685+0000 7f16f4bd7700 1 -- 172.21.15.5:0/1269135105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0107860 con 0x7f16f0106660 2023-09-23T04:37:07.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16f4bd7700 1 --2- 172.21.15.5:0/1269135105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098a0 tx=0x7f16e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448987c03039ca66 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16ef7fe700 1 -- 172.21.15.5:0/1269135105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4004160 con 0x7f16f0106660 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16ef7fe700 1 -- 172.21.15.5:0/1269135105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16e40042c0 con 0x7f16f0106660 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16ef7fe700 1 -- 172.21.15.5:0/1269135105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4011a00 con 0x7f16f0106660 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16f6e3b700 1 -- 172.21.15.5:0/1269135105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 msgr2=0x7f16f0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.686+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/1269135105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16e40098a0 tx=0x7f16e4009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.687+0000 7f16f6e3b700 1 -- 172.21.15.5:0/1269135105 shutdown_connections 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.687+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/1269135105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f0106a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.687+0000 7f16f6e3b700 1 -- 172.21.15.5:0/1269135105 >> 172.21.15.5:0/1269135105 conn(0x7f16f0101c40 msgr2=0x7f16f0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.687+0000 7f16f6e3b700 1 -- 172.21.15.5:0/1269135105 shutdown_connections 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.687+0000 7f16f6e3b700 1 -- 172.21.15.5:0/1269135105 wait complete. 2023-09-23T04:37:07.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f6e3b700 1 Processor -- start 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f6e3b700 1 -- start start 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f6e3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f6e3b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f019c920 con 0x7f16f0106660 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f4bd7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f4bd7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44186/0 (socket says 172.21.15.5:44186) 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.688+0000 7f16f4bd7700 1 -- 172.21.15.5:0/883349543 learned_addr learned my addr 172.21.15.5:0/883349543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16f4bd7700 1 -- 172.21.15.5:0/883349543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4009550 con 0x7f16f0106660 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16f4bd7700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009fb0 tx=0x7f16e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4003930 con 0x7f16f0106660 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16e4003a90 con 0x7f16f0106660 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e40195e0 con 0x7f16f0106660 2023-09-23T04:37:07.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f019cb20 con 0x7f16f0106660 2023-09-23T04:37:07.656 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.689+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f019cf40 con 0x7f16f0106660 2023-09-23T04:37:07.656 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.691+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f0062450 con 0x7f16f0106660 2023-09-23T04:37:07.657 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.691+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f16e401a9d0 con 0x7f16f0106660 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.691+0000 7f16edffb700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 0x7f16e003ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.691+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16e404cae0 con 0x7f16f0106660 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.691+0000 7f16effff700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 0x7f16e003ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.692+0000 7f16effff700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 0x7f16e003ae00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16dc006fd0 tx=0x7f16dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.696+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f16e4023b40 con 0x7f16f0106660 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:03.911+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f0102ff0 con 0x7f16e0038940 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.244+0000 7f16edffb700 1 -- 172.21.15.5:0/883349543 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f16f0102ff0 con 0x7f16e0038940 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 msgr2=0x7f16e003ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 0x7f16e003ae00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f16dc006fd0 tx=0x7f16dc006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:07.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 msgr2=0x7f16f019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009fb0 tx=0x7f16e4011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 shutdown_connections 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f16e0038940 0x7f16e003ae00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 --2- 172.21.15.5:0/883349543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16f0106660 0x7f16f019c3e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.247+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 >> 172.21.15.5:0/883349543 conn(0x7f16f0101c40 msgr2=0x7f16f01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.248+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 shutdown_connections 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:07.248+0000 7f16f6e3b700 1 -- 172.21.15.5:0/883349543 wait complete. 2023-09-23T04:37:07.659 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2023-09-23T04:37:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:08 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:08 smithi005 ceph-mon[97638]: Added host smithi005 2023-09-23T04:37:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:08 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:09.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.162+0000 7ff5066b5700 1 Processor -- start 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.162+0000 7ff5066b5700 1 -- start start 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.163+0000 7ff5066b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.163+0000 7ff5066b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500107050 con 0x7ff500106660 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.163+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.163+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44202/0 (socket says 172.21.15.5:44202) 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.163+0000 7ff4fffff700 1 -- 172.21.15.5:0/363840978 learned_addr learned my addr 172.21.15.5:0/363840978 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff4fffff700 1 -- 172.21.15.5:0/363840978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff500107860 con 0x7ff500106660 2023-09-23T04:37:09.142 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff4fffff700 1 --2- 172.21.15.5:0/363840978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098a0 tx=0x7ff4f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f0229fa723f068 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff4feffd700 1 -- 172.21.15.5:0/363840978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0004160 con 0x7ff500106660 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff4feffd700 1 -- 172.21.15.5:0/363840978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4f00042c0 con 0x7ff500106660 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff4feffd700 1 -- 172.21.15.5:0/363840978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0011a00 con 0x7ff500106660 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff5066b5700 1 -- 172.21.15.5:0/363840978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 msgr2=0x7ff500106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.164+0000 7ff5066b5700 1 --2- 172.21.15.5:0/363840978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00098a0 tx=0x7ff4f0009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.165+0000 7ff5066b5700 1 -- 172.21.15.5:0/363840978 shutdown_connections 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.165+0000 7ff5066b5700 1 --2- 172.21.15.5:0/363840978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff500106a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.165+0000 7ff5066b5700 1 -- 172.21.15.5:0/363840978 >> 172.21.15.5:0/363840978 conn(0x7ff500101c40 msgr2=0x7ff500104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.165+0000 7ff5066b5700 1 -- 172.21.15.5:0/363840978 shutdown_connections 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.165+0000 7ff5066b5700 1 -- 172.21.15.5:0/363840978 wait complete. 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff5066b5700 1 Processor -- start 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff5066b5700 1 -- start start 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff5066b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff5066b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50019c920 con 0x7ff500106660 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff4fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44214/0 (socket says 172.21.15.5:44214) 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.166+0000 7ff4fffff700 1 -- 172.21.15.5:0/1928945736 learned_addr learned my addr 172.21.15.5:0/1928945736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff4fffff700 1 -- 172.21.15.5:0/1928945736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009550 con 0x7ff500106660 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff4fffff700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006af0 tx=0x7ff4f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f0003a40 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4f0003ba0 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50019cb20 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4f001a700 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.167+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50019cf40 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.168+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7ff4f0024ad0 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.169+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff500062450 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.169+0000 7ff4fd7fa700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 0x7ff4e803ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.169+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff4f004c950 con 0x7ff500106660 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.169+0000 7ff4ff7fe700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 0x7ff4e803ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:09.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.170+0000 7ff4ff7fe700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 0x7ff4e803ae50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.174+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff4f0018b80 con 0x7ff500106660 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.398+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff50010aa60 con 0x7ff4e8038990 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.403+0000 7ff4fd7fa700 1 -- 172.21.15.5:0/1928945736 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff50010aa60 con 0x7ff4e8038990 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 msgr2=0x7ff4e803ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 0x7ff4e803ae50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4006fd0 tx=0x7ff4f4006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 msgr2=0x7ff50019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006af0 tx=0x7ff4f0011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 shutdown_connections 2023-09-23T04:37:09.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7ff4e8038990 0x7ff4e803ae50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:09.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 --2- 172.21.15.5:0/1928945736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff500106660 0x7ff50019c3e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:09.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 >> 172.21.15.5:0/1928945736 conn(0x7ff500101c40 msgr2=0x7ff5001028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:09.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 shutdown_connections 2023-09-23T04:37:09.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:08.406+0000 7ff5066b5700 1 -- 172.21.15.5:0/1928945736 wait complete. 2023-09-23T04:37:09.147 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2023-09-23T04:37:09.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:09 smithi005 ceph-mon[97638]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:09.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:09 smithi005 ceph-mon[97638]: Saving service mon spec with placement count:5 2023-09-23T04:37:09.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:09 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:10.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-23T04:37:10.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.733+0000 7f2803279700 1 Processor -- start 2023-09-23T04:37:10.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2803279700 1 -- start start 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2803279700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2803279700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc107050 con 0x7f27fc106660 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2801015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2801015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44230/0 (socket says 172.21.15.5:44230) 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.734+0000 7f2801015700 1 -- 172.21.15.5:0/800399988 learned_addr learned my addr 172.21.15.5:0/800399988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.735+0000 7f2801015700 1 -- 172.21.15.5:0/800399988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc107860 con 0x7f27fc106660 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.735+0000 7f2801015700 1 --2- 172.21.15.5:0/800399988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4f26287b691407 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.735+0000 7f27f3fff700 1 -- 172.21.15.5:0/800399988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f8004160 con 0x7f27fc106660 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.735+0000 7f27f3fff700 1 -- 172.21.15.5:0/800399988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27f80042c0 con 0x7f27fc106660 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.735+0000 7f27f3fff700 1 -- 172.21.15.5:0/800399988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f8011a00 con 0x7f27fc106660 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 -- 172.21.15.5:0/800399988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 msgr2=0x7f27fc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 --2- 172.21.15.5:0/800399988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 -- 172.21.15.5:0/800399988 shutdown_connections 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 --2- 172.21.15.5:0/800399988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc106a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 -- 172.21.15.5:0/800399988 >> 172.21.15.5:0/800399988 conn(0x7f27fc101c40 msgr2=0x7f27fc104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 -- 172.21.15.5:0/800399988 shutdown_connections 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.736+0000 7f2803279700 1 -- 172.21.15.5:0/800399988 wait complete. 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.737+0000 7f2803279700 1 Processor -- start 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.737+0000 7f2803279700 1 -- start start 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.737+0000 7f2803279700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2803279700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc19a720 con 0x7f27fc106660 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2801015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:10.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2801015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44236/0 (socket says 172.21.15.5:44236) 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2801015700 1 -- 172.21.15.5:0/1507797099 learned_addr learned my addr 172.21.15.5:0/1507797099 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2801015700 1 -- 172.21.15.5:0/1507797099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8009550 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.738+0000 7f2801015700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006af0 tx=0x7f27f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.739+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f8003a40 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.739+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27f8003ba0 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.739+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f801a700 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.739+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc19a920 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.739+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc19ad40 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.740+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f27f8024430 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.740+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc062450 con 0x7f27fc106660 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.740+0000 7f27f27fc700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 0x7f27e803ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.740+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f27f804c930 con 0x7f27fc106660 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.740+0000 7f2800814700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 0x7f27e803ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.741+0000 7f2800814700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 0x7f27e803ae50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.745+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f27f80189a0 con 0x7f27fc106660 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.815+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mon.0 v2:172.21.15.5:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f80189a0 con 0x7f27fc106660 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.981+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f27fc102ff0 con 0x7f27e8038990 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.986+0000 7f27f27fc700 1 -- 172.21.15.5:0/1507797099 <== mgr.14118 v2:172.21.15.5:6800/1973270664 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f27fc102ff0 con 0x7f27e8038990 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 msgr2=0x7f27e803ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 0x7f27e803ae50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 msgr2=0x7f27fc19a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006af0 tx=0x7f27f8011ad0 comp rx=0 tx=0).stop 2023-09-23T04:37:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 shutdown_connections 2023-09-23T04:37:10.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f27e8038990 0x7f27e803ae50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:10.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 --2- 172.21.15.5:0/1507797099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27fc106660 0x7f27fc19a1e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:10.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 >> 172.21.15.5:0/1507797099 conn(0x7f27fc101c40 msgr2=0x7f27fc1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:10.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 shutdown_connections 2023-09-23T04:37:10.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:09.988+0000 7f2803279700 1 -- 172.21.15.5:0/1507797099 wait complete. 2023-09-23T04:37:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:10 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:10 smithi005 ceph-mon[97638]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:10 smithi005 ceph-mon[97638]: Saving service mgr spec with placement count:2 2023-09-23T04:37:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:10 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:10 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:10.999+0000 7f6596f32700 1 Processor -- start 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:10.999+0000 7f6596f32700 1 -- start start 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.000+0000 7f6596f32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.000+0000 7f6596f32700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901070a0 con 0x7f65901066b0 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.000+0000 7f6594cce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.000+0000 7f6594cce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52060/0 (socket says 172.21.15.5:52060) 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.000+0000 7f6594cce700 1 -- 172.21.15.5:0/1785918251 learned_addr learned my addr 172.21.15.5:0/1785918251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.001+0000 7f6594cce700 1 -- 172.21.15.5:0/1785918251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65901078b0 con 0x7f65901066b0 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.001+0000 7f6594cce700 1 --2- 172.21.15.5:0/1785918251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65840098a0 tx=0x7f6584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6d9a02417ee19f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:11.707 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.001+0000 7f658f7fe700 1 -- 172.21.15.5:0/1785918251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6584004160 con 0x7f65901066b0 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.001+0000 7f658f7fe700 1 -- 172.21.15.5:0/1785918251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65840042c0 con 0x7f65901066b0 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.001+0000 7f658f7fe700 1 -- 172.21.15.5:0/1785918251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65840119b0 con 0x7f65901066b0 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 -- 172.21.15.5:0/1785918251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 msgr2=0x7f6590106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 --2- 172.21.15.5:0/1785918251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65840098a0 tx=0x7f6584009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 -- 172.21.15.5:0/1785918251 shutdown_connections 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 --2- 172.21.15.5:0/1785918251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f6590106ad0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 -- 172.21.15.5:0/1785918251 >> 172.21.15.5:0/1785918251 conn(0x7f6590101c70 msgr2=0x7f6590104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 -- 172.21.15.5:0/1785918251 shutdown_connections 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.002+0000 7f6596f32700 1 -- 172.21.15.5:0/1785918251 wait complete. 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.003+0000 7f6596f32700 1 Processor -- start 2023-09-23T04:37:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.003+0000 7f6596f32700 1 -- start start 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6596f32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6596f32700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659007c190 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6594cce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6594cce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52076/0 (socket says 172.21.15.5:52076) 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6594cce700 1 -- 172.21.15.5:0/1640051618 learned_addr learned my addr 172.21.15.5:0/1640051618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.004+0000 7f6594cce700 1 -- 172.21.15.5:0/1640051618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584009550 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f6594cce700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6584003e60 tx=0x7f6584011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6584003a60 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6584003bc0 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659007a2a0 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65840196c0 con 0x7f65901066b0 2023-09-23T04:37:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.005+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659007a6c0 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.006+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f658401a430 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.006+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659010aae0 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.006+0000 7f658dffb700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 0x7f658003ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.006+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f658404cbb0 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.007+0000 7f658ffff700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 0x7f658003ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.007+0000 7f658ffff700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 0x7f658003ae80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f657c006fd0 tx=0x7f657c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.012+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6584010380 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.233+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f659010ad50 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.236+0000 7f658dffb700 1 -- 172.21.15.5:0/1640051618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f65840281c0 con 0x7f65901066b0 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 msgr2=0x7f658003ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:11.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 0x7f658003ae80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f657c006fd0 tx=0x7f657c006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 msgr2=0x7f659007bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6584003e60 tx=0x7f6584011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 shutdown_connections 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f65800389c0 0x7f658003ae80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 --2- 172.21.15.5:0/1640051618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901066b0 0x7f659007bc50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 >> 172.21.15.5:0/1640051618 conn(0x7f6590101c70 msgr2=0x7f6590102980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 shutdown_connections 2023-09-23T04:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:11.239+0000 7f6596f32700 1 -- 172.21.15.5:0/1640051618 wait complete. 2023-09-23T04:37:12.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:12 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1640051618' entity='client.admin' 2023-09-23T04:37:12.999 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.340+0000 7f10b9317700 1 Processor -- start 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.340+0000 7f10b9317700 1 -- start start 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.340+0000 7f10b9317700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.340+0000 7f10b9317700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b407a8b0 con 0x7f10b407be80 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.341+0000 7f10b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.341+0000 7f10b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52084/0 (socket says 172.21.15.5:52084) 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.341+0000 7f10b2ffd700 1 -- 172.21.15.5:0/2728958253 learned_addr learned my addr 172.21.15.5:0/2728958253 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.341+0000 7f10b2ffd700 1 -- 172.21.15.5:0/2728958253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b407a9f0 con 0x7f10b407be80 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b2ffd700 1 --2- 172.21.15.5:0/2728958253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10a40098a0 tx=0x7f10a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874e0a328bc9206e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b1ffb700 1 -- 172.21.15.5:0/2728958253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10a4004160 con 0x7f10b407be80 2023-09-23T04:37:13.000 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b1ffb700 1 -- 172.21.15.5:0/2728958253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10a40042c0 con 0x7f10b407be80 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b1ffb700 1 -- 172.21.15.5:0/2728958253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10a40119b0 con 0x7f10b407be80 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b9317700 1 -- 172.21.15.5:0/2728958253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 msgr2=0x7f10b407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.342+0000 7f10b9317700 1 --2- 172.21.15.5:0/2728958253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10a40098a0 tx=0x7f10a4009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.343+0000 7f10b9317700 1 -- 172.21.15.5:0/2728958253 shutdown_connections 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.343+0000 7f10b9317700 1 --2- 172.21.15.5:0/2728958253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b407a2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.343+0000 7f10b9317700 1 -- 172.21.15.5:0/2728958253 >> 172.21.15.5:0/2728958253 conn(0x7f10b4101c40 msgr2=0x7f10b4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.343+0000 7f10b9317700 1 -- 172.21.15.5:0/2728958253 shutdown_connections 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.343+0000 7f10b9317700 1 -- 172.21.15.5:0/2728958253 wait complete. 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.344+0000 7f10b9317700 1 Processor -- start 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.344+0000 7f10b9317700 1 -- start start 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.344+0000 7f10b9317700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:13.001 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.344+0000 7f10b9317700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b419c900 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.344+0000 7f10b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f10b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52086/0 (socket says 172.21.15.5:52086) 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f10b2ffd700 1 -- 172.21.15.5:0/937130717 learned_addr learned my addr 172.21.15.5:0/937130717 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f10b2ffd700 1 -- 172.21.15.5:0/937130717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4009550 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f10b2ffd700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10a4011980 tx=0x7f10a4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10a4003a60 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10a4003bc0 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10a401a670 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.345+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b419cb00 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.346+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b419cf20 con 0x7f10b407be80 2023-09-23T04:37:13.002 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.347+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f10a4024a00 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.347+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b4062450 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.347+0000 7f109bfff700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 0x7f109c03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.347+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f10a404ca90 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.347+0000 7f10b27fc700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 0x7f109c03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.348+0000 7f10b27fc700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 0x7f109c03ae00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006fd0 tx=0x7f10a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.352+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f10a40189a0 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.570+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f10b419f960 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.572+0000 7f109bfff700 1 -- 172.21.15.5:0/937130717 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f10a4018350 con 0x7f10b407be80 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 msgr2=0x7f109c03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 0x7f109c03ae00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006fd0 tx=0x7f10a8006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 msgr2=0x7f10b419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:13.003 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10a4011980 tx=0x7f10a4011a60 comp rx=0 tx=0).stop 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 shutdown_connections 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f109c038940 0x7f109c03ae00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 --2- 172.21.15.5:0/937130717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10b407be80 0x7f10b419c3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 >> 172.21.15.5:0/937130717 conn(0x7f10b4101c40 msgr2=0x7f10b41028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 shutdown_connections 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:12.575+0000 7f10b9317700 1 -- 172.21.15.5:0/937130717 wait complete. 2023-09-23T04:37:13.004 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2023-09-23T04:37:13.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:13 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/937130717' entity='client.admin' 2023-09-23T04:37:13.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:13 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:14.854 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-09-23T04:37:14.854 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-09-23T04:37:14.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:14 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/804225489' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-23T04:37:14.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:14 smithi005 ceph-mon[97638]: from='mgr.14118 172.21.15.5:0/2275079001' entity='mgr.x' 2023-09-23T04:37:15.136 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:14.899+0000 7f8d3509a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:37:15.136 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:15.014+0000 7f8d3509a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:37:15.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.553+0000 7f8bdcfde700 1 Processor -- start 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.553+0000 7f8bdcfde700 1 -- start start 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.553+0000 7f8bdcfde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.553+0000 7f8bdcfde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd807a8d0 con 0x7f8bd807bea0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.554+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.554+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52094/0 (socket says 172.21.15.5:52094) 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.554+0000 7f8bd659c700 1 -- 172.21.15.5:0/1342566277 learned_addr learned my addr 172.21.15.5:0/1342566277 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.554+0000 7f8bd659c700 1 -- 172.21.15.5:0/1342566277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd807aa10 con 0x7f8bd807bea0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.554+0000 7f8bd659c700 1 --2- 172.21.15.5:0/1342566277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098a0 tx=0x7f8bc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89839cb3993d7efc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bd559a700 1 -- 172.21.15.5:0/1342566277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bc8004160 con 0x7f8bd807bea0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bd559a700 1 -- 172.21.15.5:0/1342566277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc80042c0 con 0x7f8bd807bea0 2023-09-23T04:37:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bd559a700 1 -- 172.21.15.5:0/1342566277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bc80119b0 con 0x7f8bd807bea0 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bdcfde700 1 -- 172.21.15.5:0/1342566277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 msgr2=0x7f8bd807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/1342566277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098a0 tx=0x7f8bc8009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bdcfde700 1 -- 172.21.15.5:0/1342566277 shutdown_connections 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/1342566277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd807a300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.555+0000 7f8bdcfde700 1 -- 172.21.15.5:0/1342566277 >> 172.21.15.5:0/1342566277 conn(0x7f8bd8101c40 msgr2=0x7f8bd8104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.556+0000 7f8bdcfde700 1 -- 172.21.15.5:0/1342566277 shutdown_connections 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.556+0000 7f8bdcfde700 1 -- 172.21.15.5:0/1342566277 wait complete. 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bdcfde700 1 Processor -- start 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bdcfde700 1 -- start start 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bdcfde700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bdcfde700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819c940 con 0x7f8bd807bea0 2023-09-23T04:37:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52112/0 (socket says 172.21.15.5:52112) 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.557+0000 7f8bd659c700 1 -- 172.21.15.5:0/804225489 learned_addr learned my addr 172.21.15.5:0/804225489 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bd659c700 1 -- 172.21.15.5:0/804225489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8009550 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bd659c700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000c00 tx=0x7f8bc8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bc80039f0 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc8003b50 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bc801a780 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd819cb40 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.558+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd819cf60 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.560+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd8062450 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.560+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45642+0+0 (secure 0 0 0) 0x7f8bc8024440 con 0x7f8bd807bea0 2023-09-23T04:37:15.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.560+0000 7f8bc77fe700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 0x7f8bc003ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.560+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8bc804c980 con 0x7f8bd807bea0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.561+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 0x7f8bc003ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.561+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 0x7f8bc003ae50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.565+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8bc80246f0 con 0x7f8bd807bea0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:13.848+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8bd810ca80 con 0x7f8bd807bea0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.576+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8bc8018350 con 0x7f8bd807bea0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.578+0000 7f8bc77fe700 1 -- 172.21.15.5:0/804225489 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f8bc802d910 con 0x7f8bd807bea0 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 msgr2=0x7f8bc003ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 0x7f8bc003ae50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 msgr2=0x7f8bd819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000c00 tx=0x7f8bc8011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:15.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 shutdown_connections 2023-09-23T04:37:15.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f8bc0038990 0x7f8bc003ae50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:15.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 --2- 172.21.15.5:0/804225489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd807bea0 0x7f8bd819c400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:15.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 >> 172.21.15.5:0/804225489 conn(0x7f8bd8101c40 msgr2=0x7f8bd81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:15.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 shutdown_connections 2023-09-23T04:37:15.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:14.580+0000 7f8bdcfde700 1 -- 172.21.15.5:0/804225489 wait complete. 2023-09-23T04:37:15.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:15.135+0000 7f8d3509a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:37:15.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:15.280+0000 7f8d3509a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:37:15.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/804225489' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-23T04:37:15.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-mon[97638]: mgrmap e9: x(active, since 17s) 2023-09-23T04:37:15.787 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:15.526+0000 7f8d3509a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:37:16.077 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:16.012+0000 7f8d3509a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:37:16.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:16.191+0000 7f8d3509a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:37:16.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:16.314+0000 7f8d3509a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:37:16.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:16 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4236612657' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-23T04:37:16.745 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:16.446+0000 7f8d3509a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:37:16.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.338+0000 7f406369e700 1 Processor -- start 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.338+0000 7f406369e700 1 -- start start 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.339+0000 7f406369e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.339+0000 7f406369e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c07a8b0 con 0x7f405c07be80 2023-09-23T04:37:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.339+0000 7f406143a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.339+0000 7f406143a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52142/0 (socket says 172.21.15.5:52142) 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.339+0000 7f406143a700 1 -- 172.21.15.5:0/4019088954 learned_addr learned my addr 172.21.15.5:0/4019088954 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.340+0000 7f406143a700 1 -- 172.21.15.5:0/4019088954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c07a9f0 con 0x7f405c07be80 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.340+0000 7f406143a700 1 --2- 172.21.15.5:0/4019088954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40580098a0 tx=0x7f4058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e40c75664dcdb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.340+0000 7f4053fff700 1 -- 172.21.15.5:0/4019088954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4058004160 con 0x7f405c07be80 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.340+0000 7f4053fff700 1 -- 172.21.15.5:0/4019088954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40580042c0 con 0x7f405c07be80 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.340+0000 7f4053fff700 1 -- 172.21.15.5:0/4019088954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40580119b0 con 0x7f405c07be80 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 -- 172.21.15.5:0/4019088954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 msgr2=0x7f405c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 --2- 172.21.15.5:0/4019088954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40580098a0 tx=0x7f4058009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 -- 172.21.15.5:0/4019088954 shutdown_connections 2023-09-23T04:37:16.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 --2- 172.21.15.5:0/4019088954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c07a2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 -- 172.21.15.5:0/4019088954 >> 172.21.15.5:0/4019088954 conn(0x7f405c101c40 msgr2=0x7f405c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 -- 172.21.15.5:0/4019088954 shutdown_connections 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.341+0000 7f406369e700 1 -- 172.21.15.5:0/4019088954 wait complete. 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.343+0000 7f406369e700 1 Processor -- start 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.343+0000 7f406369e700 1 -- start start 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.343+0000 7f406369e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.343+0000 7f406369e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c19c900 con 0x7f405c07be80 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.344+0000 7f406143a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.344+0000 7f406143a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52152/0 (socket says 172.21.15.5:52152) 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.344+0000 7f406143a700 1 -- 172.21.15.5:0/4236612657 learned_addr learned my addr 172.21.15.5:0/4236612657 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:16.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.344+0000 7f406143a700 1 -- 172.21.15.5:0/4236612657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058009550 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.344+0000 7f406143a700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4058003e60 tx=0x7f4058011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.345+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4058003970 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.345+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4058003ad0 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.345+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c19cb00 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.345+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40580195e0 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.345+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c19cf20 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.346+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f4058019890 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.346+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c062450 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.346+0000 7f40527fc700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 0x7f404803ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.347+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f405804cb40 con 0x7f405c07be80 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.347+0000 7f4060c39700 1 -- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 msgr2=0x7f404803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:16.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.347+0000 7f4060c39700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 0x7f404803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.352+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f405802b340 con 0x7f405c07be80 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.548+0000 7f4060c39700 1 -- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 msgr2=0x7f404803ae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.548+0000 7f4060c39700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 0x7f404803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.616+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f405c19f960 con 0x7f405c07be80 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.617+0000 7f40527fc700 1 -- 172.21.15.5:0/4236612657 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f40580172b0 con 0x7f405c07be80 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.619+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 msgr2=0x7f404803ae50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.619+0000 7f406369e700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 0x7f404803ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.619+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 msgr2=0x7f405c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.619+0000 7f406369e700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4058003e60 tx=0x7f4058011aa0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 shutdown_connections 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7f4048038990 0x7f404803ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 --2- 172.21.15.5:0/4236612657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f405c07be80 0x7f405c19c3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 >> 172.21.15.5:0/4236612657 conn(0x7f405c101c40 msgr2=0x7f405c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 shutdown_connections 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:16.620+0000 7f406369e700 1 -- 172.21.15.5:0/4236612657 wait complete. 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2023-09-23T04:37:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2023-09-23T04:37:17.293 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:17 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:17.018+0000 7f8d3509a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:37:18.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:17 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:17.920+0000 7f8d3509a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:37:18.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:18.028+0000 7f8d3509a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:37:19.376 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:19.017+0000 7f8d3509a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:37:19.377 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:19.141+0000 7f8d3509a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:37:19.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:19.375+0000 7f8d3509a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:37:20.035 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:19.751+0000 7f8d3509a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:37:20.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:20.034+0000 7f8d3509a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:37:20.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:20.236+0000 7f8d3509a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:37:20.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:20.592+0000 7f8d3509a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:37:21.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:21.558+0000 7f8d3509a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:37:21.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:21.741+0000 7f8d3509a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:37:22.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:22.000+0000 7f8d3509a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:37:22.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:22.154+0000 7f8d3509a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:37:22.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:22.441+0000 7f8d3509a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:37:23.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:37:23.549+0000 7f8d3509a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: Active manager daemon x restarted 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: Activating manager daemon x 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: osdmap e3: 0 total, 0 up, 0 in 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: mgrmap e10: x(active, starting, since 0.00371757s) 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:37:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: Manager daemon x is now available 2023-09-23T04:37:23.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:23.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:37:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:37:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:24 smithi005 ceph-mon[97638]: mgrmap e11: x(active, since 1.00768s) 2023-09-23T04:37:24.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-09-23T04:37:24.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-23T04:37:24.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.494+0000 7fa7420fa700 1 Processor -- start 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa7420fa700 1 -- start start 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa7420fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa7420fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c07a8b0 con 0x7fa73c07be80 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52156/0 (socket says 172.21.15.5:52156) 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.495+0000 7fa73b7fe700 1 -- 172.21.15.5:0/1320920100 learned_addr learned my addr 172.21.15.5:0/1320920100 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.496+0000 7fa73b7fe700 1 -- 172.21.15.5:0/1320920100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c07a9f0 con 0x7fa73c07be80 2023-09-23T04:37:24.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.496+0000 7fa73b7fe700 1 --2- 172.21.15.5:0/1320920100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0098a0 tx=0x7fa72c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c09ec286eb7be17 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.496+0000 7fa73a7fc700 1 -- 172.21.15.5:0/1320920100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c004160 con 0x7fa73c07be80 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.496+0000 7fa73a7fc700 1 -- 172.21.15.5:0/1320920100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72c0042c0 con 0x7fa73c07be80 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.496+0000 7fa73a7fc700 1 -- 172.21.15.5:0/1320920100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0119b0 con 0x7fa73c07be80 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 -- 172.21.15.5:0/1320920100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 msgr2=0x7fa73c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1320920100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0098a0 tx=0x7fa72c009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 -- 172.21.15.5:0/1320920100 shutdown_connections 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1320920100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c07a2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 -- 172.21.15.5:0/1320920100 >> 172.21.15.5:0/1320920100 conn(0x7fa73c101c40 msgr2=0x7fa73c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 -- 172.21.15.5:0/1320920100 shutdown_connections 2023-09-23T04:37:24.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.497+0000 7fa7420fa700 1 -- 172.21.15.5:0/1320920100 wait complete. 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.498+0000 7fa7420fa700 1 Processor -- start 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.498+0000 7fa7420fa700 1 -- start start 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa7420fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa7420fa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c19c900 con 0x7fa73c07be80 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52176/0 (socket says 172.21.15.5:52176) 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa73b7fe700 1 -- 172.21.15.5:0/1047069097 learned_addr learned my addr 172.21.15.5:0/1047069097 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.499+0000 7fa73b7fe700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009550 con 0x7fa73c07be80 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa73b7fe700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa72c003e60 tx=0x7fa72c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c003970 con 0x7fa73c07be80 2023-09-23T04:37:24.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72c003ad0 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c19cb00 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0195e0 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.500+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c19cf20 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.501+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) v1 ==== 45655+0+0 (secure 0 0 0) 0x7fa72c0198a0 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.502+0000 7fa738ff9700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.502+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa72403b510 con 0x7fa724038940 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.502+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa72c04cb90 con 0x7fa73c07be80 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.502+0000 7fa73affd700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.502+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:37:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.703+0000 7fa73affd700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:17.703+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:18.104+0000 7fa73affd700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:18.104+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:18.905+0000 7fa73affd700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:18.905+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:20.506+0000 7fa73affd700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1973270664 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:20.507+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:23.554+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) v1 ==== 45422+0+0 (secure 0 0 0) 0x7fa72c019ba0 con 0x7fa73c07be80 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:23.554+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 msgr2=0x7fa72403ae00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:37:24.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:23.554+0000 7fa738ff9700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.557+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fa72c04e2f0 con 0x7fa73c07be80 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.557+0000 7fa738ff9700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 0x7fa72403e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.557+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa72403b510 con 0x7fa72403c500 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.558+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 0x7fa72403e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.559+0000 7fa73affd700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 0x7fa72403e8f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa730003a10 tx=0x7fa7300092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.559+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa72403b510 con 0x7fa72403c500 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.567+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa73c19d700 con 0x7fa72403c500 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.567+0000 7fa738ff9700 1 -- 172.21.15.5:0/1047069097 <== mgr.14150 v2:172.21.15.5:6800/4010853589 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa73c19d700 con 0x7fa72403c500 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 msgr2=0x7fa72403e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 0x7fa72403e8f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa730003a10 tx=0x7fa7300092b0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 msgr2=0x7fa73c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa72c003e60 tx=0x7fa72c011aa0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 shutdown_connections 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa72403c500 0x7fa72403e8f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa73c07be80 0x7fa73c19c3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 --2- 172.21.15.5:0/1047069097 >> [v2:172.21.15.5:6800/1973270664,v1:172.21.15.5:6801/1973270664] conn(0x7fa724038940 0x7fa72403ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 >> 172.21.15.5:0/1047069097 conn(0x7fa73c101c40 msgr2=0x7fa73c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 shutdown_connections 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:24.568+0000 7fa7420fa700 1 -- 172.21.15.5:0/1047069097 wait complete. 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2023-09-23T04:37:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2023-09-23T04:37:25.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: [23/Sep/2023:04:37:24] ENGINE Bus STARTING 2023-09-23T04:37:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: [23/Sep/2023:04:37:24] ENGINE Serving on https://172.21.15.5:7150 2023-09-23T04:37:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-23T04:37:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: [23/Sep/2023:04:37:24] ENGINE Serving on http://172.21.15.5:8765 2023-09-23T04:37:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: [23/Sep/2023:04:37:24] ENGINE Bus STARTED 2023-09-23T04:37:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:25 smithi005 ceph-mon[97638]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-23T04:37:26.461 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-23T04:37:26.461 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.615+0000 7fc956d7b700 1 Processor -- start 2023-09-23T04:37:26.461 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.615+0000 7fc956d7b700 1 -- start start 2023-09-23T04:37:26.461 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc956d7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:26.461 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc956d7b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950109240 con 0x7fc950108850 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc954b17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc954b17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc954b17700 1 -- 172.21.15.5:0/429467881 learned_addr learned my addr 172.21.15.5:0/429467881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.616+0000 7fc954b17700 1 -- 172.21.15.5:0/429467881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950109a50 con 0x7fc950108850 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.617+0000 7fc954b17700 1 --2- 172.21.15.5:0/429467881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9440098a0 tx=0x7fc944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df35192eb149056 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.617+0000 7fc94f7fe700 1 -- 172.21.15.5:0/429467881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc944004160 con 0x7fc950108850 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.617+0000 7fc94f7fe700 1 -- 172.21.15.5:0/429467881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9440042c0 con 0x7fc950108850 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.617+0000 7fc94f7fe700 1 -- 172.21.15.5:0/429467881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9440119b0 con 0x7fc950108850 2023-09-23T04:37:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 -- 172.21.15.5:0/429467881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 msgr2=0x7fc950108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 --2- 172.21.15.5:0/429467881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc9440098a0 tx=0x7fc944009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 -- 172.21.15.5:0/429467881 shutdown_connections 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 --2- 172.21.15.5:0/429467881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc950108c70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 -- 172.21.15.5:0/429467881 >> 172.21.15.5:0/429467881 conn(0x7fc95007bd80 msgr2=0x7fc9501064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.618+0000 7fc956d7b700 1 -- 172.21.15.5:0/429467881 shutdown_connections 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.619+0000 7fc956d7b700 1 -- 172.21.15.5:0/429467881 wait complete. 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc956d7b700 1 Processor -- start 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc956d7b700 1 -- start start 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc956d7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:26.463 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc956d7b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95019c980 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc954b17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc954b17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44618/0 (socket says 172.21.15.5:44618) 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.620+0000 7fc954b17700 1 -- 172.21.15.5:0/4262456550 learned_addr learned my addr 172.21.15.5:0/4262456550 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc954b17700 1 -- 172.21.15.5:0/4262456550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009550 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc954b17700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc944000c00 tx=0x7fc944011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9440039f0 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc944003b50 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95019cb80 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.621+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc94401a780 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.622+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95019cfa0 con 0x7fc950108850 2023-09-23T04:37:26.464 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.623+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950062450 con 0x7fc950108850 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.623+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 11) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fc944024440 con 0x7fc950108850 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.623+0000 7fc94dffb700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 0x7fc94003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.623+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc94404c880 con 0x7fc950108850 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.624+0000 7fc94ffff700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 0x7fc94003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.624+0000 7fc94ffff700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 0x7fc94003ad70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006fd0 tx=0x7fc93c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.630+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc9440246f0 con 0x7fc950108850 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.851+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9501064f0 con 0x7fc9400388b0 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.955+0000 7fc94dffb700 1 -- 172.21.15.5:0/4262456550 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc9501064f0 con 0x7fc9400388b0 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 msgr2=0x7fc94003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:26.465 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 0x7fc94003ad70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006fd0 tx=0x7fc93c006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 msgr2=0x7fc95019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc944000c00 tx=0x7fc944011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 shutdown_connections 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400388b0 0x7fc94003ad70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 --2- 172.21.15.5:0/4262456550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc950108850 0x7fc95019c440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 >> 172.21.15.5:0/4262456550 conn(0x7fc95007bd80 msgr2=0x7fc950105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 shutdown_connections 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:25.958+0000 7fc956d7b700 1 -- 172.21.15.5:0/4262456550 wait complete. 2023-09-23T04:37:26.466 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2023-09-23T04:37:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:26 smithi005 ceph-mon[97638]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:27.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:26 smithi005 ceph-mon[97638]: mgrmap e12: x(active, since 2s) 2023-09-23T04:37:27.947 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$TQUY/yHnejPF2YwlX/AfaeGQRkylW4ZvaXEp2hJS.DGm.qt3TUDrW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1695443847, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.996+0000 7fbe39660700 1 Processor -- start 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.996+0000 7fbe39660700 1 -- start start 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe39660700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe39660700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe34107050 con 0x7fbe34106660 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44634/0 (socket says 172.21.15.5:44634) 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe32ffd700 1 -- 172.21.15.5:0/2857047955 learned_addr learned my addr 172.21.15.5:0/2857047955 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.997+0000 7fbe32ffd700 1 -- 172.21.15.5:0/2857047955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34107860 con 0x7fbe34106660 2023-09-23T04:37:27.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.998+0000 7fbe32ffd700 1 --2- 172.21.15.5:0/2857047955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe240098a0 tx=0x7fbe24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3ccc9532440048 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.998+0000 7fbe31ffb700 1 -- 172.21.15.5:0/2857047955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe24004160 con 0x7fbe34106660 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.998+0000 7fbe31ffb700 1 -- 172.21.15.5:0/2857047955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe240042c0 con 0x7fbe34106660 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.998+0000 7fbe31ffb700 1 -- 172.21.15.5:0/2857047955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe240119b0 con 0x7fbe34106660 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 -- 172.21.15.5:0/2857047955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 msgr2=0x7fbe34106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 --2- 172.21.15.5:0/2857047955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe240098a0 tx=0x7fbe24009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 -- 172.21.15.5:0/2857047955 shutdown_connections 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 --2- 172.21.15.5:0/2857047955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe34106a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 -- 172.21.15.5:0/2857047955 >> 172.21.15.5:0/2857047955 conn(0x7fbe34101c40 msgr2=0x7fbe34104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 -- 172.21.15.5:0/2857047955 shutdown_connections 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:26.999+0000 7fbe39660700 1 -- 172.21.15.5:0/2857047955 wait complete. 2023-09-23T04:37:27.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.000+0000 7fbe39660700 1 Processor -- start 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.000+0000 7fbe39660700 1 -- start start 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe39660700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe39660700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3419a780 con 0x7fbe34106660 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe32ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44650/0 (socket says 172.21.15.5:44650) 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe32ffd700 1 -- 172.21.15.5:0/2165485083 learned_addr learned my addr 172.21.15.5:0/2165485083 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.001+0000 7fbe32ffd700 1 -- 172.21.15.5:0/2165485083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24009550 con 0x7fbe34106660 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe32ffd700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:27.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe240039f0 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe24003b50 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3419a980 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe2401a780 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.002+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3419ada0 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.003+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7fbe24024440 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.003+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe34062450 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.004+0000 7fbe1bfff700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 0x7fbe1c03aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.004+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe2404cb10 con 0x7fbe34106660 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.004+0000 7fbe327fc700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 0x7fbe1c03aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:27.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.005+0000 7fbe327fc700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 0x7fbe1c03aea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe28006fd0 tx=0x7fbe28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.009+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbe240189a0 con 0x7fbe34106660 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.238+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbe34062270 con 0x7fbe1c0389e0 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.486+0000 7fbe1bfff700 1 -- 172.21.15.5:0/2165485083 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fbe34062270 con 0x7fbe1c0389e0 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 msgr2=0x7fbe1c03aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 0x7fbe1c03aea0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe28006fd0 tx=0x7fbe28006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 msgr2=0x7fbe3419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 shutdown_connections 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbe1c0389e0 0x7fbe1c03aea0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 --2- 172.21.15.5:0/2165485083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe34106660 0x7fbe3419a240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:27.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 >> 172.21.15.5:0/2165485083 conn(0x7fbe34101c40 msgr2=0x7fbe34102920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:27.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 shutdown_connections 2023-09-23T04:37:27.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:27.488+0000 7fbe39660700 1 -- 172.21.15.5:0/2165485083 wait complete. 2023-09-23T04:37:27.953 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2023-09-23T04:37:28.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:28 smithi005 ceph-mon[97638]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:29.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.504+0000 7f4c1c457700 1 Processor -- start 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.504+0000 7f4c1c457700 1 -- start start 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.504+0000 7f4c1c457700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.504+0000 7f4c1c457700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14107050 con 0x7f4c14106660 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.505+0000 7f4c1a1f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.505+0000 7f4c1a1f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44658/0 (socket says 172.21.15.5:44658) 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.505+0000 7f4c1a1f3700 1 -- 172.21.15.5:0/3100866786 learned_addr learned my addr 172.21.15.5:0/3100866786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.505+0000 7f4c1a1f3700 1 -- 172.21.15.5:0/3100866786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14107860 con 0x7f4c14106660 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.505+0000 7f4c1a1f3700 1 --2- 172.21.15.5:0/3100866786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098a0 tx=0x7f4c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129597546bae50a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.506+0000 7f4c191f1700 1 -- 172.21.15.5:0/3100866786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c10004160 con 0x7f4c14106660 2023-09-23T04:37:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.506+0000 7f4c191f1700 1 -- 172.21.15.5:0/3100866786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c100042c0 con 0x7f4c14106660 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.506+0000 7f4c191f1700 1 -- 172.21.15.5:0/3100866786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c100119b0 con 0x7f4c14106660 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.506+0000 7f4c1c457700 1 -- 172.21.15.5:0/3100866786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 msgr2=0x7f4c14106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.506+0000 7f4c1c457700 1 --2- 172.21.15.5:0/3100866786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c100098a0 tx=0x7f4c10009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.507+0000 7f4c1c457700 1 -- 172.21.15.5:0/3100866786 shutdown_connections 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.507+0000 7f4c1c457700 1 --2- 172.21.15.5:0/3100866786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c14106a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.507+0000 7f4c1c457700 1 -- 172.21.15.5:0/3100866786 >> 172.21.15.5:0/3100866786 conn(0x7f4c14101c40 msgr2=0x7f4c14104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.507+0000 7f4c1c457700 1 -- 172.21.15.5:0/3100866786 shutdown_connections 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.507+0000 7f4c1c457700 1 -- 172.21.15.5:0/3100866786 wait complete. 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.508+0000 7f4c1c457700 1 Processor -- start 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.508+0000 7f4c1c457700 1 -- start start 2023-09-23T04:37:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.508+0000 7f4c1c457700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.508+0000 7f4c1c457700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419a740 con 0x7f4c14106660 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.509+0000 7f4c1a1f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.509+0000 7f4c1a1f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44668/0 (socket says 172.21.15.5:44668) 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.509+0000 7f4c1a1f3700 1 -- 172.21.15.5:0/2879630919 learned_addr learned my addr 172.21.15.5:0/2879630919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.509+0000 7f4c1a1f3700 1 -- 172.21.15.5:0/2879630919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009550 con 0x7f4c14106660 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.509+0000 7f4c1a1f3700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.510+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c100039f0 con 0x7f4c14106660 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.510+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c10003b50 con 0x7f4c14106660 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.510+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c1001a780 con 0x7f4c14106660 2023-09-23T04:37:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.510+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1419a940 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.510+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1419ad60 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.511+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f4c10024440 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.511+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1410aa90 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.512+0000 7f4c0b7fe700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 0x7f4c0003aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.512+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4c1004cb80 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.512+0000 7f4c199f2700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 0x7f4c0003aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.513+0000 7f4c199f2700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 0x7f4c0003aea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.517+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4c1004ee60 con 0x7f4c14106660 2023-09-23T04:37:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.738+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f4c1419d7a0 con 0x7f4c14106660 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.738+0000 7f4c0b7fe700 1 -- 172.21.15.5:0/2879630919 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f4c10018db0 con 0x7f4c14106660 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 msgr2=0x7f4c0003aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 0x7f4c0003aea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 msgr2=0x7f4c1419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000c00 tx=0x7f4c10011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 shutdown_connections 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4c000389e0 0x7f4c0003aea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 --2- 172.21.15.5:0/2879630919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c14106660 0x7f4c1419a200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 >> 172.21.15.5:0/2879630919 conn(0x7f4c14101c40 msgr2=0x7f4c141028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 shutdown_connections 2023-09-23T04:37:29.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:28.742+0000 7f4c1c457700 1 -- 172.21.15.5:0/2879630919 wait complete. 2023-09-23T04:37:29.356 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-23T04:37:29.356 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-09-23T04:37:29.373 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-23T04:37:29.373 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-09-23T04:37:29.373 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-09-23T04:37:29.373 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout: Password: 2u5f8fcsq1 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:29.376 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config directory 2023-09-23T04:37:29.377 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2023-09-23T04:37:29.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:29 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2879630919' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-23T04:37:30.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.957+0000 7f8fb9460700 1 Processor -- start 2023-09-23T04:37:30.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.957+0000 7f8fb9460700 1 -- start start 2023-09-23T04:37:30.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.957+0000 7f8fb9460700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:30.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.957+0000 7f8fb9460700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4079dd0 con 0x7f8fb41071a0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb2ffd700 1 -- 172.21.15.5:0/2413222390 learned_addr learned my addr 172.21.15.5:0/2413222390 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb2ffd700 1 -- 172.21.15.5:0/2413222390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4079f10 con 0x7f8fb41071a0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb2ffd700 1 --2- 172.21.15.5:0/2413222390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40098a0 tx=0x7f8fa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606a455e4038d250 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb1ffb700 1 -- 172.21.15.5:0/2413222390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4004160 con 0x7f8fb41071a0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb1ffb700 1 -- 172.21.15.5:0/2413222390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa40042c0 con 0x7f8fb41071a0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.958+0000 7f8fb1ffb700 1 -- 172.21.15.5:0/2413222390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa40119b0 con 0x7f8fb41071a0 2023-09-23T04:37:30.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 -- 172.21.15.5:0/2413222390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 msgr2=0x7f8fb4109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2413222390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40098a0 tx=0x7f8fa4009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 -- 172.21.15.5:0/2413222390 shutdown_connections 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2413222390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb4109590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 -- 172.21.15.5:0/2413222390 >> 172.21.15.5:0/2413222390 conn(0x7f8fb4100b10 msgr2=0x7f8fb4102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 -- 172.21.15.5:0/2413222390 shutdown_connections 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.959+0000 7f8fb9460700 1 -- 172.21.15.5:0/2413222390 wait complete. 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.960+0000 7f8fb9460700 1 Processor -- start 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.960+0000 7f8fb9460700 1 -- start start 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb9460700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb9460700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb419c910 con 0x7f8fb41071a0 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44700/0 (socket says 172.21.15.5:44700) 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb2ffd700 1 -- 172.21.15.5:0/2030968964 learned_addr learned my addr 172.21.15.5:0/2030968964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.961+0000 7f8fb2ffd700 1 -- 172.21.15.5:0/2030968964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4009550 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8fb2ffd700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4011980 tx=0x7f8fa4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa4003a60 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa4003bc0 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa401a670 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb419cb10 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.962+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb419cf30 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.963+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45655+0+0 (secure 0 0 0) 0x7f8fa4024a00 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.963+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb410a840 con 0x7f8fb41071a0 2023-09-23T04:37:30.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.963+0000 7f8f9bfff700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 0x7f8f9c03aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.963+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fa404cbe0 con 0x7f8fb41071a0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.964+0000 7f8fb27fc700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 0x7f8f9c03aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.964+0000 7f8fb27fc700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 0x7f8f9c03aea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006fd0 tx=0x7f8fa8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:29.968+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8fa4024440 con 0x7f8fb41071a0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.175+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8fb419f970 con 0x7f8fb41071a0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.176+0000 7f8f9bfff700 1 -- 172.21.15.5:0/2030968964 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8fa4018350 con 0x7f8fb41071a0 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 msgr2=0x7f8f9c03aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 0x7f8f9c03aea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8006fd0 tx=0x7f8fa8006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 msgr2=0x7f8fb419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:30.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4011980 tx=0x7f8fa4011a60 comp rx=0 tx=0).stop 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 shutdown_connections 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f9c0389e0 0x7f8f9c03aea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 --2- 172.21.15.5:0/2030968964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb41071a0 0x7f8fb419c3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 >> 172.21.15.5:0/2030968964 conn(0x7f8fb4100b10 msgr2=0x7f8fb4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 shutdown_connections 2023-09-23T04:37:30.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:30.178+0000 7f8fb9460700 1 -- 172.21.15.5:0/2030968964 wait complete. 2023-09-23T04:37:31.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:31 smithi005 ceph-mon[97638]: mgrmap e13: x(active, since 6s) 2023-09-23T04:37:31.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.098+0000 7f02f9af5700 1 Processor -- start 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.098+0000 7f02f9af5700 1 -- start start 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.098+0000 7f02f9af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.098+0000 7f02f9af5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f4107050 con 0x7f02f4106660 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.099+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.099+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50890/0 (socket says 172.21.15.5:50890) 2023-09-23T04:37:31.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.099+0000 7f02f37fe700 1 -- 172.21.15.5:0/3182870974 learned_addr learned my addr 172.21.15.5:0/3182870974 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.099+0000 7f02f37fe700 1 -- 172.21.15.5:0/3182870974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f4107860 con 0x7f02f4106660 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.099+0000 7f02f37fe700 1 --2- 172.21.15.5:0/3182870974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02e40098a0 tx=0x7f02e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2508a7a6224b1145 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f27fc700 1 -- 172.21.15.5:0/3182870974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02e4004160 con 0x7f02f4106660 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f27fc700 1 -- 172.21.15.5:0/3182870974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e40042c0 con 0x7f02f4106660 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f27fc700 1 -- 172.21.15.5:0/3182870974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02e40119b0 con 0x7f02f4106660 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 -- 172.21.15.5:0/3182870974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 msgr2=0x7f02f4106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 --2- 172.21.15.5:0/3182870974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02e40098a0 tx=0x7f02e4009bb0 comp rx=0 tx=0).stop 2023-09-23T04:37:31.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 -- 172.21.15.5:0/3182870974 shutdown_connections 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 --2- 172.21.15.5:0/3182870974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f4106a80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 -- 172.21.15.5:0/3182870974 >> 172.21.15.5:0/3182870974 conn(0x7f02f4101c40 msgr2=0x7f02f4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 -- 172.21.15.5:0/3182870974 shutdown_connections 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.100+0000 7f02f9af5700 1 -- 172.21.15.5:0/3182870974 wait complete. 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.101+0000 7f02f9af5700 1 Processor -- start 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f9af5700 1 -- start start 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f9af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f9af5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f419c940 con 0x7f02f4106660 2023-09-23T04:37:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50892/0 (socket says 172.21.15.5:50892) 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.102+0000 7f02f37fe700 1 -- 172.21.15.5:0/1652803602 learned_addr learned my addr 172.21.15.5:0/1652803602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f37fe700 1 -- 172.21.15.5:0/1652803602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4009550 con 0x7f02f4106660 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f37fe700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02e4000c00 tx=0x7f02e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02e40039f0 con 0x7f02f4106660 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e4003b50 con 0x7f02f4106660 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02e401a780 con 0x7f02f4106660 2023-09-23T04:37:31.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f419cb40 con 0x7f02f4106660 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.103+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f419cf60 con 0x7f02f4106660 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.104+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f02e4024440 con 0x7f02f4106660 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.104+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f4062450 con 0x7f02f4106660 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.105+0000 7f02f0ff9700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 0x7f02dc03af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.105+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02e404cbd0 con 0x7f02f4106660 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.105+0000 7f02f2ffd700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 0x7f02dc03af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.106+0000 7f02f2ffd700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 0x7f02dc03af40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02e8006fd0 tx=0x7f02e8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:31.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.110+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f02e4018d90 con 0x7f02f4106660 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.428+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f02f419f9a0 con 0x7f02f4106660 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.431+0000 7f02f0ff9700 1 -- 172.21.15.5:0/1652803602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f02e4018350 con 0x7f02f4106660 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 msgr2=0x7f02dc03af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 0x7f02dc03af40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02e8006fd0 tx=0x7f02e8006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 msgr2=0x7f02f419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02e4000c00 tx=0x7f02e4011a30 comp rx=0 tx=0).stop 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 shutdown_connections 2023-09-23T04:37:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02dc038a80 0x7f02dc03af40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 --2- 172.21.15.5:0/1652803602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f4106660 0x7f02f419c400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 >> 172.21.15.5:0/1652803602 conn(0x7f02f4101c40 msgr2=0x7f02f41028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 shutdown_connections 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-09-23T04:37:31.433+0000 7f02f9af5700 1 -- 172.21.15.5:0/1652803602 wait complete. 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2023-09-23T04:37:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.854 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2023-09-23T04:37:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-23T04:37:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:31.854 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2023-09-23T04:37:31.904 INFO:tasks.cephadm:Fetching config... 2023-09-23T04:37:31.905 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:37:31.905 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-23T04:37:31.950 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-23T04:37:31.950 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:37:31.950 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-23T04:37:32.036 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-23T04:37:32.036 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:37:32.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-09-23T04:37:32.147 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-23T04:37:32.147 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:37:32.147 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-23T04:37:32.232 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-23T04:37:32.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T04:37:32.375 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:37:32.407 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T04:37:32.451 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:37:32.464 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-23T04:37:32.510 INFO:teuthology.orchestra.run.smithi118.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwA996MB497C038fGWE/Wl03agfJsM7Slw2unbgt0si3N7DPXkqmIgEznjFRdDkZIgGhyX6yGD07aDI2xIkZZ4rPxl4vQfp//eClvP36pBMa0VJog1xR1vjdVEC3v+m3ul/s3D9RI+Afw/cIF23s9Sgh0aOMM5gn4LhB5N7h9vxgPnA7d9iI3UDopFOjA/JzStZf8SX0gJBq4ZEIyaCH/YJaGe/gUassIC1e0Ni6BtikM9jic3qoamk6NAbjHVLRWKyprbK0gKUZzlUbLFBuokK+8krbocvNDp7EWpTs4BHMzVARQ++Lhz3vmLA49uJwG71g5lCVACxE/tfnr6yOzncC3ie2KghlyNVf50avWX4Lc0NZOm+9e40Y8JomW2tLuRCHsL2YFvX8Pff1HHMcZmpqn4MYZuZ7r4qBg8Lz1YkDSP4L/9u7LN3hcMPKJTNag006tNV9iEM6ZEB1yhr3Bt/4vdnwLLVsEli9RgYLTOM8zwBDtjDB9YnL/YpVDOkU8= ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:37:32.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-23T04:37:32.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:32 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1652803602' entity='client.admin' 2023-09-23T04:37:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 -- 172.21.15.5:0/3778125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 msgr2=0x7f908c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 --2- 172.21.15.5:0/3778125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c100d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f907c009910 tx=0x7f907c009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 -- 172.21.15.5:0/3778125968 shutdown_connections 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 --2- 172.21.15.5:0/3778125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c100d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 -- 172.21.15.5:0/3778125968 >> 172.21.15.5:0/3778125968 conn(0x7f908c0fbeb0 msgr2=0x7f908c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 -- 172.21.15.5:0/3778125968 shutdown_connections 2023-09-23T04:37:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.953+0000 7f90912d3700 1 -- 172.21.15.5:0/3778125968 wait complete. 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.954+0000 7f90912d3700 1 Processor -- start 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.954+0000 7f90912d3700 1 -- start start 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f90912d3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f90912d3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c198100 con 0x7f908c100930 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f908affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f908affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50922/0 (socket says 172.21.15.5:50922) 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f908affd700 1 -- 172.21.15.5:0/690717745 learned_addr learned my addr 172.21.15.5:0/690717745 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:34.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f908affd700 1 -- 172.21.15.5:0/690717745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c0095f0 con 0x7f908c100930 2023-09-23T04:37:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.955+0000 7f908affd700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f907c004f40 tx=0x7f907c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.956+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f907c01d070 con 0x7f908c100930 2023-09-23T04:37:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.956+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f907c00bc10 con 0x7f908c100930 2023-09-23T04:37:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.956+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f907c003a70 con 0x7f908c100930 2023-09-23T04:37:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.956+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c198300 con 0x7f908c100930 2023-09-23T04:37:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.956+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c1987a0 con 0x7f908c100930 2023-09-23T04:37:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.957+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c191840 con 0x7f908c100930 2023-09-23T04:37:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.957+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f907c003d30 con 0x7f908c100930 2023-09-23T04:37:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.958+0000 7f9073fff700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 0x7f907403ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.958+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f907c04c6b0 con 0x7f908c100930 2023-09-23T04:37:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.958+0000 7f908a7fc700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 0x7f907403ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.959+0000 7f908a7fc700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 0x7f907403ade0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:34.963+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f907c02abf0 con 0x7f908c100930 2023-09-23T04:37:35.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.189+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f908c106850 con 0x7f908c100930 2023-09-23T04:37:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.192+0000 7f9073fff700 1 -- 172.21.15.5:0/690717745 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f907c026070 con 0x7f908c100930 2023-09-23T04:37:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 msgr2=0x7f907403ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 0x7f907403ade0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 msgr2=0x7f908c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f907c004f40 tx=0x7f907c0057c0 comp rx=0 tx=0).stop 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 shutdown_connections 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9074038920 0x7f907403ade0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 --2- 172.21.15.5:0/690717745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f908c100930 0x7f908c197bc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 >> 172.21.15.5:0/690717745 conn(0x7f908c0fbeb0 msgr2=0x7f908c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 shutdown_connections 2023-09-23T04:37:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:35.195+0000 7f90912d3700 1 -- 172.21.15.5:0/690717745 wait complete. 2023-09-23T04:37:35.666 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-23T04:37:35.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-23T04:37:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:36.552 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/690717745' entity='client.admin' 2023-09-23T04:37:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 -- 172.21.15.5:0/3782195413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 msgr2=0x7fbfb4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/3782195413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4100d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009910 tx=0x7fbfb0009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 -- 172.21.15.5:0/3782195413 shutdown_connections 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/3782195413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4100d50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 -- 172.21.15.5:0/3782195413 >> 172.21.15.5:0/3782195413 conn(0x7fbfb40fbeb0 msgr2=0x7fbfb40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 -- 172.21.15.5:0/3782195413 shutdown_connections 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.218+0000 7fbfbb54f700 1 -- 172.21.15.5:0/3782195413 wait complete. 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.219+0000 7fbfbb54f700 1 Processor -- start 2023-09-23T04:37:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.219+0000 7fbfbb54f700 1 -- start start 2023-09-23T04:37:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfbb54f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfbb54f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4075100 con 0x7fbfb4100930 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfb92eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfb92eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50938/0 (socket says 172.21.15.5:50938) 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfb92eb700 1 -- 172.21.15.5:0/762795878 learned_addr learned my addr 172.21.15.5:0/762795878 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.220+0000 7fbfb92eb700 1 -- 172.21.15.5:0/762795878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00095f0 con 0x7fbfb4100930 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfb92eb700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0004f40 tx=0x7fbfb00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb001d070 con 0x7fbfb4100930 2023-09-23T04:37:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb000bc10 con 0x7fbfb4100930 2023-09-23T04:37:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb0003a70 con 0x7fbfb4100930 2023-09-23T04:37:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4073210 con 0x7fbfb4100930 2023-09-23T04:37:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.221+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb40736b0 con 0x7fbfb4100930 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.222+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fbfb0003ce0 con 0x7fbfb4100930 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.222+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4062450 con 0x7fbfb4100930 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.222+0000 7fbfaa7fc700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 0x7fbfa003ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.223+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbfb004c740 con 0x7fbfb4100930 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.223+0000 7fbfb8aea700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 0x7fbfa003ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.223+0000 7fbfb8aea700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 0x7fbfa003ae30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4006fd0 tx=0x7fbfa4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.228+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbfb002abf0 con 0x7fbfb4100930 2023-09-23T04:37:38.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.441+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfb418f150 con 0x7fbfa0038970 2023-09-23T04:37:38.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.445+0000 7fbfaa7fc700 1 -- 172.21.15.5:0/762795878 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbfb418f150 con 0x7fbfa0038970 2023-09-23T04:37:38.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 msgr2=0x7fbfa003ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:38.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 0x7fbfa003ae30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4006fd0 tx=0x7fbfa4006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 msgr2=0x7fbfb4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0004f40 tx=0x7fbfb00057c0 comp rx=0 tx=0).stop 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 shutdown_connections 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbfa0038970 0x7fbfa003ae30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 --2- 172.21.15.5:0/762795878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfb4100930 0x7fbfb4074bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 >> 172.21.15.5:0/762795878 conn(0x7fbfb40fbeb0 msgr2=0x7fbfb40fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 shutdown_connections 2023-09-23T04:37:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:38.447+0000 7fbfbb54f700 1 -- 172.21.15.5:0/762795878 wait complete. 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:38.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:38.921 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi069 2023-09-23T04:37:38.921 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:37:38.921 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.conf 2023-09-23T04:37:38.941 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:37:38.941 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:39.001 INFO:tasks.cephadm:Adding host smithi069 to orchestrator... 2023-09-23T04:37:39.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch host add smithi069 2023-09-23T04:37:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:37:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:39 smithi005 ceph-mon[97638]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:37:40.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:40 smithi005 ceph-mon[97638]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:37:40.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:40 smithi005 ceph-mon[97638]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 -- 172.21.15.5:0/456310773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 msgr2=0x7fc048106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/456310773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048106c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc044009910 tx=0x7fc044009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 -- 172.21.15.5:0/456310773 shutdown_connections 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/456310773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048106c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 -- 172.21.15.5:0/456310773 >> 172.21.15.5:0/456310773 conn(0x7fc0480fc3e0 msgr2=0x7fc0480fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 -- 172.21.15.5:0/456310773 shutdown_connections 2023-09-23T04:37:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.256+0000 7fc04fdc8700 1 -- 172.21.15.5:0/456310773 wait complete. 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.257+0000 7fc04fdc8700 1 Processor -- start 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.257+0000 7fc04fdc8700 1 -- start start 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04fdc8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04fdc8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0481984b0 con 0x7fc0481068b0 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04db64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04db64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58220/0 (socket says 172.21.15.5:58220) 2023-09-23T04:37:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04db64700 1 -- 172.21.15.5:0/4028041646 learned_addr learned my addr 172.21.15.5:0/4028041646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04db64700 1 -- 172.21.15.5:0/4028041646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440095f0 con 0x7fc0481068b0 2023-09-23T04:37:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.258+0000 7fc04db64700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc044004f40 tx=0x7fc044005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.259+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc04401d070 con 0x7fc0481068b0 2023-09-23T04:37:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.259+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc044004370 con 0x7fc0481068b0 2023-09-23T04:37:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.259+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc048198710 con 0x7fc0481068b0 2023-09-23T04:37:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.259+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc044003970 con 0x7fc0481068b0 2023-09-23T04:37:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.259+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc048198b90 con 0x7fc0481068b0 2023-09-23T04:37:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.260+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc048062450 con 0x7fc0481068b0 2023-09-23T04:37:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.260+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fc044003c30 con 0x7fc0481068b0 2023-09-23T04:37:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.261+0000 7fc03effd700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 0x7fc03403ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.261+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc04404c780 con 0x7fc0481068b0 2023-09-23T04:37:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.261+0000 7fc04d363700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 0x7fc03403ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.262+0000 7fc04d363700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 0x7fc03403ade0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc038006fd0 tx=0x7fc038006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.266+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc04402a590 con 0x7fc0481068b0 2023-09-23T04:37:41.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:41.478+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0480008d0 con 0x7fc034038920 2023-09-23T04:37:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:41 smithi005 ceph-mon[97638]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:37:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:42.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:42 smithi005 ceph-mon[97638]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:43.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.882+0000 7fc03effd700 1 -- 172.21.15.5:0/4028041646 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fc0480008d0 con 0x7fc034038920 2023-09-23T04:37:43.883 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi069' with addr '172.21.15.69' 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.884+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 msgr2=0x7fc03403ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.884+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 0x7fc03403ade0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc038006fd0 tx=0x7fc038006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.884+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 msgr2=0x7fc048197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.884+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc044004f40 tx=0x7fc044005020 comp rx=0 tx=0).stop 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 shutdown_connections 2023-09-23T04:37:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc034038920 0x7fc03403ade0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 --2- 172.21.15.5:0/4028041646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0481068b0 0x7fc048197f70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 >> 172.21.15.5:0/4028041646 conn(0x7fc0480fc3e0 msgr2=0x7fc048108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 shutdown_connections 2023-09-23T04:37:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:43.885+0000 7fc04fdc8700 1 -- 172.21.15.5:0/4028041646 wait complete. 2023-09-23T04:37:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:43 smithi005 ceph-mon[97638]: Deploying cephadm binary to smithi069 2023-09-23T04:37:44.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-23T04:37:44.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:44 smithi005 ceph-mon[97638]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:44.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:44.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:44.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:45.065 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:45.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:45 smithi005 ceph-mon[97638]: Added host smithi069 2023-09-23T04:37:45.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:46.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 -- 172.21.15.5:0/2920068784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 msgr2=0x7fb100106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 --2- 172.21.15.5:0/2920068784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100106c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009910 tx=0x7fb0f0009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 -- 172.21.15.5:0/2920068784 shutdown_connections 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 --2- 172.21.15.5:0/2920068784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100106c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 -- 172.21.15.5:0/2920068784 >> 172.21.15.5:0/2920068784 conn(0x7fb1000fc3e0 msgr2=0x7fb1000fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 -- 172.21.15.5:0/2920068784 shutdown_connections 2023-09-23T04:37:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.538+0000 7fb1064d4700 1 -- 172.21.15.5:0/2920068784 wait complete. 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.539+0000 7fb1064d4700 1 Processor -- start 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.539+0000 7fb1064d4700 1 -- start start 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb1064d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb1064d4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001984b0 con 0x7fb1001068b0 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb0fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb0fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58248/0 (socket says 172.21.15.5:58248) 2023-09-23T04:37:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb0fffff700 1 -- 172.21.15.5:0/1691152776 learned_addr learned my addr 172.21.15.5:0/1691152776 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.540+0000 7fb0fffff700 1 -- 172.21.15.5:0/1691152776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00095f0 con 0x7fb1001068b0 2023-09-23T04:37:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb0fffff700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0004f40 tx=0x7fb0f0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f001d070 con 0x7fb1001068b0 2023-09-23T04:37:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f0004370 con 0x7fb1001068b0 2023-09-23T04:37:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb100198710 con 0x7fb1001068b0 2023-09-23T04:37:46.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f0003970 con 0x7fb1001068b0 2023-09-23T04:37:46.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.541+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb100198b90 con 0x7fb1001068b0 2023-09-23T04:37:46.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.542+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fb0f0003c30 con 0x7fb1001068b0 2023-09-23T04:37:46.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.542+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb100062450 con 0x7fb1001068b0 2023-09-23T04:37:46.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.543+0000 7fb0fd7fa700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 0x7fb0e803ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.543+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb0f004c780 con 0x7fb1001068b0 2023-09-23T04:37:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.543+0000 7fb0ff7fe700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 0x7fb0e803ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.544+0000 7fb0ff7fe700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 0x7fb0e803ade0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006fd0 tx=0x7fb0f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.548+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb0f002a590 con 0x7fb1001068b0 2023-09-23T04:37:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.764+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1000008d0 con 0x7fb0e8038920 2023-09-23T04:37:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.766+0000 7fb0fd7fa700 1 -- 172.21.15.5:0/1691152776 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7fb1000008d0 con 0x7fb0e8038920 2023-09-23T04:37:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}] 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 msgr2=0x7fb0e803ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 0x7fb0e803ade0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4006fd0 tx=0x7fb0f4006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 msgr2=0x7fb100197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0004f40 tx=0x7fb0f0005020 comp rx=0 tx=0).stop 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 shutdown_connections 2023-09-23T04:37:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb0e8038920 0x7fb0e803ade0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 --2- 172.21.15.5:0/1691152776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1001068b0 0x7fb100197f70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 >> 172.21.15.5:0/1691152776 conn(0x7fb1000fc3e0 msgr2=0x7fb100108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 shutdown_connections 2023-09-23T04:37:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:46.768+0000 7fb1064d4700 1 -- 172.21.15.5:0/1691152776 wait complete. 2023-09-23T04:37:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:46 smithi005 ceph-mon[97638]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:47.240 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi118 2023-09-23T04:37:47.240 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:37:47.241 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/etc/ceph/ceph.conf 2023-09-23T04:37:47.261 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:37:47.261 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:47.320 INFO:tasks.cephadm:Adding host smithi118 to orchestrator... 2023-09-23T04:37:47.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch host add smithi118 2023-09-23T04:37:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:48.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:48 smithi005 ceph-mon[97638]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:37:48.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:48 smithi005 ceph-mon[97638]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 -- 172.21.15.5:0/2025800748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 msgr2=0x7fbf44106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 --2- 172.21.15.5:0/2025800748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44106c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009910 tx=0x7fbf34009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 -- 172.21.15.5:0/2025800748 shutdown_connections 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 --2- 172.21.15.5:0/2025800748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44106c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 -- 172.21.15.5:0/2025800748 >> 172.21.15.5:0/2025800748 conn(0x7fbf440fc3e0 msgr2=0x7fbf440fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 -- 172.21.15.5:0/2025800748 shutdown_connections 2023-09-23T04:37:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.425+0000 7fbf49606700 1 -- 172.21.15.5:0/2025800748 wait complete. 2023-09-23T04:37:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.426+0000 7fbf49606700 1 Processor -- start 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.426+0000 7fbf49606700 1 -- start start 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.426+0000 7fbf49606700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.426+0000 7fbf49606700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441984b0 con 0x7fbf441068b0 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.427+0000 7fbf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.427+0000 7fbf42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58260/0 (socket says 172.21.15.5:58260) 2023-09-23T04:37:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.427+0000 7fbf42ffd700 1 -- 172.21.15.5:0/3353079862 learned_addr learned my addr 172.21.15.5:0/3353079862 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.427+0000 7fbf42ffd700 1 -- 172.21.15.5:0/3353079862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf441068b0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.427+0000 7fbf42ffd700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf34004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.428+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf3401d070 con 0x7fbf441068b0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.428+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf44198710 con 0x7fbf441068b0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.428+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3400bbb0 con 0x7fbf441068b0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.428+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf34003930 con 0x7fbf441068b0 2023-09-23T04:37:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.428+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf44198b90 con 0x7fbf441068b0 2023-09-23T04:37:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.429+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf44062450 con 0x7fbf441068b0 2023-09-23T04:37:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.429+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fbf3400bd20 con 0x7fbf441068b0 2023-09-23T04:37:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.429+0000 7fbf2bfff700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 0x7fbf2c03ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.429+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf3404c760 con 0x7fbf441068b0 2023-09-23T04:37:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.430+0000 7fbf427fc700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 0x7fbf2c03ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.430+0000 7fbf427fc700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 0x7fbf2c03ade0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf38006fd0 tx=0x7fbf38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.434+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbf34029770 con 0x7fbf441068b0 2023-09-23T04:37:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:49.650+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf4410ad30 con 0x7fbf2c038920 2023-09-23T04:37:50.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:50 smithi005 ceph-mon[97638]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: Deploying cephadm binary to smithi118 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:52.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.163+0000 7fbf2bfff700 1 -- 172.21.15.5:0/3353079862 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbf4410ad30 con 0x7fbf2c038920 2023-09-23T04:37:52.164 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi118' with addr '172.21.15.118' 2023-09-23T04:37:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 msgr2=0x7fbf2c03ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 0x7fbf2c03ade0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf38006fd0 tx=0x7fbf38006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 msgr2=0x7fbf44197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf34004ff0 comp rx=0 tx=0).stop 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 shutdown_connections 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf2c038920 0x7fbf2c03ade0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 --2- 172.21.15.5:0/3353079862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf441068b0 0x7fbf44197f70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 >> 172.21.15.5:0/3353079862 conn(0x7fbf440fc3e0 msgr2=0x7fbf44108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 shutdown_connections 2023-09-23T04:37:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:52.165+0000 7fbf49606700 1 -- 172.21.15.5:0/3353079862 wait complete. 2023-09-23T04:37:52.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:52 smithi005 ceph-mon[97638]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:52.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:52.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:37:52.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch host ls --format=json 2023-09-23T04:37:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:53.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:53 smithi005 ceph-mon[97638]: Added host smithi118 2023-09-23T04:37:53.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:53.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.744+0000 7f793760e700 1 -- 172.21.15.5:0/3259660444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 msgr2=0x7f7930106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.744+0000 7f793760e700 1 --2- 172.21.15.5:0/3259660444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930106c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f792c009910 tx=0x7f792c009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.744+0000 7f793760e700 1 -- 172.21.15.5:0/3259660444 shutdown_connections 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.744+0000 7f793760e700 1 --2- 172.21.15.5:0/3259660444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930106c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.744+0000 7f793760e700 1 -- 172.21.15.5:0/3259660444 >> 172.21.15.5:0/3259660444 conn(0x7f79300fc3e0 msgr2=0x7f79300fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.745+0000 7f793760e700 1 -- 172.21.15.5:0/3259660444 shutdown_connections 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.745+0000 7f793760e700 1 -- 172.21.15.5:0/3259660444 wait complete. 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.746+0000 7f793760e700 1 Processor -- start 2023-09-23T04:37:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.746+0000 7f793760e700 1 -- start start 2023-09-23T04:37:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.747+0000 7f793760e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.747+0000 7f793760e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300750a0 con 0x7f79301068b0 2023-09-23T04:37:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.747+0000 7f79353aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.747+0000 7f79353aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46090/0 (socket says 172.21.15.5:46090) 2023-09-23T04:37:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.747+0000 7f79353aa700 1 -- 172.21.15.5:0/1601567130 learned_addr learned my addr 172.21.15.5:0/1601567130 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f79353aa700 1 -- 172.21.15.5:0/1601567130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0095f0 con 0x7f79301068b0 2023-09-23T04:37:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f79353aa700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f792c006010 tx=0x7f792c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c01d070 con 0x7f79301068b0 2023-09-23T04:37:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f792c00bc10 con 0x7f79301068b0 2023-09-23T04:37:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f792c003910 con 0x7f79301068b0 2023-09-23T04:37:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7930073660 con 0x7f79301068b0 2023-09-23T04:37:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.748+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7930073ae0 con 0x7f79301068b0 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.749+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f792c00b710 con 0x7f79301068b0 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.749+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7930104b60 con 0x7f79301068b0 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.750+0000 7f79267fc700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 0x7f791c03ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.750+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f792c04c7a0 con 0x7f79301068b0 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.750+0000 7f7934ba9700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 0x7f791c03ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:54.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.750+0000 7f7934ba9700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 0x7f791c03ae30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7920006fd0 tx=0x7f7920006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.755+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f792c029540 con 0x7f79301068b0 2023-09-23T04:37:54.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:54 smithi005 ceph-mon[97638]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.970+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7930073de0 con 0x7f791c038970 2023-09-23T04:37:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.971+0000 7f79267fc700 1 -- 172.21.15.5:0/1601567130 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f7930073de0 con 0x7f791c038970 2023-09-23T04:37:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:37:54.972 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.118", "hostname": "smithi118", "labels": [], "status": ""}] 2023-09-23T04:37:54.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.973+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 msgr2=0x7f791c03ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.973+0000 7f793760e700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 0x7f791c03ae30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7920006fd0 tx=0x7f7920006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.973+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 msgr2=0x7f7930073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.973+0000 7f793760e700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f792c006010 tx=0x7f792c004ff0 comp rx=0 tx=0).stop 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.973+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 shutdown_connections 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.974+0000 7f793760e700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f791c038970 0x7f791c03ae30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.974+0000 7f793760e700 1 --2- 172.21.15.5:0/1601567130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79301068b0 0x7f7930073120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.974+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 >> 172.21.15.5:0/1601567130 conn(0x7f79300fc3e0 msgr2=0x7f7930108d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.974+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 shutdown_connections 2023-09-23T04:37:54.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:54.974+0000 7f793760e700 1 -- 172.21.15.5:0/1601567130 wait complete. 2023-09-23T04:37:55.450 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-23T04:37:55.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd crush tunables default 2023-09-23T04:37:55.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:55 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:37:56.722 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:56 smithi005 ceph-mon[97638]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:37:56.722 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:56 smithi005 ceph-mon[97638]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.555+0000 7f8a70c0a700 1 -- 172.21.15.5:0/542556764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 msgr2=0x7f8a6c106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.555+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/542556764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c106aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009910 tx=0x7f8a5c009c20 comp rx=0 tx=0).stop 2023-09-23T04:37:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.556+0000 7f8a70c0a700 1 -- 172.21.15.5:0/542556764 shutdown_connections 2023-09-23T04:37:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.556+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/542556764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c106aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.556+0000 7f8a70c0a700 1 -- 172.21.15.5:0/542556764 >> 172.21.15.5:0/542556764 conn(0x7f8a6c0fc1f0 msgr2=0x7f8a6c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.556+0000 7f8a70c0a700 1 -- 172.21.15.5:0/542556764 shutdown_connections 2023-09-23T04:37:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.556+0000 7f8a70c0a700 1 -- 172.21.15.5:0/542556764 wait complete. 2023-09-23T04:37:57.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.557+0000 7f8a70c0a700 1 Processor -- start 2023-09-23T04:37:57.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.557+0000 7f8a70c0a700 1 -- start start 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.557+0000 7f8a70c0a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.557+0000 7f8a70c0a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c198360 con 0x7f8a6c1066c0 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a6b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a6b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46100/0 (socket says 172.21.15.5:46100) 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a6b7fe700 1 -- 172.21.15.5:0/2360004743 learned_addr learned my addr 172.21.15.5:0/2360004743 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a6b7fe700 1 -- 172.21.15.5:0/2360004743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0095f0 con 0x7f8a6c1066c0 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a6b7fe700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006010 tx=0x7f8a5c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.558+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c01d070 con 0x7f8a6c1066c0 2023-09-23T04:37:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.559+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5c00bbb0 con 0x7f8a6c1066c0 2023-09-23T04:37:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.559+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c003930 con 0x7f8a6c1066c0 2023-09-23T04:37:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.559+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c1985c0 con 0x7f8a6c1066c0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.559+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c198a40 con 0x7f8a6c1066c0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.560+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f8a5c00bd20 con 0x7f8a6c1066c0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.560+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c062450 con 0x7f8a6c1066c0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.560+0000 7f8a68ff9700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 0x7f8a5403ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.560+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a5c04c790 con 0x7f8a6c1066c0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.560+0000 7f8a6affd700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 0x7f8a5403ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:37:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.561+0000 7f8a6affd700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 0x7f8a5403ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:37:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.566+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8a5c029360 con 0x7f8a6c1066c0 2023-09-23T04:37:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:57.783+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8a6c10aac0 con 0x7f8a6c1066c0 2023-09-23T04:37:58.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.484+0000 7f8a68ff9700 1 -- 172.21.15.5:0/2360004743 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8a5c026020 con 0x7f8a6c1066c0 2023-09-23T04:37:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 msgr2=0x7f8a5403ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 0x7f8a5403ae30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).stop 2023-09-23T04:37:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 msgr2=0x7f8a6c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:37:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006010 tx=0x7f8a5c004ff0 comp rx=0 tx=0).stop 2023-09-23T04:37:58.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 shutdown_connections 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8a54038970 0x7f8a5403ae30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 --2- 172.21.15.5:0/2360004743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6c1066c0 0x7f8a6c197e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 >> 172.21.15.5:0/2360004743 conn(0x7f8a6c0fc1f0 msgr2=0x7f8a6c108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 shutdown_connections 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:37:58.487+0000 7f8a70c0a700 1 -- 172.21.15.5:0/2360004743 wait complete. 2023-09-23T04:37:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:37:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:37:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:37:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T04:37:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:37:58.966 INFO:tasks.cephadm:Adding mon.a on smithi005 2023-09-23T04:37:58.966 INFO:tasks.cephadm:Adding mon.b on smithi069 2023-09-23T04:37:58.966 INFO:tasks.cephadm:Adding mon.c on smithi118 2023-09-23T04:37:58.967 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c' 2023-09-23T04:37:59.547 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:37:59.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:59 smithi005 ceph-mon[97638]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:37:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:59 smithi005 ceph-mon[97638]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:37:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:59 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T04:37:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:37:59 smithi005 ceph-mon[97638]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 -- 172.21.15.118:0/903955239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 msgr2=0x7fd4b4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 --2- 172.21.15.118:0/903955239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4106c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009910 tx=0x7fd4b0009c20 comp rx=0 tx=0).stop 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 -- 172.21.15.118:0/903955239 shutdown_connections 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 --2- 172.21.15.118:0/903955239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4106c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 -- 172.21.15.118:0/903955239 >> 172.21.15.118:0/903955239 conn(0x7fd4b4075320 msgr2=0x7fd4b4075730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:00.762 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 -- 172.21.15.118:0/903955239 shutdown_connections 2023-09-23T04:38:00.763 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.761+0000 7fd4bc338700 1 -- 172.21.15.118:0/903955239 wait complete. 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4bc338700 1 Processor -- start 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4bc338700 1 -- start start 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4bc338700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4bc338700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4196290 con 0x7fd4b41068b0 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4ba0d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4ba0d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.118:33932/0 (socket says 172.21.15.118:33932) 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4ba0d4700 1 -- 172.21.15.118:0/1844426352 learned_addr learned my addr 172.21.15.118:0/1844426352 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:38:00.764 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.762+0000 7fd4ba0d4700 1 -- 172.21.15.118:0/1844426352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00095f0 con 0x7fd4b41068b0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4ba0d4700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0004f40 tx=0x7fd4b0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4b001d070 con 0x7fd4b41068b0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b41964f0 con 0x7fd4b41068b0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b4196970 con 0x7fd4b41068b0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4b0004370 con 0x7fd4b41068b0 2023-09-23T04:38:00.765 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.763+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4b00038c0 con 0x7fd4b41068b0 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.764+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fd4b0003b30 con 0x7fd4b41068b0 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.764+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b4062450 con 0x7fd4b41068b0 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.764+0000 7fd4ab7fe700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 0x7fd4a003ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.764+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4b004c910 con 0x7fd4b41068b0 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.765+0000 7fd4b98d3700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 0x7fd4a003ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:00.766 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.765+0000 7fd4b98d3700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 0x7fd4a003ade0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:00.771 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.770+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd4b000f460 con 0x7fd4b41068b0 2023-09-23T04:38:00.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:00 smithi005 ceph-mon[97638]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:00.968 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.967+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4b410ae80 con 0x7fd4a0038920 2023-09-23T04:38:00.975 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.975+0000 7fd4ab7fe700 1 -- 172.21.15.118:0/1844426352 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd4b410ae80 con 0x7fd4a0038920 2023-09-23T04:38:00.976 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled mon update... 2023-09-23T04:38:00.977 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 msgr2=0x7fd4a003ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 0x7fd4a003ade0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006fd0 tx=0x7fd4a4006e40 comp rx=0 tx=0).stop 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 msgr2=0x7fd4b4195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0004f40 tx=0x7fd4b0005020 comp rx=0 tx=0).stop 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 shutdown_connections 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd4a0038920 0x7fd4a003ade0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 --2- 172.21.15.118:0/1844426352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b41068b0 0x7fd4b4195d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 >> 172.21.15.118:0/1844426352 conn(0x7fd4b4075320 msgr2=0x7fd4b40fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 shutdown_connections 2023-09-23T04:38:00.978 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:00.977+0000 7fd4bc338700 1 -- 172.21.15.118:0/1844426352 wait complete. 2023-09-23T04:38:01.426 DEBUG:teuthology.orchestra.run.smithi069:mon.b> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.b.service 2023-09-23T04:38:01.429 DEBUG:teuthology.orchestra.run.smithi118:mon.c> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.c.service 2023-09-23T04:38:01.431 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T04:38:01.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T04:38:01.462 INFO:journalctl@ceph.mon.c.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:38:01.465 INFO:journalctl@ceph.mon.b.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:38:02.029 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c;count:3 2023-09-23T04:38:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: Deploying daemon mon.c on smithi118 2023-09-23T04:38:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:01 smithi005 ceph-mon[97638]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:03.885 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 -- 172.21.15.118:0/1466822020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 msgr2=0x7f41500ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:03.885 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 --2- 172.21.15.118:0/1466822020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500ff3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f414c009910 tx=0x7f414c009c20 comp rx=0 tx=0).stop 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 -- 172.21.15.118:0/1466822020 shutdown_connections 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 --2- 172.21.15.118:0/1466822020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500ff3f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 -- 172.21.15.118:0/1466822020 >> 172.21.15.118:0/1466822020 conn(0x7f4150075e80 msgr2=0x7f4150076290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 -- 172.21.15.118:0/1466822020 shutdown_connections 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.884+0000 7f41576cc700 1 -- 172.21.15.118:0/1466822020 wait complete. 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f41576cc700 1 Processor -- start 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f41576cc700 1 -- start start 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f41576cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:03.886 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f41576cc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150101f10 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f4155468700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f4155468700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.118:33938/0 (socket says 172.21.15.118:33938) 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f4155468700 1 -- 172.21.15.118:0/1579758158 learned_addr learned my addr 172.21.15.118:0/1579758158 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.885+0000 7f4155468700 1 -- 172.21.15.118:0/1579758158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c0095f0 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f4155468700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f414c004f40 tx=0x7f414c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f414c01d070 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f414c004370 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41501004d0 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f414c0039b0 con 0x7f41500ff010 2023-09-23T04:38:03.888 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.886+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4150100950 con 0x7f41500ff010 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.887+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f414c003c70 con 0x7f41500ff010 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.887+0000 7f41467fc700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 0x7f413c03ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.887+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f414c04c770 con 0x7f41500ff010 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.887+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4150108e10 con 0x7f41500ff010 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.887+0000 7f4154c67700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 0x7f413c03ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:03.889 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.888+0000 7f4154c67700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 0x7f413c03ade0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4140006fd0 tx=0x7f4140006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:03.893 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:03.893+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f414c026070 con 0x7f41500ff010 2023-09-23T04:38:04.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:03 smithi118 systemd[1]: Starting Ceph mon.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:38:04.188 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.186+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4150100b90 con 0x7f41500ff010 2023-09-23T04:38:04.188 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.187+0000 7f41467fc700 1 -- 172.21.15.118:0/1579758158 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f414c00f460 con 0x7f41500ff010 2023-09-23T04:38:04.188 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:38:04.189 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":1,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","modified":"2023-09-23T04:36:21.998950Z","created":"2023-09-23T04:36:21.998950Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-23T04:38:04.190 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 msgr2=0x7f413c03ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:04.190 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 0x7f413c03ade0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4140006fd0 tx=0x7f4140006e40 comp rx=0 tx=0).stop 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 msgr2=0x7f41500fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f414c004f40 tx=0x7f414c005020 comp rx=0 tx=0).stop 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 shutdown_connections 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f413c038920 0x7f413c03ade0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 --2- 172.21.15.118:0/1579758158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41500ff010 0x7f41500fff90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 >> 172.21.15.118:0/1579758158 conn(0x7f4150075e80 msgr2=0x7f4150104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 shutdown_connections 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:04.189+0000 7f41576cc700 1 -- 172.21.15.118:0/1579758158 wait complete. 2023-09-23T04:38:04.191 INFO:teuthology.orchestra.run.smithi118.stderr:dumped monmap epoch 1 2023-09-23T04:38:04.466 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 podman[134242]: 2023-09-23T04:38:04.852 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pidfile_write: ignore empty --pid-file 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: load: jerasure load: lrc 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: RocksDB version: 7.9.2 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Git sha 0 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: DB SUMMARY 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: DB Session ID: I0PW95YVWC37I5GC5TXV 2023-09-23T04:38:04.853 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: CURRENT file: CURRENT 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.error_if_exists: 0 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.create_if_missing: 0 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.paranoid_checks: 1 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T04:38:04.854 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.env: 0x560bbabaeda0 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.info_log: 0x560bbccb95a0 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.statistics: (nil) 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.use_fsync: 0 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_log_file_size: 0 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T04:38:04.855 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_fallocate: 1 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.use_direct_reads: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.db_log_dir: 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.wal_dir: 2023-09-23T04:38:04.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T04:38:04.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.write_buffer_manager: 0x560bbcc714a0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T04:38:04.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.unordered_write: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.row_cache: None 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.wal_filter: None 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.two_write_queues: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.wal_compression: 0 2023-09-23T04:38:04.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.atomic_flush: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.log_readahead_size: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T04:38:04.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_background_jobs: 2 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_background_compactions: -1 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_subcompactions: 1 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T04:38:04.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_open_files: -1 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_background_flushes: -1 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Compression algorithms supported: 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kZSTD supported: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kXpressCompression supported: 0 2023-09-23T04:38:04.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kZlibCompression supported: 1 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kSnappyCompression supported: 1 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kLZ4Compression supported: 1 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: kBZip2Compression supported: 0 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T04:38:04.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.merge_operator: 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_filter: None 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560bbccb96a0) 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks: 1 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T04:38:04.865 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_top_level_index_and_filter: 1 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_type: 0 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_index_type: 0 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_shortening: 1 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: checksum: 4 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: no_block_cache: 0 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache: 0x560bbcd0f1f0 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_name: BinnedLRUCache 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_options: 2023-09-23T04:38:04.866 INFO:journalctl@ceph.mon.c.smithi118.stdout: capacity : 536870912 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_shard_bits : 4 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: strict_capacity_limit : 0 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: high_pri_pool_ratio: 0.000 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_compressed: (nil) 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: persistent_cache: (nil) 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size: 4096 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size_deviation: 10 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_restart_interval: 16 2023-09-23T04:38:04.867 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_block_restart_interval: 1 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: metadata_block_size: 4096 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: partition_filters: 0 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: use_delta_encoding: 1 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: filter_policy: bloomfilter 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: whole_key_filtering: 1 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: verify_compression: 0 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: read_amp_bytes_per_bit: 0 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: format_version: 5 2023-09-23T04:38:04.868 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_index_compression: 1 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_align: 0 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_auto_readahead_size: 262144 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout: prepopulate_block_cache: 0 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout: initial_auto_readahead_size: 8192 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression: NoCompression 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T04:38:04.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.num_levels: 7 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:38:04.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:38:04.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T04:38:04.872 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T04:38:04.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T04:38:04.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.table_properties_collectors: 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.inplace_update_support: 0 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T04:38:04.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.bloom_locality: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.max_successive_merges: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.ttl: 2592000 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T04:38:04.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enable_blob_files: false 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.min_blob_size: 0 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-23T04:38:04.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 93c7f15b-6740-4148-bdf0-4a96fd85c056 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443884473221, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443884473795, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695443884, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "93c7f15b-6740-4148-bdf0-4a96fd85c056", "db_session_id": "I0PW95YVWC37I5GC5TXV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443884473883, "job": 1, "event": "recovery_finished"} 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560bbcdac000 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: DB pointer 0x560bbcd98000 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T04:38:04.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** DB Stats ** 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2023-09-23T04:38:04.879 INFO:journalctl@ceph.mon.c.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.880 INFO:journalctl@ceph.mon.c.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache BinnedLRUCache@0x560bbcd0f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.881 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: using public_addr v2:172.21.15.118:0/0 -> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: starting mon.c rank -1 at public addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] at bind addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(???) e0 preinit fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).mds e1 new map 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout: e1 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T04:38:04.882 INFO:journalctl@ceph.mon.c.smithi118.stdout: legacy client fscid: -1 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout: No filesystems configured 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:04.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:04.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Deploying cephadm binary to smithi118 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:04.885 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Added host smithi118 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.886 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.887 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c;count:3 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:04.888 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: Deploying daemon mon.c on smithi118 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expand map: {default=false} 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta from 'false' to 'false' 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expanded map: {default=false} 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expand map: {default=info} 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta from 'info' to 'info' 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expanded map: {default=info} 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expand map: {default=daemon} 2023-09-23T04:38:04.889 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expand map: {default=debug} 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: expand_channel_meta expanded map: {default=debug} 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 ceph-mon[134297]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 bash[134242]: ec712ed13761a01bd35cd27222dff9fc3a9634115ec480c33472bc41debc6d92 2023-09-23T04:38:04.890 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:04 smithi118 systemd[1]: Started Ceph mon.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:38:05.892 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T04:38:05.892 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T04:38:06.469 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: mon.a calling monitor election 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: mon.c calling monitor election 2023-09-23T04:38:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: fsmap 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: mgrmap e13: x(active, since 45s) 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:09.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: mon.a calling monitor election 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: mon.c calling monitor election 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:09.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: fsmap 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: mgrmap e13: x(active, since 45s) 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:09.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.680+0000 7fd435971700 1 -- 172.21.15.118:0/1069915487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 msgr2=0x7fd418007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.680+0000 7fd435971700 1 --2- 172.21.15.118:0/1069915487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd418007820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd420005fa0 tx=0x7fd42000ffc0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.680+0000 7fd435971700 1 -- 172.21.15.118:0/1069915487 shutdown_connections 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.680+0000 7fd435971700 1 --2- 172.21.15.118:0/1069915487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd418007820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.680+0000 7fd435971700 1 -- 172.21.15.118:0/1069915487 >> 172.21.15.118:0/1069915487 conn(0x7fd430074b70 msgr2=0x7fd430074f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.681+0000 7fd435971700 1 -- 172.21.15.118:0/1069915487 shutdown_connections 2023-09-23T04:38:10.682 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.681+0000 7fd435971700 1 -- 172.21.15.118:0/1069915487 wait complete. 2023-09-23T04:38:10.684 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.681+0000 7fd435971700 1 Processor -- start 2023-09-23T04:38:10.684 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd435971700 1 -- start start 2023-09-23T04:38:10.684 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd435971700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd430199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:10.684 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd435971700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:10.684 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd435971700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43019a530 con 0x7fd430100440 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd435971700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43019a6a0 con 0x7fd430199ff0 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52154/0 (socket says 172.21.15.118:52154) 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 -- 172.21.15.118:0/165103334 learned_addr learned my addr 172.21.15.118:0/165103334 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 -- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 msgr2=0x7fd430193b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 -- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 msgr2=0x7fd430193b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42effd700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd430199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.682+0000 7fd42e7fc700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:38:10.685 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd42effd700 1 -- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 msgr2=0x7fd430193b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd42effd700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd42effd700 1 -- 172.21.15.118:0/165103334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42000f9c0 con 0x7fd430100440 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd42effd700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd430199ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd420000c00 tx=0x7fd420004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd4200053b0 con 0x7fd430100440 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd430194130 con 0x7fd430100440 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd420021040 con 0x7fd430100440 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd420011470 con 0x7fd430100440 2023-09-23T04:38:10.686 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.683+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430194680 con 0x7fd430100440 2023-09-23T04:38:10.687 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.685+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fd4200116e0 con 0x7fd430100440 2023-09-23T04:38:10.687 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.685+0000 7fd43496f700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 0x7fd41c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:10.687 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.685+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd430062450 con 0x7fd430100440 2023-09-23T04:38:10.687 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.685+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd42004dd60 con 0x7fd430100440 2023-09-23T04:38:10.687 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.685+0000 7fd42e7fc700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 0x7fd41c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:10.688 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.686+0000 7fd42e7fc700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 0x7fd41c03ac20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd424009180 tx=0x7fd424008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:10.692 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.691+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd420019330 con 0x7fd430100440 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: Deploying daemon mon.b on smithi069 2023-09-23T04:38:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:10.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:10.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: Deploying daemon mon.b on smithi069 2023-09-23T04:38:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:10.958 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.958+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd430194980 con 0x7fd430100440 2023-09-23T04:38:10.962 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.959+0000 7fd43496f700 1 -- 172.21.15.118:0/165103334 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7fd420010bc0 con 0x7fd430100440 2023-09-23T04:38:10.962 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:38:10.962 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":2,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","modified":"2023-09-23T04:38:04.499425Z","created":"2023-09-23T04:36:21.998950Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-23T04:38:10.963 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.960+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 msgr2=0x7fd41c03ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:10.963 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.960+0000 7fd435971700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 0x7fd41c03ac20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd424009180 tx=0x7fd424008040 comp rx=0 tx=0).stop 2023-09-23T04:38:10.963 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.960+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 msgr2=0x7fd430199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.960+0000 7fd435971700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd430199ab0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd420000c00 tx=0x7fd420004cb0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 shutdown_connections 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd41c038760 0x7fd41c03ac20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd430100440 0x7fd430199ab0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 --2- 172.21.15.118:0/165103334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd430199ff0 0x7fd430193b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 >> 172.21.15.118:0/165103334 conn(0x7fd430074b70 msgr2=0x7fd430103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:10.964 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 shutdown_connections 2023-09-23T04:38:10.965 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:10.961+0000 7fd435971700 1 -- 172.21.15.118:0/165103334 wait complete. 2023-09-23T04:38:10.965 INFO:teuthology.orchestra.run.smithi118.stderr:dumped monmap epoch 2 2023-09-23T04:38:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:11 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/165103334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:11 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/165103334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 systemd[1]: Starting Ceph mon.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:38:12.319 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-23T04:38:12.319 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph mon dump -f json 2023-09-23T04:38:12.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 podman[133891]: 2023-09-23T04:38:12.585 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: set uid:gid to 167:167 (ceph:ceph) 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev), process ceph-mon, pid 7 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pidfile_write: ignore empty --pid-file 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: load: jerasure load: lrc 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: RocksDB version: 7.9.2 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Git sha 0 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Compile date 2023-09-22 16:46:33 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: DB SUMMARY 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: DB Session ID: 1ZFUOO2O49SNX900LWIM 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: CURRENT file: CURRENT 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: IDENTITY file: IDENTITY 2023-09-23T04:38:12.586 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.error_if_exists: 0 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.create_if_missing: 0 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.paranoid_checks: 1 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.env: 0x560dcea2ada0 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.fs: PosixFileSystem 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.info_log: 0x560dd0a135a0 2023-09-23T04:38:12.587 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_file_opening_threads: 16 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.statistics: (nil) 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.use_fsync: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_log_file_size: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.keep_log_file_num: 1000 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.recycle_log_file_num: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_fallocate: 1 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_mmap_reads: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_mmap_writes: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.use_direct_reads: 0 2023-09-23T04:38:12.588 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.create_missing_column_families: 0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.db_log_dir: 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.wal_dir: 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.advise_random_on_open: 1 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.db_write_buffer_size: 0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.write_buffer_manager: 0x560dd09cb4a0 2023-09-23T04:38:12.589 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.rate_limiter: (nil) 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.wal_recovery_mode: 2 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enable_thread_tracking: 0 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enable_pipelined_write: 0 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.unordered_write: 0 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-23T04:38:12.590 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.row_cache: None 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.wal_filter: None 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_ingest_behind: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.two_write_queues: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.manual_wal_flush: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.wal_compression: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.atomic_flush: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-23T04:38:12.591 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.log_readahead_size: 0 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.best_efforts_recovery: 0 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.allow_data_in_errors: 0 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.db_host_id: __hostname__ 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_background_jobs: 2 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_background_compactions: -1 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_subcompactions: 1 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-23T04:38:12.592 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_total_wal_size: 0 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_open_files: -1 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bytes_per_sync: 0 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_readahead_size: 0 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_background_flushes: -1 2023-09-23T04:38:12.593 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Compression algorithms supported: 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kZSTD supported: 0 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kXpressCompression supported: 0 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kLZ4HCCompression supported: 1 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kZlibCompression supported: 1 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kSnappyCompression supported: 1 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kLZ4Compression supported: 1 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: kBZip2Compression supported: 0 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-23T04:38:12.594 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.merge_operator: 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_filter: None 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_filter_factory: None 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.sst_partitioner_factory: None 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560dd0a136a0) 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout: cache_index_and_filter_blocks: 1 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-23T04:38:12.595 INFO:journalctl@ceph.mon.b.smithi069.stdout: pin_top_level_index_and_filter: 1 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_type: 0 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: data_block_index_type: 0 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_shortening: 1 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: checksum: 4 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: no_block_cache: 0 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache: 0x560dd0a691f0 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_name: BinnedLRUCache 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_options: 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: capacity : 536870912 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: num_shard_bits : 4 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: strict_capacity_limit : 0 2023-09-23T04:38:12.596 INFO:journalctl@ceph.mon.b.smithi069.stdout: high_pri_pool_ratio: 0.000 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_cache_compressed: (nil) 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: persistent_cache: (nil) 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_size: 4096 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_size_deviation: 10 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_restart_interval: 16 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: index_block_restart_interval: 1 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: metadata_block_size: 4096 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: partition_filters: 0 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: use_delta_encoding: 1 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: filter_policy: bloomfilter 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: whole_key_filtering: 1 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: verify_compression: 0 2023-09-23T04:38:12.597 INFO:journalctl@ceph.mon.b.smithi069.stdout: read_amp_bytes_per_bit: 0 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: format_version: 5 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: enable_index_compression: 1 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: block_align: 0 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: max_auto_readahead_size: 262144 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: prepopulate_block_cache: 0 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: initial_auto_readahead_size: 8192 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.write_buffer_size: 33554432 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_write_buffer_number: 2 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression: NoCompression 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression: Disabled 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.prefix_extractor: nullptr 2023-09-23T04:38:12.598 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.num_levels: 7 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 bash[133891]: 001b0d2f62c2289fa9fe292d9cdf0d4f5777b7a9cc31d4a03791f61722bc1fa6 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-23T04:38:12.599 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.level: 32767 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.strategy: 0 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.enabled: false 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-23T04:38:12.600 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.target_file_size_base: 67108864 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-23T04:38:12.601 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.arena_block_size: 1048576 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-23T04:38:12.602 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.disable_auto_compactions: 0 2023-09-23T04:38:12.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-23T04:38:12.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-23T04:38:12.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-23T04:38:12.603 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.table_properties_collectors: 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.inplace_update_support: 0 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-23T04:38:12.604 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.bloom_locality: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.max_successive_merges: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.paranoid_file_checks: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.force_consistency_checks: 1 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.report_bg_io_stats: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.ttl: 2592000 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enable_blob_files: false 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.min_blob_size: 0 2023-09-23T04:38:12.605 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_file_size: 268435456 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.blob_file_starting_level: 0 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2d093fa9-0c1e-407f-9283-65ae892b6511 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443892517329, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-23T04:38:12.606 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443892517727, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695443892, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2d093fa9-0c1e-407f-9283-65ae892b6511", "db_session_id": "1ZFUOO2O49SNX900LWIM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695443892517793, "job": 1, "event": "recovery_finished"} 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560dd0b06000 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: DB pointer 0x560dd0af2000 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** DB Stats ** 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:12.607 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** Compaction Stats [default] ** 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T04:38:12.608 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** Compaction Stats [default] ** 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-23T04:38:12.609 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: Cumulative compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: Interval compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: Block cache BinnedLRUCache@0x560dd0a691f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: using public_addr v2:172.21.15.69:0/0 -> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: starting mon.b rank -1 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:12.610 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(???) e0 preinit fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).mds e1 new map 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: e1 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: legacy client fscid: -1 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout: No filesystems configured 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-23T04:38:12.611 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.612 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Deploying cephadm binary to smithi118 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.613 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Added host smithi118 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.614 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.615 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2360004743' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c;count:3 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:12.616 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Deploying daemon mon.c on smithi118 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.a calling monitor election 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.c calling monitor election 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.617 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: fsmap 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mgrmap e13: x(active, since 45s) 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:12.618 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: Deploying daemon mon.b on smithi069 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/165103334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expand map: {default=false} 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta from 'false' to 'false' 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expanded map: {default=false} 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expand map: {default=info} 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta from 'info' to 'info' 2023-09-23T04:38:12.619 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expanded map: {default=info} 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expand map: {default=daemon} 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expanded map: {default=daemon} 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expand map: {default=debug} 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta from 'debug' to 'debug' 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 ceph-mon[133931]: expand_channel_meta expanded map: {default=debug} 2023-09-23T04:38:12.620 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:12 smithi069 systemd[1]: Started Ceph mon.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:38:12.877 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:38:17.633 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.632+0000 7f5ed844e700 1 -- 172.21.15.118:0/3405051875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed01058d0 msgr2=0x7f5ebc007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.632+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3405051875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed01058d0 0x7f5ebc007820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0005fa0 tx=0x7f5ec000f4a0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.632+0000 7f5ed844e700 1 -- 172.21.15.118:0/3405051875 shutdown_connections 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.632+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3405051875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed01058d0 0x7f5ebc007820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.632+0000 7f5ed844e700 1 -- 172.21.15.118:0/3405051875 >> 172.21.15.118:0/3405051875 conn(0x7f5ed00fb3e0 msgr2=0x7f5ed00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.633+0000 7f5ed844e700 1 -- 172.21.15.118:0/3405051875 shutdown_connections 2023-09-23T04:38:17.634 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.633+0000 7f5ed844e700 1 -- 172.21.15.118:0/3405051875 wait complete. 2023-09-23T04:38:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.633+0000 7f5ed844e700 1 Processor -- start 2023-09-23T04:38:17.636 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.633+0000 7f5ed844e700 1 -- start start 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 0x7f5ed019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0198d70 con 0x7f5ed019d030 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed01a4da0 con 0x7f5ed01058d0 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed844e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed01a4ee0 con 0x7f5ed0198680 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52176/0 (socket says 172.21.15.118:52176) 2023-09-23T04:38:17.637 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 -- 172.21.15.118:0/3792664809 learned_addr learned my addr 172.21.15.118:0/3792664809 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed69eb700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 0x7f5ed019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed61ea700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 msgr2=0x7f5ed019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 msgr2=0x7f5ed019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed59e9700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:38:17.638 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed61ea700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 msgr2=0x7f5ed0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed61ea700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 msgr2=0x7f5ed0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed61ea700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.634+0000 7f5ed61ea700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 msgr2=0x7f5ed019caf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 msgr2=0x7f5ed0198140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 -- 172.21.15.118:0/3792664809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00149d0 con 0x7f5ed019d030 2023-09-23T04:38:17.639 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed69eb700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 0x7f5ed019d4e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800bd50 tx=0x7f5ec800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:17.640 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800cd10 con 0x7f5ed019d030 2023-09-23T04:38:17.640 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ec800ce70 con 0x7f5ed019d030 2023-09-23T04:38:17.640 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8012840 con 0x7f5ed019d030 2023-09-23T04:38:17.640 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed01a51c0 con 0x7f5ed019d030 2023-09-23T04:38:17.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.635+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed01a56c0 con 0x7f5ed019d030 2023-09-23T04:38:17.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.636+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f5ec80129a0 con 0x7f5ed019d030 2023-09-23T04:38:17.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.636+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed0062450 con 0x7f5ed019d030 2023-09-23T04:38:17.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.636+0000 7f5ec77fe700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 0x7f5eb403b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:17.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.636+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ec804afb0 con 0x7f5ed019d030 2023-09-23T04:38:17.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.637+0000 7f5ed61ea700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 0x7f5eb403b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:17.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.637+0000 7f5ed61ea700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 0x7f5eb403b260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:17.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.642+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5ec801d3b0 con 0x7f5ed019d030 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: mon.c calling monitor election 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: mon.a calling monitor election 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: fsmap 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: mgrmap e13: x(active, since 54s) 2023-09-23T04:38:17.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T04:38:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T04:38:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T04:38:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: mon.b (rank 2) addr [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] is down (out of quorum) 2023-09-23T04:38:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: mon.c calling monitor election 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: mon.a calling monitor election 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: fsmap 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: mgrmap e13: x(active, since 54s) 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T04:38:17.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T04:38:17.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T04:38:17.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: mon.b (rank 2) addr [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] is down (out of quorum) 2023-09-23T04:38:17.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:17.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:17.921 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.921+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5ed0106870 con 0x7f5ed019d030 2023-09-23T04:38:17.922 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.921+0000 7f5ec77fe700 1 -- 172.21.15.118:0/3792664809 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1266 (secure 0 0 0) 0x7f5ec80112a0 con 0x7f5ed019d030 2023-09-23T04:38:17.922 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-23T04:38:17.922 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":3,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","modified":"2023-09-23T04:38:12.548383Z","created":"2023-09-23T04:36:21.998950Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 msgr2=0x7f5eb403b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 0x7f5eb403b260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0016000 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 msgr2=0x7f5ed019d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 0x7f5ed019d4e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800bd50 tx=0x7f5ec800be30 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 shutdown_connections 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5eb4038da0 0x7f5eb403b260 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed019d030 0x7f5ed019d4e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ed01058d0 0x7f5ed0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 --2- 172.21.15.118:0/3792664809 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5ed0198680 0x7f5ed019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:17.926 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 >> 172.21.15.118:0/3792664809 conn(0x7f5ed00fb3e0 msgr2=0x7f5ed0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:17.927 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 shutdown_connections 2023-09-23T04:38:17.927 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:17.925+0000 7f5ed844e700 1 -- 172.21.15.118:0/3792664809 wait complete. 2023-09-23T04:38:17.927 INFO:teuthology.orchestra.run.smithi118.stderr:dumped monmap epoch 3 2023-09-23T04:38:18.334 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-23T04:38:18.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph config generate-minimal-conf 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/3792664809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/3792664809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: mon.c calling monitor election 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: mon.a calling monitor election 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:38:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: fsmap 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: mgrmap e13: x(active, since 54s) 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: mon.b (rank 2) addr [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] is down (out of quorum) 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/3792664809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-23T04:38:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:19.994 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.504+0000 7fe10b8ef700 1 -- 172.21.15.5:0/3015351411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe104073250 msgr2=0x7fe104073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.504+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/3015351411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe104073250 0x7fe104073630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe100009910 tx=0x7fe100009c20 comp rx=0 tx=0).stop 2023-09-23T04:38:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.504+0000 7fe10b8ef700 1 -- 172.21.15.5:0/3015351411 shutdown_connections 2023-09-23T04:38:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.504+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/3015351411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe104073250 0x7fe104073630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.504+0000 7fe10b8ef700 1 -- 172.21.15.5:0/3015351411 >> 172.21.15.5:0/3015351411 conn(0x7fe1040fc3e0 msgr2=0x7fe1040fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.505+0000 7fe10b8ef700 1 -- 172.21.15.5:0/3015351411 shutdown_connections 2023-09-23T04:38:20.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.505+0000 7fe10b8ef700 1 -- 172.21.15.5:0/3015351411 wait complete. 2023-09-23T04:38:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 Processor -- start 2023-09-23T04:38:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 -- start start 2023-09-23T04:38:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:20.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 0x7fe10419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe104198de0 con 0x7fe1041986f0 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1041a4e10 con 0x7fe104073250 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.506+0000 7fe10b8ef700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1041a4f50 con 0x7fe10419d0a0 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe10968b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50882/0 (socket says 172.21.15.5:50882) 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe10968b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:42834/0 (socket says 172.21.15.5:42834) 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 -- 172.21.15.5:0/2147101340 learned_addr learned my addr 172.21.15.5:0/2147101340 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:38:20.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe109e8c700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 0x7fe10419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 -- 172.21.15.5:0/2147101340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 msgr2=0x7fe10419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 0x7fe10419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 -- 172.21.15.5:0/2147101340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 msgr2=0x7fe1041981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe108e8a700 1 -- 172.21.15.5:0/2147101340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000095f0 con 0x7fe1041986f0 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe109e8c700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 0x7fe10419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:38:20.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.507+0000 7fe10968b700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe108e8a700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b780 tx=0x7fe0f400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4004230 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1041a5230 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f4004390 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4011b90 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.508+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1041a5780 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.510+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10410a550 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.510+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fe0f40036a0 con 0x7fe1041986f0 2023-09-23T04:38:20.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.510+0000 7fe0fa7fc700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 0x7fe0f003b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:20.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.511+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe0f404a230 con 0x7fe1041986f0 2023-09-23T04:38:20.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.511+0000 7fe10968b700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 0x7fe0f003b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:20.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.511+0000 7fe10968b700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 0x7fe0f003b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe100009fd0 tx=0x7fe1000058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:20.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.517+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe0f401a3b0 con 0x7fe1041986f0 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mon.b calling monitor election 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mon.b calling monitor election 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mon.c calling monitor election 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mon.a calling monitor election 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T04:38:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: fsmap 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: mgrmap e13: x(active, since 56s) 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.729+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe104062450 con 0x7fe1041986f0 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.730+0000 7fe0fa7fc700 1 -- 172.21.15.5:0/2147101340 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7fe0f4018e60 con 0x7fe1041986f0 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:38:20.730 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.732+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 msgr2=0x7fe0f003b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.732+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 0x7fe0f003b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe100009fd0 tx=0x7fe1000058e0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.732+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 msgr2=0x7fe10419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.732+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b780 tx=0x7fe0f400ba90 comp rx=0 tx=0).stop 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 shutdown_connections 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0f0038df0 0x7fe0f003b2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe10419d0a0 0x7fe10419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104073250 0x7fe1041981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 --2- 172.21.15.5:0/2147101340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1041986f0 0x7fe10419cb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 >> 172.21.15.5:0/2147101340 conn(0x7fe1040fc3e0 msgr2=0x7fe104107690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 shutdown_connections 2023-09-23T04:38:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:20.733+0000 7fe10b8ef700 1 -- 172.21.15.5:0/2147101340 wait complete. 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mon.b calling monitor election 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mon.b calling monitor election 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mon.c calling monitor election 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mon.a calling monitor election 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T04:38:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: fsmap 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: mgrmap e13: x(active, since 56s) 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.743 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mon.b calling monitor election 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:20.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mon.b calling monitor election 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mon.c calling monitor election 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mon.a calling monitor election 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0],c=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: fsmap 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: osdmap e4: 0 total, 0 up, 0 in 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: mgrmap e13: x(active, since 56s) 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:20.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:20.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.177 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-23T04:38:21.178 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:38:21.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T04:38:21.254 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:38:21.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:21.374 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:38:21.374 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T04:38:21.410 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:38:21.410 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:21.482 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:38:21.482 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.conf 2023-09-23T04:38:21.517 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:38:21.518 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:38:21.587 INFO:tasks.cephadm:Adding mgr.x on smithi005 2023-09-23T04:38:21.587 INFO:tasks.cephadm:Adding mgr.y on smithi069 2023-09-23T04:38:21.588 INFO:tasks.cephadm:Adding mgr.z on smithi118 2023-09-23T04:38:21.588 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch apply mgr '3;smithi005=x;smithi069=y;smithi118=z' 2023-09-23T04:38:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2147101340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:21.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2147101340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:21.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:21.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2147101340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:22.176 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:38:22.755 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: Reconfiguring daemon mon.a on smithi005 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:22.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:22 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: Reconfiguring daemon mon.a on smithi005 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:22.783 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:22.784 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:22 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: Reconfiguring mon.a (unknown last config time)... 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: Reconfiguring daemon mon.a on smithi005 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:22.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:22 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:23.429 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.427+0000 7fa8e177d700 1 -- 172.21.15.118:0/3738964621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc0ffe60 msgr2=0x7fa8dc100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.427+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3738964621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc100280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009910 tx=0x7fa8d0009c20 comp rx=0 tx=0).stop 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.427+0000 7fa8e177d700 1 -- 172.21.15.118:0/3738964621 shutdown_connections 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3738964621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc100280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 -- 172.21.15.118:0/3738964621 >> 172.21.15.118:0/3738964621 conn(0x7fa8dc0fa660 msgr2=0x7fa8dc0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 -- 172.21.15.118:0/3738964621 shutdown_connections 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 -- 172.21.15.118:0/3738964621 wait complete. 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 Processor -- start 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 -- start start 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.428+0000 7fa8e177d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc104930 0x7fa8dc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:23.430 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8e177d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a02b0 con 0x7fa8dc104930 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8e177d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a03f0 con 0x7fa8dc0ffe60 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8e177d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1a0530 con 0x7fa8dc103f70 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52186/0 (socket says 172.21.15.118:52186) 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 -- 172.21.15.118:0/3945567505 learned_addr learned my addr 172.21.15.118:0/3945567505 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8daffd700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8db7fe700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc104930 0x7fa8dc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 -- 172.21.15.118:0/3945567505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc0ffe60 msgr2=0x7fa8dc105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc105920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 -- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc104930 msgr2=0x7fa8dc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc104930 0x7fa8dc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.431 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 -- 172.21.15.118:0/3945567505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00095f0 con 0x7fa8dc103f70 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8da7fc700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b780 tx=0x7fa8cc00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004230 con 0x7fa8dc103f70 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8cc004390 con 0x7fa8dc103f70 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc1a0810 con 0x7fa8dc103f70 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011b90 con 0x7fa8dc103f70 2023-09-23T04:38:23.432 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.429+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc1a0d90 con 0x7fa8dc103f70 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.430+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fa8cc011cf0 con 0x7fa8dc103f70 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.430+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc18f500 con 0x7fa8dc103f70 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.430+0000 7fa8c3fff700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 0x7fa8c403b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.430+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa8cc04a250 con 0x7fa8dc103f70 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.431+0000 7fa8daffd700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 0x7fa8c403b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:23.433 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.431+0000 7fa8daffd700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 0x7fa8c403b260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0004690 tx=0x7fa8d000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:23.435 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.436+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa8cc018cf0 con 0x7fa8dc103f70 2023-09-23T04:38:23.647 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.647+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi069=y;smithi118=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8dc02d240 con 0x7fa8c4038da0 2023-09-23T04:38:23.654 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.654+0000 7fa8c3fff700 1 -- 172.21.15.118:0/3945567505 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa8dc02d240 con 0x7fa8c4038da0 2023-09-23T04:38:23.654 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled mgr update... 2023-09-23T04:38:23.656 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 msgr2=0x7fa8c403b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:23.656 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 0x7fa8c403b260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0004690 tx=0x7fa8d000b540 comp rx=0 tx=0).stop 2023-09-23T04:38:23.656 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 msgr2=0x7fa8dc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:23.656 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b780 tx=0x7fa8cc00ba90 comp rx=0 tx=0).stop 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 shutdown_connections 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa8c4038da0 0x7fa8c403b260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8dc104930 0x7fa8dc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8dc0ffe60 0x7fa8dc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 --2- 172.21.15.118:0/3945567505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa8dc103f70 0x7fa8dc1043f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 >> 172.21.15.118:0/3945567505 conn(0x7fa8dc0fa660 msgr2=0x7fa8dc0fb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 shutdown_connections 2023-09-23T04:38:23.657 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:38:23.656+0000 7fa8e177d700 1 -- 172.21.15.118:0/3945567505 wait complete. 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: Reconfiguring mon.b (monmap changed)... 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: Reconfiguring daemon mon.b on smithi069 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:23.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:23.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:23.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: Reconfiguring mon.b (monmap changed)... 2023-09-23T04:38:23.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: Reconfiguring daemon mon.b on smithi069 2023-09-23T04:38:23.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:23.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:23.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: Reconfiguring mon.b (monmap changed)... 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: Reconfiguring daemon mon.b on smithi069 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-23T04:38:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-23T04:38:23.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:24.021 DEBUG:teuthology.orchestra.run.smithi069:mgr.y> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.y.service 2023-09-23T04:38:24.023 DEBUG:teuthology.orchestra.run.smithi118:mgr.z> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.z.service 2023-09-23T04:38:24.025 INFO:tasks.cephadm:Deploying OSDs... 2023-09-23T04:38:24.026 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:38:24.026 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T04:38:24.053 INFO:journalctl@ceph.mgr.z.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:38:24.054 INFO:journalctl@ceph.mgr.y.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:38:24.074 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T04:38:24.074 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 449 Links: 1 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-09-23 04:37:35.919283382 +0000 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-09-23 04:31:52.391473834 +0000 2023-09-23T04:38:24.121 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-09-23 04:31:52.391473834 +0000 2023-09-23T04:38:24.122 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-09-23 04:31:52.366474321 +0000 2023-09-23T04:38:24.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T04:38:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-09-23T04:38:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-09-23T04:38:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000183661 s, 2.8 MB/s 2023-09-23T04:38:24.232 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T04:38:24.277 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2023-09-23T04:38:24.360 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T04:38:24.360 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:24.360 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 455 Links: 1 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-09-23 04:37:36.002281768 +0000 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-09-23 04:31:53.201458075 +0000 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-09-23 04:31:53.201458075 +0000 2023-09-23T04:38:24.361 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-09-23 04:31:53.177458542 +0000 2023-09-23T04:38:24.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T04:38:24.466 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-09-23T04:38:24.466 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-09-23T04:38:24.466 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000187837 s, 2.7 MB/s 2023-09-23T04:38:24.469 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T04:38:24.514 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2023-09-23T04:38:24.598 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T04:38:24.598 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 461 Links: 1 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-09-23 04:37:36.081280231 +0000 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-09-23 04:31:53.991442706 +0000 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-09-23 04:31:53.991442706 +0000 2023-09-23T04:38:24.599 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-09-23 04:31:53.969443134 +0000 2023-09-23T04:38:24.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T04:38:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-09-23T04:38:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-09-23T04:38:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000182469 s, 2.8 MB/s 2023-09-23T04:38:24.709 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T04:38:24.753 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2023-09-23T04:38:24.838 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T04:38:24.838 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 467 Links: 1 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-09-23 04:37:36.166278579 +0000 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-09-23 04:31:54.818426616 +0000 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-09-23 04:31:54.818426616 +0000 2023-09-23T04:38:24.839 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-09-23 04:31:54.794427083 +0000 2023-09-23T04:38:24.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T04:38:24.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: Reconfiguring mon.c (monmap changed)... 2023-09-23T04:38:24.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: Reconfiguring daemon mon.c on smithi118 2023-09-23T04:38:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi069=y;smithi118=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: Saving service mgr spec with placement smithi005=x;smithi069=y;smithi118=z;count:3 2023-09-23T04:38:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:24.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: Reconfiguring mon.c (monmap changed)... 2023-09-23T04:38:24.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: Reconfiguring daemon mon.c on smithi118 2023-09-23T04:38:24.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:24.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi069=y;smithi118=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:24.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: Saving service mgr spec with placement smithi005=x;smithi069=y;smithi118=z;count:3 2023-09-23T04:38:24.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:24.945 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-09-23T04:38:24.945 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-09-23T04:38:24.945 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000174565 s, 2.9 MB/s 2023-09-23T04:38:24.947 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T04:38:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: Reconfiguring mon.c (monmap changed)... 2023-09-23T04:38:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: Reconfiguring daemon mon.c on smithi118 2023-09-23T04:38:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi069=y;smithi118=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: Saving service mgr spec with placement smithi005=x;smithi069=y;smithi118=z;count:3 2023-09-23T04:38:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.033 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2023-09-23T04:38:25.033 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T04:38:25.054 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T04:38:25.054 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 380668 Links: 1 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-23 04:38:11.954309279 +0000 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-23 04:28:33.149931628 +0000 2023-09-23T04:38:25.115 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-23 04:28:33.149931628 +0000 2023-09-23T04:38:25.116 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-23T04:38:25.116 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T04:38:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-23T04:38:25.205 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-23T04:38:25.205 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000322634 s, 1.6 MB/s 2023-09-23T04:38:25.207 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T04:38:25.267 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 384446 Links: 1 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-23 04:38:11.955309257 +0000 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-23 04:28:33.655920347 +0000 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-23 04:28:33.655920347 +0000 2023-09-23T04:38:25.326 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-23T04:38:25.327 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T04:38:25.393 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-23T04:38:25.393 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-23T04:38:25.393 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000232813 s, 2.2 MB/s 2023-09-23T04:38:25.395 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T04:38:25.454 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2023-09-23T04:38:25.511 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 391874 Links: 1 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-23 04:38:11.955309257 +0000 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-23 04:28:34.158909134 +0000 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-23 04:28:34.158909134 +0000 2023-09-23T04:38:25.512 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-23T04:38:25.512 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T04:38:25.578 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-23T04:38:25.578 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-23T04:38:25.578 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000206579 s, 2.5 MB/s 2023-09-23T04:38:25.579 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T04:38:25.637 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2023-09-23T04:38:25.694 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T04:38:25.694 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 6h/6d Inode: 384879 Links: 1 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2023-09-23 04:38:11.955309257 +0000 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2023-09-23 04:28:34.650898165 +0000 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2023-09-23 04:28:34.650898165 +0000 2023-09-23T04:38:25.695 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2023-09-23T04:38:25.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T04:38:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2023-09-23T04:38:25.759 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2023-09-23T04:38:25.760 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000193347 s, 2.6 MB/s 2023-09-23T04:38:25.760 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T04:38:25.818 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2023-09-23T04:38:25.818 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T04:38:25.835 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T04:38:25.835 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2023-09-23T04:38:25.892 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 381763 Links: 1 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-09-23 04:38:03.879725873 +0000 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-09-23 04:28:38.293841762 +0000 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-09-23 04:28:38.293841762 +0000 2023-09-23T04:38:25.893 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2023-09-23T04:38:25.893 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:25.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:25.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:25.938 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-09-23T04:38:25.938 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-09-23T04:38:25.938 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000124449 s, 4.1 MB/s 2023-09-23T04:38:25.939 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:25.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:25.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:25.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:25.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:25.997 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2023-09-23T04:38:26.054 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 384900 Links: 1 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-09-23 04:38:03.879725873 +0000 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-09-23 04:28:38.789830682 +0000 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-09-23 04:28:38.789830682 +0000 2023-09-23T04:38:26.055 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2023-09-23T04:38:26.055 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T04:38:26.123 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-09-23T04:38:26.123 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-09-23T04:38:26.123 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000143061 s, 3.6 MB/s 2023-09-23T04:38:26.124 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T04:38:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:26.183 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 380804 Links: 1 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-09-23 04:38:26.188231990 +0000 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-09-23 04:28:39.293819422 +0000 2023-09-23T04:38:26.242 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-09-23 04:28:39.293819422 +0000 2023-09-23T04:38:26.243 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2023-09-23T04:38:26.243 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T04:38:26.313 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-09-23T04:38:26.314 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-09-23T04:38:26.314 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000100085 s, 5.1 MB/s 2023-09-23T04:38:26.315 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T04:38:26.374 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2023-09-23T04:38:26.431 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T04:38:26.431 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 6h/6d Inode: 389443 Links: 1 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2023-09-23 04:38:26.188231990 +0000 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2023-09-23 04:28:39.796808185 +0000 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2023-09-23 04:28:39.796808185 +0000 2023-09-23T04:38:26.432 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: - 2023-09-23T04:38:26.432 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T04:38:26.495 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 systemd[1]: Starting Ceph mgr.z for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:38:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2023-09-23T04:38:26.497 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2023-09-23T04:38:26.498 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 9.7063e-05 s, 5.3 MB/s 2023-09-23T04:38:26.501 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T04:38:26.558 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2023-09-23T04:38:26.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T04:38:26.746 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 podman[136357]: 2023-09-23T04:38:26.746 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:26 smithi118 ceph-mon[134297]: Deploying daemon mgr.z on smithi118 2023-09-23T04:38:26.746 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:26 smithi118 ceph-mon[134297]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:26 smithi005 ceph-mon[97638]: Deploying daemon mgr.z on smithi118 2023-09-23T04:38:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:26 smithi005 ceph-mon[97638]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:26 smithi069 ceph-mon[133931]: Deploying daemon mgr.z on smithi118 2023-09-23T04:38:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:26 smithi069 ceph-mon[133931]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:27.042 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 bash[136357]: c165a59d9a1a10c053a8a3095b6e6d4ecb8681eaee458a99345231df03f8c2c4 2023-09-23T04:38:27.042 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 ceph-mgr[136459]: -- 172.21.15.118:0/772423759 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5570c5f771e0 con 0x5570c5f4cc00 2023-09-23T04:38:27.043 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 systemd[1]: Started Ceph mgr.z for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:38:27.043 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:26.874+0000 7f1964b36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:38:27.043 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:26.955+0000 7f1964b36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:38:27.043 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.042+0000 7f1964b36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:38:27.301 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.134+0000 7f1964b36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:38:27.660 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.301+0000 7f1964b36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:38:27.917 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.917 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: Deploying daemon mgr.y on smithi069 2023-09-23T04:38:27.918 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:27 smithi069 ceph-mon[133931]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.661+0000 7f1964b36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.790+0000 7f1964b36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.877+0000 7f1964b36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.971 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: Deploying daemon mgr.y on smithi069 2023-09-23T04:38:27.972 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-mon[134297]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:28.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: Deploying daemon mgr.y on smithi069 2023-09-23T04:38:28.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:27 smithi005 ceph-mon[97638]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:28.660 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:27.971+0000 7f1964b36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:38:28.661 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:28.384+0000 7f1964b36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:38:28.737 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:28 smithi069 systemd[1]: Starting Ceph mgr.y for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:38:28.991 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:28 smithi069 podman[135368]: 2023-09-23T04:38:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T04:38:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T04:38:29.260 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-09-23T04:38:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-09-23T04:38:29.261 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.047671 s, 220 MB/s 2023-09-23T04:38:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:28 smithi069 bash[135368]: febd4f34e9f783fd41229651fe66ffbaadb1fd634f33645b358e9fa9d488bf5d 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:28 smithi069 ceph-mgr[135409]: -- 172.21.15.69:0/1278286534 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a6d90b1e0 con 0x559a6d8e0800 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 systemd[1]: Started Ceph mgr.y for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.110+0000 7fd362cff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.192+0000 7fd362cff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:38:29.283 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.281+0000 7fd362cff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:38:29.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:29.043+0000 7f1964b36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:38:29.411 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:29.124+0000 7f1964b36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:38:29.566 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.382+0000 7fd362cff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:38:29.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2023-09-23T04:38:29.955 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.565+0000 7fd362cff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:38:29.956 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:29.954+0000 7fd362cff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:38:30.197 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:29.925+0000 7f1964b36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:30.015+0000 7f1964b36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:30.198 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-mon[134297]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:30.089+0000 7fd362cff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:30.178+0000 7fd362cff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-mon[133931]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:30 smithi005 ceph-mon[97638]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:38:30.485 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:30.197+0000 7f1964b36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:38:30.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:30.281+0000 7fd362cff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:38:30.877 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:30.485+0000 7f1964b36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:38:30.877 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:30.725+0000 7f1964b36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:38:31.142 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:30.877+0000 7f1964b36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:38:31.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:30.743+0000 7fd362cff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:38:31.411 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:31.142+0000 7f1964b36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:38:31.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:31.424+0000 7fd362cff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:38:31.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:31.505+0000 7fd362cff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 -- 172.21.15.5:0/1116828750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0100e60 msgr2=0x7ff0e0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/1116828750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0100e60 0x7ff0e0101280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009990 tx=0x7ff0dc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 -- 172.21.15.5:0/1116828750 shutdown_connections 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/1116828750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102a10 0x7ff0e0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/1116828750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0100e60 0x7ff0e0101280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/1116828750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0102050 0x7ff0e01024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 -- 172.21.15.5:0/1116828750 >> 172.21.15.5:0/1116828750 conn(0x7ff0e00fc410 msgr2=0x7ff0e00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:31.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.961+0000 7ff0e72e7700 1 -- 172.21.15.5:0/1116828750 shutdown_connections 2023-09-23T04:38:31.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.962+0000 7ff0e72e7700 1 -- 172.21.15.5:0/1116828750 wait complete. 2023-09-23T04:38:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.962+0000 7ff0e72e7700 1 Processor -- start 2023-09-23T04:38:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 -- start start 2023-09-23T04:38:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102050 0x7ff0e0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:31.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0102a10 0x7ff0e01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e01a09a0 con 0x7ff0e0102050 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e01a0ae0 con 0x7ff0e0102a10 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e72e7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e01a0c70 con 0x7ff0e0100e60 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e5083700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e5083700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58730/0 (socket says 172.21.15.5:58730) 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e5083700 1 -- 172.21.15.5:0/458724522 learned_addr learned my addr 172.21.15.5:0/458724522 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e4882700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102050 0x7ff0e0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.963+0000 7ff0e5884700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0102a10 0x7ff0e01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 -- 172.21.15.5:0/458724522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0102a10 msgr2=0x7ff0e01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:31.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0102a10 0x7ff0e01a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 -- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102050 msgr2=0x7ff0e0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102050 0x7ff0e0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 -- 172.21.15.5:0/458724522 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc0095f0 con 0x7ff0e0100e60 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e5083700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005770 tx=0x7ff0dc0057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc01d070 con 0x7ff0e0100e60 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0dc021410 con 0x7ff0e0100e60 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e01a0ef0 con 0x7ff0e0100e60 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc017650 con 0x7ff0e0100e60 2023-09-23T04:38:31.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.964+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e01a12a0 con 0x7ff0e0100e60 2023-09-23T04:38:31.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.965+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7ff0dc00fb00 con 0x7ff0e0100e60 2023-09-23T04:38:31.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.965+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0193520 con 0x7ff0e0100e60 2023-09-23T04:38:31.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.966+0000 7ff0d67fc700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 0x7ff0cc03b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:31.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.966+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff0dc04c950 con 0x7ff0e0100e60 2023-09-23T04:38:31.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.966+0000 7ff0e4882700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 0x7ff0cc03b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:31.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.967+0000 7ff0e4882700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 0x7ff0cc03b010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0100b90 tx=0x7ff0d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:31.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:31.971+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff0dc029540 con 0x7ff0e0100e60 2023-09-23T04:38:32.153 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:31.833+0000 7f1964b36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:38:32.153 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:31.964+0000 7f1964b36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T04:38:32.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: Reconfiguring daemon mgr.x on smithi005 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:31 smithi118 ceph-mon[134297]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: Reconfiguring daemon mgr.x on smithi005 2023-09-23T04:38:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:31 smithi005 ceph-mon[97638]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:32.195+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0e002d2b0 con 0x7ff0cc038b50 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: Reconfiguring mgr.x (unknown last config time)... 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: Reconfiguring daemon mgr.x on smithi005 2023-09-23T04:38:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:31 smithi069 ceph-mon[133931]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:32.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:32.153+0000 7f1964b36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:38:32.411 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:32.265+0000 7f1964b36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:38:32.535 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:32.273+0000 7fd362cff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:38:32.536 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:32.364+0000 7fd362cff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:38:32.812 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:32.535+0000 7fd362cff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:38:32.911 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:32.480+0000 7f1964b36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.187 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:32.812+0000 7fd362cff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:38:33.187 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:33.037+0000 7fd362cff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:33.188 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:33.336 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:33.337 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:33.448 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:33.187+0000 7fd362cff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:38:33.660 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:38:33.336+0000 7f1964b36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:38:33.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:33.448+0000 7fd362cff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:38:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:34.009+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mgrmap(e 14) v1 ==== 90916+0+0 (secure 0 0 0) 0x7ff0dc04ed00 con 0x7ff0e0100e60 2023-09-23T04:38:34.273 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:34.143+0000 7fd362cff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:38:34.273 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: Standby manager daemon z started 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:34.274 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: Standby manager daemon z started 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:34.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: Standby manager daemon z started 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/3676103710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:34.578 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:34.272+0000 7fd362cff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:38:34.579 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:34.466+0000 7fd362cff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:38:34.991 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:34.578+0000 7fd362cff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:38:34.991 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:34.794+0000 7fd362cff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:38:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:35 smithi005 ceph-mon[97638]: mgrmap e14: x(active, since 70s), standbys: z 2023-09-23T04:38:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:38:35.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:35 smithi118 ceph-mon[134297]: mgrmap e14: x(active, since 70s), standbys: z 2023-09-23T04:38:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:38:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:35 smithi069 ceph-mon[133931]: mgrmap e14: x(active, since 70s), standbys: z 2023-09-23T04:38:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:38:35.991 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:38:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:38:35.679+0000 7fd362cff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:38:36.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:36.392+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mon.1 v2:172.21.15.118:3300/0 8 ==== mgrmap(e 15) v1 ==== 136135+0+0 (secure 0 0 0) 0x7ff0dc02b440 con 0x7ff0e0100e60 2023-09-23T04:38:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]: dispatch 2023-09-23T04:38:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]': finished 2023-09-23T04:38:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T04:38:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: Standby manager daemon y started 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:36.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2276870328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:38:36.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]': finished 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: Standby manager daemon y started 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:36 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2276870328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:38:36.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/994962723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35700daf-4df6-4103-ade1-12999750ad33"}]': finished 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: osdmap e5: 1 total, 0 up, 1 in 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: Standby manager daemon y started 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1818940905' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:38:36.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:36 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2276870328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:38:37.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:37 smithi005 ceph-mon[97638]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-09-23T04:38:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:37 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:38:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:37 smithi118 ceph-mon[134297]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-09-23T04:38:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:37 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:38:37.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:37 smithi069 ceph-mon[133931]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-09-23T04:38:37.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:37 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:38:38.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:38 smithi005 ceph-mon[97638]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:38 smithi118 ceph-mon[134297]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:38.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:38 smithi069 ceph-mon[133931]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:40.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:40 smithi005 ceph-mon[97638]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:40.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:40 smithi118 ceph-mon[134297]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:40.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:40 smithi069 ceph-mon[133931]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:42.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:42 smithi005 ceph-mon[97638]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:42.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:42 smithi118 ceph-mon[134297]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:42 smithi069 ceph-mon[133931]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T04:38:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:43.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T04:38:43.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-23T04:38:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:44.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:44 smithi005 ceph-mon[97638]: Deploying daemon osd.0 on smithi005 2023-09-23T04:38:44.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:44 smithi005 ceph-mon[97638]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:44.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:44 smithi118 ceph-mon[134297]: Deploying daemon osd.0 on smithi005 2023-09-23T04:38:44.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:44 smithi118 ceph-mon[134297]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:44.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:44 smithi069 ceph-mon[133931]: Deploying daemon osd.0 on smithi005 2023-09-23T04:38:44.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:44 smithi069 ceph-mon[133931]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:46.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:46 smithi005 ceph-mon[97638]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:46.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:46 smithi118 ceph-mon[134297]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:46 smithi069 ceph-mon[133931]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:48.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:48 smithi005 ceph-mon[97638]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:48.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:48 smithi118 ceph-mon[134297]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:48 smithi069 ceph-mon[133931]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:50.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:50.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:49 smithi069 ceph-mon[133931]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:50.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:50.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:49 smithi005 ceph-mon[97638]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:50.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:50.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:50.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:49 smithi118 ceph-mon[134297]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:52.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.256+0000 7ff0d67fc700 1 -- 172.21.15.5:0/458724522 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff0e002d2b0 con 0x7ff0cc038b50 2023-09-23T04:38:52.258 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.258+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 msgr2=0x7ff0cc03b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.258+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 0x7ff0cc03b010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0100b90 tx=0x7ff0d0009260 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 msgr2=0x7ff0e0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005770 tx=0x7ff0dc0057a0 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 shutdown_connections 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff0cc038b50 0x7ff0cc03b010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0e0102a10 0x7ff0e01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff0e0100e60 0x7ff0e0198e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 --2- 172.21.15.5:0/458724522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e0102050 0x7ff0e0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 >> 172.21.15.5:0/458724522 conn(0x7ff0e00fc410 msgr2=0x7ff0e0107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 shutdown_connections 2023-09-23T04:38:52.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:38:52.259+0000 7ff0e72e7700 1 -- 172.21.15.5:0/458724522 wait complete. 2023-09-23T04:38:52.662 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.0.service 2023-09-23T04:38:52.666 INFO:tasks.cephadm:Deploying osd.1 on smithi069 with /dev/vg_nvme/lv_4... 2023-09-23T04:38:52.667 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T04:38:52.715 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:52.746 INFO:journalctl@ceph.osd.0.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:52.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:38:53.154 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:38:52 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T04:38:52.949+0000 7f04bdfa9640 -1 osd.0 0 log_to_monitors true 2023-09-23T04:38:53.232 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:38:53.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:53 smithi069 ceph-mon[133931]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:53.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:53 smithi069 ceph-mon[133931]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:53 smithi005 ceph-mon[97638]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:53 smithi005 ceph-mon[97638]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:53.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:53 smithi118 ceph-mon[134297]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:53.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:53 smithi118 ceph-mon[134297]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150733 s, 696 MB/s 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 2023-09-23T04:38:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-09-23T04:38:54.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:54.889 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T04:38:54.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T04:38:54.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:54.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:54.890 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:54 smithi069 ceph-mon[133931]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:54.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:54.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T04:38:54.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T04:38:54.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:54.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:54.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-mon[97638]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:54.906 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T04:38:54.633+0000 7f04b3621700 -1 osd.0 0 waiting for initial osdmap 2023-09-23T04:38:54.906 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:38:54 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T04:38:54.637+0000 7f04adc13700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:38:54.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:54.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-23T04:38:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: osdmap e6: 1 total, 0 up, 1 in 2023-09-23T04:38:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: from='osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:54 smithi118 ceph-mon[134297]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:38:55.285 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2023-09-23T04:38:55.838 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:38:55.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:55 smithi005 ceph-mon[97638]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:38:55.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:55 smithi005 ceph-mon[97638]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T04:38:55.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:55 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:55.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:55 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:55.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:55 smithi118 ceph-mon[134297]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:38:55.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:55 smithi118 ceph-mon[134297]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T04:38:55.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:55 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:55.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:55 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:55 smithi069 ceph-mon[133931]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:38:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:55 smithi069 ceph-mon[133931]: osdmap e7: 1 total, 0 up, 1 in 2023-09-23T04:38:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:55 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:55 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] boot 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] boot 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:56.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:56.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 -- 172.21.15.69:0/3096606027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 msgr2=0x7fa9201017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:56.955 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 --2- 172.21.15.69:0/3096606027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa9201017d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa914009990 tx=0x7fa914009ca0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 -- 172.21.15.69:0/3096606027 shutdown_connections 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 --2- 172.21.15.69:0/3096606027 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 0x7fa9201082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 --2- 172.21.15.69:0/3096606027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa9201017d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 --2- 172.21.15.69:0/3096606027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 0x7fa9201004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 -- 172.21.15.69:0/3096606027 >> 172.21.15.69:0/3096606027 conn(0x7fa9200faad0 msgr2=0x7fa9200fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 -- 172.21.15.69:0/3096606027 shutdown_connections 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.953+0000 7fa9251ec700 1 -- 172.21.15.69:0/3096606027 wait complete. 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 Processor -- start 2023-09-23T04:38:56.956 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 -- start start 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 0x7fa9200ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 0x7fa92019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92019e6c0 con 0x7fa920100060 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92019e830 con 0x7fa9201013b0 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa9251ec700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92019e9f0 con 0x7fa920100a20 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa91f59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa91f59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58072/0 (socket says 172.21.15.69:58072) 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa91f59e700 1 -- 172.21.15.69:0/2547696306 learned_addr learned my addr 172.21.15.69:0/2547696306 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-09-23T04:38:56.957 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa91e59c700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 0x7fa92019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.954+0000 7fa91ed9d700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 0x7fa9200ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 -- 172.21.15.69:0/2547696306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 msgr2=0x7fa92019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 0x7fa92019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 -- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 msgr2=0x7fa9200ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 0x7fa9200ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 -- 172.21.15.69:0/2547696306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140095f0 con 0x7fa9201013b0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa91f59e700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa918007ee0 tx=0x7fa91800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918017070 con 0x7fa9201013b0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92019ecd0 con 0x7fa9201013b0 2023-09-23T04:38:56.958 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91800f040 con 0x7fa9201013b0 2023-09-23T04:38:56.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918005580 con 0x7fa9201013b0 2023-09-23T04:38:56.959 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.955+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92019f180 con 0x7fa9201013b0 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.956+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa920107450 con 0x7fa9201013b0 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.957+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) v1 ==== 136135+0+0 (secure 0 0 0) 0x7fa91800db20 con 0x7fa9201013b0 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.957+0000 7fa907fff700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 0x7fa9080a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.957+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fa9180cac50 con 0x7fa9201013b0 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.958+0000 7fa91ed9d700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 0x7fa9080a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:38:56.960 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.958+0000 7fa91ed9d700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 0x7fa9080a3a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:38:56.962 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:56.962+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa918095c10 con 0x7fa9201013b0 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: osd.0 [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] boot 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: osdmap e8: 1 total, 1 up, 1 in 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:38:56.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:56.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:56.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:38:56.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:38:57.158 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:38:57.157+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa92002d230 con 0x7fa9080a1590 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi005 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 17272M 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:57 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:57.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi005 2023-09-23T04:38:57.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 17272M 2023-09-23T04:38:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T04:38:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:57 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi005 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 17272M 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: osdmap e9: 1 total, 1 up, 1 in 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:38:57.936 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:57 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:38:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:58 smithi005 ceph-mon[97638]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:58 smithi005 ceph-mon[97638]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:38:58.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:58 smithi118 ceph-mon[134297]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:58.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:58 smithi118 ceph-mon[134297]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:38:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:58 smithi069 ceph-mon[133931]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:38:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:58 smithi069 ceph-mon[133931]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:38:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:59 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/4092210598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:59 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:59 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]': finished 2023-09-23T04:38:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:59 smithi005 ceph-mon[97638]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T04:38:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:38:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:38:59.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:59 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/4092210598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:59 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:59 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]': finished 2023-09-23T04:38:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:59 smithi118 ceph-mon[134297]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T04:38:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:38:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:38:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:59 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/4092210598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:59 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]: dispatch 2023-09-23T04:38:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:59 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0d9aada-e5c2-4198-8a8e-a9db453fd897"}]': finished 2023-09-23T04:38:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:59 smithi069 ceph-mon[133931]: osdmap e10: 2 total, 1 up, 2 in 2023-09-23T04:38:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:38:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:00 smithi005 ceph-mon[97638]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/3490030865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:00.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:00 smithi118 ceph-mon[134297]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:00.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/3490030865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:00 smithi069 ceph-mon[133931]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/3490030865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:02 smithi005 ceph-mon[97638]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:02.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:02 smithi118 ceph-mon[134297]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:02 smithi069 ceph-mon[133931]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:04.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:04 smithi118 ceph-mon[134297]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:04 smithi069 ceph-mon[133931]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:04 smithi005 ceph-mon[97638]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:06.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:06 smithi005 ceph-mon[97638]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:06 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T04:39:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:06 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:06.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:06 smithi118 ceph-mon[134297]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:06 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T04:39:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:06 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:06 smithi069 ceph-mon[133931]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:06 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-23T04:39:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:06 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:07.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:07 smithi005 ceph-mon[97638]: Deploying daemon osd.1 on smithi069 2023-09-23T04:39:07.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:07 smithi118 ceph-mon[134297]: Deploying daemon osd.1 on smithi069 2023-09-23T04:39:07.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:07 smithi069 ceph-mon[133931]: Deploying daemon osd.1 on smithi069 2023-09-23T04:39:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:08 smithi005 ceph-mon[97638]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:08.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:08 smithi118 ceph-mon[134297]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:08.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:08 smithi069 ceph-mon[133931]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:10 smithi005 ceph-mon[97638]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:10.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:10 smithi118 ceph-mon[134297]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:10 smithi069 ceph-mon[133931]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:12.877 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.875+0000 7fa907fff700 1 -- 172.21.15.69:0/2547696306 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa92002d230 con 0x7fa9080a1590 2023-09-23T04:39:12.877 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 1 on host 'smithi069' 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 msgr2=0x7fa9080a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 0x7fa9080a3a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa914000c00 tx=0x7fa914005fb0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 msgr2=0x7fa92019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa918007ee0 tx=0x7fa91800d3b0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 shutdown_connections 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9080a1590 0x7fa9080a3a50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9201013b0 0x7fa92019df20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa920100060 0x7fa9200ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 --2- 172.21.15.69:0/2547696306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa920100a20 0x7fa92019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:12.879 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 >> 172.21.15.69:0/2547696306 conn(0x7fa9200faad0 msgr2=0x7fa920104590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:39:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 shutdown_connections 2023-09-23T04:39:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:39:12.877+0000 7fa9251ec700 1 -- 172.21.15.69:0/2547696306 wait complete. 2023-09-23T04:39:12.896 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:12.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:12.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:12.897 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:12.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:12.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:12.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:13.324 DEBUG:teuthology.orchestra.run.smithi069:osd.1> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.1.service 2023-09-23T04:39:13.326 INFO:tasks.cephadm:Deploying osd.2 on smithi118 with /dev/vg_nvme/lv_4... 2023-09-23T04:39:13.326 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-23T04:39:13.360 INFO:journalctl@ceph.osd.1.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:39:13.893 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:39:14.077 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:13 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.077 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:13 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.077 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:13 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:14.077 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:13 smithi069 ceph-mon[133931]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:13 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:13 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:13 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:13 smithi005 ceph-mon[97638]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:13 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:13 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:13 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:13 smithi118 ceph-mon[134297]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:14.467 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T04:39:14.150+0000 7fd291963640 -1 osd.1 0 log_to_monitors true 2023-09-23T04:39:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:14 smithi005 ceph-mon[97638]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:14 smithi005 ceph-mon[97638]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:14 smithi118 ceph-mon[134297]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:14 smithi118 ceph-mon[134297]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:14 smithi069 ceph-mon[133931]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:14 smithi069 ceph-mon[133931]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-23T04:39:15.479 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-23T04:39:15.479 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-23T04:39:15.480 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2023-09-23T04:39:15.480 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2023-09-23T04:39:15.480 INFO:teuthology.orchestra.run.smithi118.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00825905 s, 1.3 GB/s 2023-09-23T04:39:15.480 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2023-09-23T04:39:15.932 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi118:vg_nvme/lv_4 2023-09-23T04:39:15.991 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T04:39:15.871+0000 7fd286fdb700 -1 osd.1 0 waiting for initial osdmap 2023-09-23T04:39:15.991 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T04:39:15.875+0000 7fd282dd0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:39:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T04:39:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T04:39:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:16.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:16.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: osdmap e11: 2 total, 1 up, 2 in 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:16.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:16.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:16.485 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:39:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi069 2023-09-23T04:39:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 17168M 2023-09-23T04:39:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:39:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T04:39:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:16 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi069 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 17168M 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:16 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi069 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 17168M 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: osdmap e12: 2 total, 1 up, 2 in 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:16 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 -- 172.21.15.118:0/345142142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 msgr2=0x7f1838101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 --2- 172.21.15.118:0/345142142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f1838101760 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f182c009990 tx=0x7f182c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 -- 172.21.15.118:0/345142142 shutdown_connections 2023-09-23T04:39:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 --2- 172.21.15.118:0/345142142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1838100a20 0x7f18381082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.597 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 --2- 172.21.15.118:0/345142142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f1838101760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.598 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 --2- 172.21.15.118:0/345142142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100080 0x7f18381004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.598 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 -- 172.21.15.118:0/345142142 >> 172.21.15.118:0/345142142 conn(0x7f18380faad0 msgr2=0x7f18380fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:39:17.598 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 -- 172.21.15.118:0/345142142 shutdown_connections 2023-09-23T04:39:17.598 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.597+0000 7f183e1e0700 1 -- 172.21.15.118:0/345142142 wait complete. 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 Processor -- start 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 -- start start 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1838100080 0x7f1838196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 0x7f1838197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819e760 con 0x7f1838100080 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819e8a0 con 0x7f1838100a20 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f183e1e0700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183819ea60 con 0x7f1838101340 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:17.600 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52220/0 (socket says 172.21.15.118:52220) 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 -- 172.21.15.118:0/1879825544 learned_addr learned my addr 172.21.15.118:0/1879825544 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1836ffd700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 0x7f1838197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 -- 172.21.15.118:0/1879825544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 msgr2=0x7f1838197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 0x7f1838197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 -- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1838100080 msgr2=0x7f1838196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1838100080 0x7f1838196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1837fff700 1 -- 172.21.15.118:0/1879825544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0095f0 con 0x7f1838101340 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.598+0000 7f1836ffd700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 0x7f1838197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f1837fff700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1830007ee0 tx=0x7f183000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830017070 con 0x7f1838101340 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183000f040 con 0x7f1838101340 2023-09-23T04:39:17.601 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183819ed40 con 0x7f1838101340 2023-09-23T04:39:17.602 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830005740 con 0x7f1838101340 2023-09-23T04:39:17.602 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.599+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183819f190 con 0x7f1838101340 2023-09-23T04:39:17.602 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.600+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183810b620 con 0x7f1838101340 2023-09-23T04:39:17.602 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.601+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 15) v1 ==== 136135+0+0 (secure 0 0 0) 0x7f18300058a0 con 0x7f1838101340 2023-09-23T04:39:17.603 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.601+0000 7f1834ff9700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 0x7f18200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:17.603 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.602+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f18300caf50 con 0x7f1838101340 2023-09-23T04:39:17.603 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.602+0000 7f18377fe700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 0x7f18200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:17.603 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.602+0000 7f18377fe700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 0x7f18200a39d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098e0 tx=0x7f182c009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:39:17.606 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.606+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1830095d90 con 0x7f1838101340 2023-09-23T04:39:17.807 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:17.807+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f18380ff740 con 0x7f18200a1510 2023-09-23T04:39:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] boot 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] boot 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: osd.1 [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] boot 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: osdmap e13: 2 total, 2 up, 2 in 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-23T04:39:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:18.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:18.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:18 smithi005 ceph-mon[97638]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:18 smithi005 ceph-mon[97638]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T04:39:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:18 smithi118 ceph-mon[134297]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:18 smithi118 ceph-mon[134297]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T04:39:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:18 smithi069 ceph-mon[133931]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:18 smithi069 ceph-mon[133931]: osdmap e14: 2 total, 2 up, 2 in 2023-09-23T04:39:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:19 smithi005 ceph-mon[97638]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:19 smithi118 ceph-mon[134297]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:19 smithi069 ceph-mon[133931]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/3514629865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]': finished 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/2599180424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/3514629865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]': finished 2023-09-23T04:39:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T04:39:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/2599180424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/3514629865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]: dispatch 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "101be3f4-34d5-43eb-a131-2144d5a0f309"}]': finished 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: osdmap e15: 3 total, 2 up, 3 in 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/2599180424' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:21 smithi005 ceph-mon[97638]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:21 smithi118 ceph-mon[134297]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:21 smithi069 ceph-mon[133931]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:24.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:24 smithi005 ceph-mon[97638]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:24.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:24 smithi118 ceph-mon[134297]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:24 smithi069 ceph-mon[133931]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:26 smithi118 ceph-mon[134297]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:26 smithi005 ceph-mon[97638]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:26.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:26 smithi069 ceph-mon[133931]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:27.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T04:39:27.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:27 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:27.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:27 smithi005 ceph-mon[97638]: Deploying daemon osd.2 on smithi118 2023-09-23T04:39:27.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T04:39:27.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:27 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:27.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:27 smithi118 ceph-mon[134297]: Deploying daemon osd.2 on smithi118 2023-09-23T04:39:27.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-23T04:39:27.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:27 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:27.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:27 smithi069 ceph-mon[133931]: Deploying daemon osd.2 on smithi118 2023-09-23T04:39:28.650 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:28 smithi118 ceph-mon[134297]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:28.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:28 smithi005 ceph-mon[97638]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:28 smithi069 ceph-mon[133931]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:30.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:30 smithi005 ceph-mon[97638]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:30.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:30 smithi118 ceph-mon[134297]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:30 smithi069 ceph-mon[133931]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:31.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:31.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:31.837 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:32.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:32 smithi118 ceph-mon[134297]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:32 smithi069 ceph-mon[133931]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:32 smithi005 ceph-mon[97638]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:33.220 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.219+0000 7f1834ff9700 1 -- 172.21.15.118:0/1879825544 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f18380ff740 con 0x7f18200a1510 2023-09-23T04:39:33.221 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 2 on host 'smithi118' 2023-09-23T04:39:33.222 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 msgr2=0x7f18200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 0x7f18200a39d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098e0 tx=0x7f182c009530 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 msgr2=0x7f183819e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1830007ee0 tx=0x7f183000d3b0 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 shutdown_connections 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f18200a1510 0x7f18200a39d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1838101340 0x7f183819e190 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1838100080 0x7f1838196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 --2- 172.21.15.118:0/1879825544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1838100a20 0x7f1838197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 >> 172.21.15.118:0/1879825544 conn(0x7f18380faad0 msgr2=0x7f18380ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:39:33.223 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 shutdown_connections 2023-09-23T04:39:33.224 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:39:33.222+0000 7f183e1e0700 1 -- 172.21.15.118:0/1879825544 wait complete. 2023-09-23T04:39:33.569 DEBUG:teuthology.orchestra.run.smithi118:osd.2> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.2.service 2023-09-23T04:39:33.571 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2023-09-23T04:39:33.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T04:39:33.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:33.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:33.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:33.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:39:34.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:34 smithi005 ceph-mon[97638]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:34.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:34 smithi005 ceph-mon[97638]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T04:39:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:34 smithi069 ceph-mon[133931]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:34 smithi069 ceph-mon[133931]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T04:39:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:34 smithi118 ceph-mon[134297]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:34 smithi118 ceph-mon[134297]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-23T04:39:35.162 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T04:39:34.665+0000 7f0050bdf640 -1 osd.2 0 log_to_monitors true 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0298642 s, 351 MB/s 2023-09-23T04:39:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-09-23T04:39:36.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T04:39:36.196+0000 7f0046257700 -1 osd.2 0 waiting for initial osdmap 2023-09-23T04:39:36.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T04:39:36.200+0000 7f003f847700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:39:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T04:39:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T04:39:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:39:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:36.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:36.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:36.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-23T04:39:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: osdmap e16: 3 total, 2 up, 3 in 2023-09-23T04:39:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:39:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:39:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:36.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi118 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 17168M 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:37 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:39:37.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi118 2023-09-23T04:39:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 17168M 2023-09-23T04:39:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:39:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T04:39:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:37 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:37.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi118 2023-09-23T04:39:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 17168M 2023-09-23T04:39:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: from='osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:39:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: osdmap e17: 3 total, 2 up, 3 in 2023-09-23T04:39:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:37 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] boot 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:38.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:38.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:39:38.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:39:38.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] boot 2023-09-23T04:39:38.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T04:39:38.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:38.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:38.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: osd.2 [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] boot 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: osdmap e18: 3 total, 3 up, 3 in 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-23T04:39:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:39.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 -- 172.21.15.5:0/3476351712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8104300 msgr2=0x7f3ce8104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:39.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/3476351712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8104300 0x7f3ce8104760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80099c0 tx=0x7f3cd8009cd0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 -- 172.21.15.5:0/3476351712 shutdown_connections 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/3476351712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 0x7f3ce810b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/3476351712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8103100 0x7f3ce8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/3476351712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8104300 0x7f3ce8104760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 -- 172.21.15.5:0/3476351712 >> 172.21.15.5:0/3476351712 conn(0x7f3ce80fe680 msgr2=0x7f3ce8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:39:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 -- 172.21.15.5:0/3476351712 shutdown_connections 2023-09-23T04:39:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.215+0000 7f3cf00ed700 1 -- 172.21.15.5:0/3476351712 wait complete. 2023-09-23T04:39:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.216+0000 7f3cf00ed700 1 Processor -- start 2023-09-23T04:39:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.216+0000 7f3cf00ed700 1 -- start start 2023-09-23T04:39:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.216+0000 7f3cf00ed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 0x7f3ce8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.216+0000 7f3cf00ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cf00ed700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 0x7f3ce81a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cf00ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a0970 con 0x7f3ce8104300 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cf00ed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a0ab0 con 0x7f3ce8103100 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cf00ed700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81a0c40 con 0x7f3ce8104ca0 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60670/0 (socket says 172.21.15.5:60670) 2023-09-23T04:39:39.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 -- 172.21.15.5:0/2881389388 learned_addr learned my addr 172.21.15.5:0/2881389388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cede89700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 0x7f3ce8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cee68a700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 0x7f3ce81a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 -- 172.21.15.5:0/2881389388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 msgr2=0x7f3ce81a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 0x7f3ce81a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 -- 172.21.15.5:0/2881389388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 msgr2=0x7f3ce8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 0x7f3ce8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3ced688700 1 -- 172.21.15.5:0/2881389388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80095f0 con 0x7f3ce8104300 2023-09-23T04:39:39.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.217+0000 7f3cede89700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 0x7f3ce8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3ced688700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b580 tx=0x7f3cd800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd801d070 con 0x7f3ce8104300 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd8005050 con 0x7f3ce8104300 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce81a0ec0 con 0x7f3ce8104300 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8004a50 con 0x7f3ce8104300 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.218+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81a1270 con 0x7f3ce8104300 2023-09-23T04:39:39.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.219+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce810a6f0 con 0x7f3ce8104300 2023-09-23T04:39:39.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.221+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) v1 ==== 136135+0+0 (secure 0 0 0) 0x7f3cd80051c0 con 0x7f3ce8104300 2023-09-23T04:39:39.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.222+0000 7f3cdeffd700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 0x7f3cd40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:39:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.223+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f3cd80ccd50 con 0x7f3ce8104300 2023-09-23T04:39:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.223+0000 7f3cede89700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 0x7f3cd40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:39:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.223+0000 7f3cede89700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 0x7f3cd40a3a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8104160 tx=0x7f3ce4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:39:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.225+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3cd80977e0 con 0x7f3ce8104300 2023-09-23T04:39:39.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:39.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:39 smithi005 ceph-mon[97638]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T04:39:39.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:39.446+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ce802d270 con 0x7f3cd40a1590 2023-09-23T04:39:39.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:39.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:39 smithi069 ceph-mon[133931]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T04:39:39.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:39.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:39.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:39 smithi118 ceph-mon[134297]: osdmap e19: 3 total, 3 up, 3 in 2023-09-23T04:39:39.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:40 smithi069 ceph-mon[133931]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:40.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:40 smithi005 ceph-mon[97638]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:40.559 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122038]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:40.559 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:40.560 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122038]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:40.560 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122038]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: osdmap e20: 3 total, 3 up, 3 in 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:40 smithi118 ceph-mon[134297]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:41.155 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122103]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:41.155 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:41.155 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122103]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:41.155 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 04:39:40 smithi005 sudo[122103]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:41.446 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:41 smithi069 ceph-mon[133931]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T04:39:41.447 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:41 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:41.447 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143830]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:41.447 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:41.447 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143830]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:41.447 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143830]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:41.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:41 smithi005 ceph-mon[97638]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T04:39:41.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:41.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:41 smithi118 ceph-mon[134297]: osdmap e21: 3 total, 3 up, 3 in 2023-09-23T04:39:41.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:41 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:41.741 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143833]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:41.741 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:41.741 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143833]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:41.741 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 04:39:41 smithi069 sudo[143833]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:41.972 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145487]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:41.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145487]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:41.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145487]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:41.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145487]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:42.410 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145490]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:42.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145490]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:42.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145490]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:42.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 04:39:41 smithi118 sudo[145490]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:42 smithi005 sudo[122193]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:42 smithi005 sudo[122193]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:42 smithi005 sudo[122193]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:42 smithi005 sudo[122193]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:39:43.094+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f3cd80054a0 con 0x7f3ce8104300 2023-09-23T04:39:43.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 sudo[122197]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 sudo[122197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 sudo[122197]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 sudo[122197]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 ceph-mon[97638]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:43 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 ceph-mon[133931]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:43.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145493]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:43.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145493]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:43.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145493]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:43.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145493]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:43.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145496]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145496]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 sudo[145496]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 ceph-mon[134297]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:43.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:43.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:43 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.064 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 sudo[143836]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-23T04:39:44.064 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 sudo[143836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:44.064 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 sudo[143836]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:44.065 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:43 smithi069 sudo[143836]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/886322633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]': finished 2023-09-23T04:39:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:44 smithi005 ceph-mon[97638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/886322633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]': finished 2023-09-23T04:39:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:44 smithi118 ceph-mon[134297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 sudo[143839]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 sudo[143839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 sudo[143839]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 sudo[143839]: pam_unix(sudo:session): session closed for user root 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/886322633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b08d036a-cfc0-424d-963b-fc5fbeb8af79"}]': finished 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: osdmap e22: 4 total, 3 up, 4 in 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:39:44.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:44 smithi069 ceph-mon[133931]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-23T04:39:45.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:45 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2664264725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:45 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2664264725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:45 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2664264725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:39:46.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:46 smithi005 ceph-mon[97638]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:46.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:46 smithi118 ceph-mon[134297]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:46 smithi069 ceph-mon[133931]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:48 smithi005 ceph-mon[97638]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:48.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:48 smithi118 ceph-mon[134297]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:48 smithi069 ceph-mon[133931]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:39:50.774 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:50 smithi005 ceph-mon[97638]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-09-23T04:39:50.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:50 smithi118 ceph-mon[134297]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-09-23T04:39:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:50 smithi069 ceph-mon[133931]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 40 KiB/s wr, 3 op/s 2023-09-23T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T04:39:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T04:39:51.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-23T04:39:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:39:52.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:52 smithi005 ceph-mon[97638]: Deploying daemon osd.3 on smithi005 2023-09-23T04:39:52.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:52 smithi005 ceph-mon[97638]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2023-09-23T04:39:52.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:52 smithi118 ceph-mon[134297]: Deploying daemon osd.3 on smithi005 2023-09-23T04:39:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:52 smithi118 ceph-mon[134297]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2023-09-23T04:39:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:52 smithi069 ceph-mon[133931]: Deploying daemon osd.3 on smithi005 2023-09-23T04:39:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:52 smithi069 ceph-mon[133931]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 31 KiB/s wr, 2 op/s 2023-09-23T04:39:54.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:54 smithi005 ceph-mon[97638]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2023-09-23T04:39:54.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:54 smithi118 ceph-mon[134297]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2023-09-23T04:39:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:54 smithi069 ceph-mon[133931]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 975 B/s rd, 25 KiB/s wr, 2 op/s 2023-09-23T04:39:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:56 smithi005 ceph-mon[97638]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:39:56.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:56 smithi118 ceph-mon[134297]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:39:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:56 smithi069 ceph-mon[133931]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:39:58.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:58.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:39:58 smithi005 ceph-mon[97638]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:39:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:39:58 smithi118 ceph-mon[134297]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:39:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:39:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:39:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:39:58 smithi069 ceph-mon[133931]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:40:00.665 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2023-09-23T04:40:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.664+0000 7f3cdeffd700 1 -- 172.21.15.5:0/2881389388 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3ce802d270 con 0x7f3cd40a1590 2023-09-23T04:40:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 msgr2=0x7f3cd40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 0x7f3cd40a3a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8104160 tx=0x7f3ce4009260 comp rx=0 tx=0).stop 2023-09-23T04:40:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 msgr2=0x7f3ce8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b580 tx=0x7f3cd800f690 comp rx=0 tx=0).stop 2023-09-23T04:40:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 shutdown_connections 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3cd40a1590 0x7f3cd40a3a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ce8104ca0 0x7f3ce81a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ce8103100 0x7f3ce8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 --2- 172.21.15.5:0/2881389388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ce8104300 0x7f3ce8199320 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.667+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 >> 172.21.15.5:0/2881389388 conn(0x7f3ce80fe680 msgr2=0x7f3ce8107830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.668+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 shutdown_connections 2023-09-23T04:40:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:00.668+0000 7f3cf00ed700 1 -- 172.21.15.5:0/2881389388 wait complete. 2023-09-23T04:40:00.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:00.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:01.055 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.3.service 2023-09-23T04:40:01.061 INFO:tasks.cephadm:Deploying osd.4 on smithi069 with /dev/vg_nvme/lv_3... 2023-09-23T04:40:01.061 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T04:40:01.142 INFO:journalctl@ceph.osd.3.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:40:01.625 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:40:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:01 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:01 smithi005 ceph-mon[97638]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T04:40:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:01 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:01.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:01 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:01.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:01 smithi118 ceph-mon[134297]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T04:40:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:01 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:01 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:01 smithi069 ceph-mon[133931]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-23T04:40:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-mon[97638]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-mon[97638]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T04:40:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-mon[97638]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T04:40:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-mon[97638]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:40:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:02.906 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T04:40:02.653+0000 7fba12546700 -1 osd.3 0 waiting for initial osdmap 2023-09-23T04:40:02.906 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 04:40:02 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T04:40:02.661+0000 7fba0a333700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:40:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:02 smithi118 ceph-mon[134297]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:02 smithi118 ceph-mon[134297]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T04:40:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:02 smithi118 ceph-mon[134297]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T04:40:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:02 smithi118 ceph-mon[134297]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:40:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:02 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:02 smithi069 ceph-mon[133931]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:02 smithi069 ceph-mon[133931]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-23T04:40:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:02 smithi069 ceph-mon[133931]: osdmap e23: 4 total, 3 up, 4 in 2023-09-23T04:40:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:02 smithi069 ceph-mon[133931]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:40:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:02 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012905 s, 813 MB/s 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 2023-09-23T04:40:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-09-23T04:40:03.472 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2023-09-23T04:40:03.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:03 smithi069 ceph-mon[133931]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:40:03.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:03 smithi069 ceph-mon[133931]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T04:40:03.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:03 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:03 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:03 smithi005 ceph-mon[97638]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:40:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:03 smithi005 ceph-mon[97638]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T04:40:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:03 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:03 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:03 smithi118 ceph-mon[134297]: from='osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:40:03.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:03 smithi118 ceph-mon[134297]: osdmap e24: 4 total, 3 up, 4 in 2023-09-23T04:40:03.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:03 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:03.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:03 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:04.024 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] boot 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:04 smithi118 ceph-mon[134297]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T04:40:04.983 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:40:04.983 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] boot 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:04.984 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:04.985 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:04.985 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:04 smithi069 ceph-mon[133931]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: osd.3 [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] boot 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: osdmap e25: 4 total, 4 up, 4 in 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:05.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:04 smithi005 ceph-mon[97638]: osdmap e26: 4 total, 4 up, 4 in 2023-09-23T04:40:05.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/822357942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 msgr2=0x7ff4f8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:05.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/822357942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f8108200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b0d0 tx=0x7ff4f000b3e0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/822357942 shutdown_connections 2023-09-23T04:40:05.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/822357942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f8108200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.173 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/822357942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f80ffdb0 0x7ff4f81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/822357942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f8100fb0 0x7ff4f8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/822357942 >> 172.21.15.69:0/822357942 conn(0x7ff4f80fb330 msgr2=0x7ff4f80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:05.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/822357942 shutdown_connections 2023-09-23T04:40:05.174 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/822357942 wait complete. 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 Processor -- start 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.173+0000 7ff4ffbe5700 1 -- start start 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f80ffdb0 0x7ff4f8069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f8100fb0 0x7ff4f819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81a0b40 con 0x7ff4f80ffdb0 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81a0cb0 con 0x7ff4f8101950 2023-09-23T04:40:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81a0e70 con 0x7ff4f8100fb0 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58146/0 (socket says 172.21.15.69:58146) 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 -- 172.21.15.69:0/458152343 learned_addr learned my addr 172.21.15.69:0/458152343 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fd180700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f8100fb0 0x7ff4f819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fd981700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f80ffdb0 0x7ff4f8069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 -- 172.21.15.69:0/458152343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f8100fb0 msgr2=0x7ff4f819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f8100fb0 0x7ff4f819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 -- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f80ffdb0 msgr2=0x7ff4f8069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f80ffdb0 0x7ff4f8069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 -- 172.21.15.69:0/458152343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009d00 con 0x7ff4f8101950 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4fe182700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007a00 tx=0x7ff4f00094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000e040 con 0x7ff4f8101950 2023-09-23T04:40:05.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4f0003e10 con 0x7ff4f8101950 2023-09-23T04:40:05.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0012b10 con 0x7ff4f8101950 2023-09-23T04:40:05.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f81a10f0 con 0x7ff4f8101950 2023-09-23T04:40:05.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.174+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81a1580 con 0x7ff4f8101950 2023-09-23T04:40:05.178 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.175+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f81073b0 con 0x7ff4f8101950 2023-09-23T04:40:05.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.176+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7ff4f0012c90 con 0x7ff4f8101950 2023-09-23T04:40:05.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.177+0000 7ff4eeffd700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 0x7ff4e40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:05.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.177+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7ff4f00cdcc0 con 0x7ff4f8101950 2023-09-23T04:40:05.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.178+0000 7ff4fd981700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 0x7ff4e40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:05.179 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.178+0000 7ff4fd981700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 0x7ff4e40a3ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:05.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.181+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff4f0098540 con 0x7ff4f8101950 2023-09-23T04:40:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:05.379+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4f802d230 con 0x7ff4e40a1620 2023-09-23T04:40:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:05 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi005 2023-09-23T04:40:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:05 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 8636M 2023-09-23T04:40:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:05 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi005 2023-09-23T04:40:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:05 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 8636M 2023-09-23T04:40:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:05 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi005 2023-09-23T04:40:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:05 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 8636M 2023-09-23T04:40:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:06 smithi069 ceph-mon[133931]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:06 smithi069 ceph-mon[133931]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:06 smithi069 ceph-mon[133931]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T04:40:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:06 smithi005 ceph-mon[97638]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:06 smithi005 ceph-mon[97638]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:06 smithi005 ceph-mon[97638]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T04:40:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:06 smithi118 ceph-mon[134297]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:06 smithi118 ceph-mon[134297]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:06 smithi118 ceph-mon[134297]: osdmap e27: 4 total, 4 up, 4 in 2023-09-23T04:40:07.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:07 smithi069 ceph-mon[133931]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T04:40:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:07 smithi005 ceph-mon[97638]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T04:40:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:07 smithi118 ceph-mon[134297]: osdmap e28: 4 total, 4 up, 4 in 2023-09-23T04:40:08.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/1074301530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]': finished 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:08.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:08 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/4188892834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/1074301530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]': finished 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T04:40:09.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:09.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:08 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/4188892834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: osdmap e29: 4 total, 4 up, 4 in 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/1074301530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]: dispatch 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c71626-c985-43df-92a1-aa005836c3cf"}]': finished 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: osdmap e30: 5 total, 4 up, 5 in 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:08 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/4188892834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:10 smithi069 ceph-mon[133931]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:40:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:10 smithi005 ceph-mon[97638]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:40:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:10 smithi118 ceph-mon[134297]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:40:12.859 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:12 smithi069 ceph-mon[133931]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-23T04:40:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:12 smithi005 ceph-mon[97638]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-23T04:40:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:12 smithi118 ceph-mon[134297]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-23T04:40:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:14 smithi069 ceph-mon[133931]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-23T04:40:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T04:40:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:14 smithi005 ceph-mon[97638]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-23T04:40:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T04:40:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:14 smithi118 ceph-mon[134297]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-23T04:40:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-23T04:40:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:15.856 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:15 smithi069 ceph-mon[133931]: Deploying daemon osd.4 on smithi069 2023-09-23T04:40:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:15 smithi005 ceph-mon[97638]: Deploying daemon osd.4 on smithi069 2023-09-23T04:40:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:15 smithi118 ceph-mon[134297]: Deploying daemon osd.4 on smithi069 2023-09-23T04:40:16.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:16 smithi069 ceph-mon[133931]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:16 smithi005 ceph-mon[97638]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:16 smithi118 ceph-mon[134297]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:18 smithi069 ceph-mon[133931]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T04:40:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:18 smithi005 ceph-mon[97638]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T04:40:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:18 smithi118 ceph-mon[134297]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-23T04:40:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:20 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.842+0000 7ff4eeffd700 1 -- 172.21.15.69:0/458152343 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff4f802d230 con 0x7ff4e40a1620 2023-09-23T04:40:20.843 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 4 on host 'smithi069' 2023-09-23T04:40:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 msgr2=0x7ff4e40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 0x7ff4e40a3ae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006fd0 tx=0x7ff4e8008040 comp rx=0 tx=0).stop 2023-09-23T04:40:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 msgr2=0x7ff4f81a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007a00 tx=0x7ff4f00094c0 comp rx=0 tx=0).stop 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 shutdown_connections 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff4e40a1620 0x7ff4e40a3ae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4f8101950 0x7ff4f81a0490 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4f80ffdb0 0x7ff4f8069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 --2- 172.21.15.69:0/458152343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff4f8100fb0 0x7ff4f819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 >> 172.21.15.69:0/458152343 conn(0x7ff4f80fb330 msgr2=0x7ff4f81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 shutdown_connections 2023-09-23T04:40:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:40:20.845+0000 7ff4ffbe5700 1 -- 172.21.15.69:0/458152343 wait complete. 2023-09-23T04:40:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-23T04:40:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:20 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:20 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:21.301 DEBUG:teuthology.orchestra.run.smithi069:osd.4> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.4.service 2023-09-23T04:40:21.302 INFO:tasks.cephadm:Deploying osd.5 on smithi118 with /dev/vg_nvme/lv_3... 2023-09-23T04:40:21.303 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-23T04:40:21.333 INFO:journalctl@ceph.osd.4.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:40:21.865 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:40:22.050 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.050 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.050 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:22.050 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:21 smithi069 ceph-mon[133931]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:22.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:22.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:21 smithi118 ceph-mon[134297]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:21 smithi005 ceph-mon[97638]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:22.327 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 04:40:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T04:40:22.255+0000 7f90c7a58640 -1 osd.4 0 log_to_monitors true 2023-09-23T04:40:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:22 smithi069 ceph-mon[133931]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:22.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:22 smithi069 ceph-mon[133931]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:22 smithi005 ceph-mon[97638]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:22 smithi005 ceph-mon[97638]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:22 smithi118 ceph-mon[134297]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:22 smithi118 ceph-mon[134297]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00807848 s, 1.3 GB/s 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 2023-09-23T04:40:23.345 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2023-09-23T04:40:23.718 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi118:vg_nvme/lv_3 2023-09-23T04:40:24.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T04:40:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T04:40:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:23 smithi005 ceph-mon[97638]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T04:40:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T04:40:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:23 smithi118 ceph-mon[134297]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:24.241 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T04:40:23.825+0000 7f90bd0d0700 -1 osd.4 0 waiting for initial osdmap 2023-09-23T04:40:24.241 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T04:40:23.832+0000 7f90b8ec5700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:40:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-23T04:40:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: from='osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: osdmap e31: 5 total, 4 up, 5 in 2023-09-23T04:40:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:40:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:23 smithi069 ceph-mon[133931]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:24.274 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi069 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 8584M 2023-09-23T04:40:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi069 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 8584M 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: osdmap e32: 5 total, 4 up, 5 in 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi069 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 8584M 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:25.467 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/225249721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102080 msgr2=0x7f8dcc102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/225249721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102080 0x7f8dcc102500 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009990 tx=0x7f8dc8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/225249721 shutdown_connections 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/225249721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dcc102a40 0x7f8dcc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/225249721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc100e90 0x7f8dcc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/225249721 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102080 0x7f8dcc102500 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.466+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/225249721 >> 172.21.15.118:0/225249721 conn(0x7f8dcc0fc4a0 msgr2=0x7f8dcc0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/225249721 shutdown_connections 2023-09-23T04:40:25.468 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/225249721 wait complete. 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 Processor -- start 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- start start 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dcc100e90 0x7f8dcc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc102080 0x7f8dcc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc19e6e0 con 0x7f8dcc100e90 2023-09-23T04:40:25.470 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc19e820 con 0x7f8dcc102080 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd3c1e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc19e9b0 con 0x7f8dcc102a40 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd21bb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd21bb700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52304/0 (socket says 172.21.15.118:52304) 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.467+0000 7f8dd21bb700 1 -- 172.21.15.118:0/451182983 learned_addr learned my addr 172.21.15.118:0/451182983 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd11b9700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc102080 0x7f8dcc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 -- 172.21.15.118:0/451182983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc102080 msgr2=0x7f8dcc197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc102080 0x7f8dcc197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 -- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dcc100e90 msgr2=0x7f8dcc196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dcc100e90 0x7f8dcc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 -- 172.21.15.118:0/451182983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80095f0 con 0x7f8dcc102a40 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd21bb700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400b9a0 tx=0x7f8dc400bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:25.471 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400c930 con 0x7f8dcc102a40 2023-09-23T04:40:25.472 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dc4014440 con 0x7f8dcc102a40 2023-09-23T04:40:25.472 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc19ec90 con 0x7f8dcc102a40 2023-09-23T04:40:25.472 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40075d0 con 0x7f8dcc102a40 2023-09-23T04:40:25.472 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.468+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc19f190 con 0x7f8dcc102a40 2023-09-23T04:40:25.472 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.469+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc10c730 con 0x7f8dcc102a40 2023-09-23T04:40:25.473 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.470+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f8dc40145b0 con 0x7f8dcc102a40 2023-09-23T04:40:25.473 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.471+0000 7f8dc2ffd700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 0x7f8db80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:25.473 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.471+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8dc4006e40 con 0x7f8dcc102a40 2023-09-23T04:40:25.473 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.471+0000 7f8dd19ba700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 0x7f8db80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:25.473 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.472+0000 7f8dd19ba700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 0x7f8db80a3a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc101bf0 tx=0x7f8dbc009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:25.475 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.475+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8dc4095fa0 con 0x7f8dcc102a40 2023-09-23T04:40:25.669 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:25.669+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dcc061260 con 0x7f8db80a15a0 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] boot 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:40:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:40:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] boot 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: osd.4 [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] boot 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: osdmap e33: 5 total, 5 up, 5 in 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-09-23T04:40:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:26 smithi005 ceph-mon[97638]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T04:40:27.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:26 smithi118 ceph-mon[134297]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T04:40:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:26 smithi069 ceph-mon[133931]: osdmap e34: 5 total, 5 up, 5 in 2023-09-23T04:40:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:27 smithi005 ceph-mon[97638]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:27 smithi118 ceph-mon[134297]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:28.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:27 smithi069 ceph-mon[133931]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/4212634134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]': finished 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/2149309308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/4212634134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]': finished 2023-09-23T04:40:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T04:40:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/2149309308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/4212634134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]: dispatch 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35af4018-9f6b-4950-85d4-4af32e09ef2a"}]': finished 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: osdmap e35: 6 total, 5 up, 6 in 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/2149309308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:29 smithi005 ceph-mon[97638]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:29 smithi118 ceph-mon[134297]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:29 smithi069 ceph-mon[133931]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:32 smithi005 ceph-mon[97638]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:32 smithi118 ceph-mon[134297]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:32 smithi069 ceph-mon[133931]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:34 smithi005 ceph-mon[97638]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T04:40:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:34 smithi118 ceph-mon[134297]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:34.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T04:40:34.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:34 smithi069 ceph-mon[133931]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-23T04:40:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:35.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:35 smithi005 ceph-mon[97638]: Deploying daemon osd.5 on smithi118 2023-09-23T04:40:35.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:35 smithi118 ceph-mon[134297]: Deploying daemon osd.5 on smithi118 2023-09-23T04:40:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:35 smithi069 ceph-mon[133931]: Deploying daemon osd.5 on smithi118 2023-09-23T04:40:36.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:36 smithi005 ceph-mon[97638]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:36.907 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:36 smithi118 ceph-mon[134297]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:36 smithi069 ceph-mon[133931]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:38.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:38 smithi005 ceph-mon[97638]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:38.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:38 smithi118 ceph-mon[134297]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:38 smithi069 ceph-mon[133931]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:39.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:39.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:39.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:40.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:40.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:40.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:40.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:41.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.018 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.018 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:41.018 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:41.018 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:41.282 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 5 on host 'smithi118' 2023-09-23T04:40:41.283 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.282+0000 7f8dc2ffd700 1 -- 172.21.15.118:0/451182983 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8dcc061260 con 0x7f8db80a15a0 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 msgr2=0x7f8db80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 0x7f8db80a3a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc101bf0 tx=0x7f8dbc009470 comp rx=0 tx=0).stop 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 msgr2=0x7f8dcc19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400b9a0 tx=0x7f8dc400bcb0 comp rx=0 tx=0).stop 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 shutdown_connections 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8db80a15a0 0x7f8db80a3a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:41.285 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8dcc102a40 0x7f8dcc19e110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:41.286 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dcc100e90 0x7f8dcc196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:41.286 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 --2- 172.21.15.118:0/451182983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8dcc102080 0x7f8dcc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:41.286 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 >> 172.21.15.118:0/451182983 conn(0x7f8dcc0fc4a0 msgr2=0x7f8dcc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:41.286 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 shutdown_connections 2023-09-23T04:40:41.286 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:40:41.285+0000 7f8dd3c1e700 1 -- 172.21.15.118:0/451182983 wait complete. 2023-09-23T04:40:41.864 DEBUG:teuthology.orchestra.run.smithi118:osd.5> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.5.service 2023-09-23T04:40:41.866 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2023-09-23T04:40:41.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T04:40:41.913 INFO:journalctl@ceph.osd.5.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:40:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:42 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:42 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:42 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:42 smithi118 ceph-mon[134297]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:40:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:42 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:42 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:42 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:42 smithi005 ceph-mon[97638]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:42.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:42 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:42 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:40:42.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:42 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:42.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:42 smithi069 ceph-mon[133931]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:42.864 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 04:40:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T04:40:42.635+0000 7f35ecef3640 -1 osd.5 0 log_to_monitors true 2023-09-23T04:40:43.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:43 smithi005 ceph-mon[97638]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:43.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:43 smithi005 ceph-mon[97638]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:43.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:43 smithi118 ceph-mon[134297]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:43.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:43 smithi118 ceph-mon[134297]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:43.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:43 smithi069 ceph-mon[133931]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:43.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:43 smithi069 ceph-mon[133931]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-23T04:40:44.411 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T04:40:44.285+0000 7f35e256b700 -1 osd.5 0 waiting for initial osdmap 2023-09-23T04:40:44.411 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T04:40:44.291+0000 7f35de360700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:40:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T04:40:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T04:40:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:44.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:44 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0475888 s, 220 MB/s 2023-09-23T04:40:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-09-23T04:40:44.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:44.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:44.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:44 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: osdmap e36: 6 total, 5 up, 6 in 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-09-23T04:40:44.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:40:44.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:44 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:40:45.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi118 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 8584M 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi118 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 8584M 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi118 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 8584M 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: osdmap e37: 6 total, 5 up, 6 in 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] boot 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T04:40:46.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:46 smithi005 ceph-mon[97638]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] boot 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T04:40:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:46 smithi118 ceph-mon[134297]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: osd.5 [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] boot 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: osdmap e38: 6 total, 6 up, 6 in 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: osdmap e39: 6 total, 6 up, 6 in 2023-09-23T04:40:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:46 smithi069 ceph-mon[133931]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 -- 172.21.15.5:0/942926818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 msgr2=0x7fa2f41024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/942926818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f41024d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009990 tx=0x7fa2e4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 -- 172.21.15.5:0/942926818 shutdown_connections 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/942926818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 0x7fa2f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/942926818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 0x7fa2f4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/942926818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f41024d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 -- 172.21.15.5:0/942926818 >> 172.21.15.5:0/942926818 conn(0x7fa2f40fc410 msgr2=0x7fa2f40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 -- 172.21.15.5:0/942926818 shutdown_connections 2023-09-23T04:40:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.263+0000 7fa2fc01a700 1 -- 172.21.15.5:0/942926818 wait complete. 2023-09-23T04:40:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.264+0000 7fa2fc01a700 1 Processor -- start 2023-09-23T04:40:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.264+0000 7fa2fc01a700 1 -- start start 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 0x7fa2f4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 0x7fa2f419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419e780 con 0x7fa2f4102050 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419e8c0 con 0x7fa2f4102a10 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fc01a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419ea80 con 0x7fa2f4100e60 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56966/0 (socket says 172.21.15.5:56966) 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 -- 172.21.15.5:0/1165553090 learned_addr learned my addr 172.21.15.5:0/1165553090 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2fa5b7700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 0x7fa2f419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f9db6700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 0x7fa2f4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 -- 172.21.15.5:0/1165553090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 msgr2=0x7fa2f4196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 0x7fa2f4196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 -- 172.21.15.5:0/1165553090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 msgr2=0x7fa2f419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:40:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 0x7fa2f419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:40:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.265+0000 7fa2f95b5700 1 -- 172.21.15.5:0/1165553090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e40095f0 con 0x7fa2f4102050 2023-09-23T04:40:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.266+0000 7fa2f95b5700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4004730 tx=0x7fa2e4004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.266+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e401d070 con 0x7fa2f4102050 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.266+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f419ed00 con 0x7fa2f4102050 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.266+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e4021410 con 0x7fa2f4102050 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.266+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4017650 con 0x7fa2f4102050 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.267+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f419f120 con 0x7fa2f4102050 2023-09-23T04:40:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.268+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f410c730 con 0x7fa2f4102050 2023-09-23T04:40:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.270+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa2e4021580 con 0x7fa2f4102050 2023-09-23T04:40:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.271+0000 7fa2eaffd700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 0x7fa2e00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:40:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.271+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fa2e40cd310 con 0x7fa2f4102050 2023-09-23T04:40:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.271+0000 7fa2f9db6700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 0x7fa2e00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:40:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.272+0000 7fa2f9db6700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 0x7fa2e00a3a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4198110 tx=0x7fa2f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:40:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.276+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa2e4097790 con 0x7fa2f4102050 2023-09-23T04:40:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:40:47.496+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2f4061260 con 0x7fa2e00a15a0 2023-09-23T04:40:47.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:47 smithi005 ceph-mon[97638]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T04:40:47.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:47 smithi118 ceph-mon[134297]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T04:40:47.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:47 smithi069 ceph-mon[133931]: osdmap e40: 6 total, 6 up, 6 in 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:48 smithi118 ceph-mon[134297]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:48.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T04:40:48.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:48.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:48.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:48.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:48.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:48 smithi005 ceph-mon[97638]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:48.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: osdmap e41: 6 total, 6 up, 6 in 2023-09-23T04:40:48.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:40:48.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:40:48.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:40:48.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:48.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:48 smithi069 ceph-mon[133931]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:49.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:49 smithi118 ceph-mon[134297]: osdmap e42: 6 total, 6 up, 6 in 2023-09-23T04:40:49.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:49 smithi069 ceph-mon[133931]: osdmap e42: 6 total, 6 up, 6 in 2023-09-23T04:40:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:49 smithi005 ceph-mon[97638]: osdmap e42: 6 total, 6 up, 6 in 2023-09-23T04:40:50.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:50 smithi069 ceph-mon[133931]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-23T04:40:50.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:50 smithi005 ceph-mon[97638]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-23T04:40:50.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:50 smithi118 ceph-mon[134297]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-09-23T04:40:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:51 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]: dispatch 2023-09-23T04:40:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:51 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]': finished 2023-09-23T04:40:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:51 smithi069 ceph-mon[133931]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T04:40:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:40:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:51 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3014285586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:51 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]: dispatch 2023-09-23T04:40:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:51 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]': finished 2023-09-23T04:40:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:51 smithi005 ceph-mon[97638]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T04:40:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:40:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:51 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3014285586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:51 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]: dispatch 2023-09-23T04:40:51.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:51 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2435939369' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a9d6647-95c6-4cf0-ac70-52cb2639c1b4"}]': finished 2023-09-23T04:40:51.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:51 smithi118 ceph-mon[134297]: osdmap e43: 7 total, 6 up, 7 in 2023-09-23T04:40:51.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:40:51.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:51 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3014285586' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:40:52.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:52 smithi069 ceph-mon[133931]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-09-23T04:40:52.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:52 smithi005 ceph-mon[97638]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-09-23T04:40:52.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:52 smithi118 ceph-mon[134297]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-09-23T04:40:54.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:54 smithi005 ceph-mon[97638]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:40:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:54 smithi118 ceph-mon[134297]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:40:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:54 smithi069 ceph-mon[133931]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:40:56.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:56 smithi005 ceph-mon[97638]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:56.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:56 smithi118 ceph-mon[134297]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:56 smithi069 ceph-mon[133931]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:40:58.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:58 smithi005 ceph-mon[97638]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T04:40:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:40:58 smithi005 ceph-mon[97638]: Deploying daemon osd.6 on smithi005 2023-09-23T04:40:58.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:58 smithi118 ceph-mon[134297]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T04:40:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:40:58 smithi118 ceph-mon[134297]: Deploying daemon osd.6 on smithi005 2023-09-23T04:40:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:58 smithi069 ceph-mon[133931]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:40:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-23T04:40:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:40:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:40:58 smithi069 ceph-mon[133931]: Deploying daemon osd.6 on smithi005 2023-09-23T04:41:00.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:00 smithi005 ceph-mon[97638]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:00.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:00 smithi118 ceph-mon[134297]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:00 smithi069 ceph-mon[133931]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:02 smithi005 ceph-mon[97638]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:02.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:02 smithi118 ceph-mon[134297]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:02 smithi069 ceph-mon[133931]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:04 smithi118 ceph-mon[134297]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:04 smithi069 ceph-mon[133931]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:05.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:04 smithi005 ceph-mon[97638]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:05.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:05.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:05.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:06 smithi005 ceph-mon[97638]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:06 smithi118 ceph-mon[134297]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:06 smithi069 ceph-mon[133931]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:07.459 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2023-09-23T04:41:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.458+0000 7fa2eaffd700 1 -- 172.21.15.5:0/1165553090 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa2f4061260 con 0x7fa2e00a15a0 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.460+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 msgr2=0x7fa2e00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.460+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 0x7fa2e00a3a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4198110 tx=0x7fa2f0009260 comp rx=0 tx=0).stop 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 msgr2=0x7fa2f4197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4004730 tx=0x7fa2e4004760 comp rx=0 tx=0).stop 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 shutdown_connections 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa2e00a15a0 0x7fa2e00a3a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2f4102a10 0x7fa2f419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2f4100e60 0x7fa2f4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:07.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 --2- 172.21.15.5:0/1165553090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f4102050 0x7fa2f4197170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 >> 172.21.15.5:0/1165553090 conn(0x7fa2f40fc410 msgr2=0x7fa2f40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 shutdown_connections 2023-09-23T04:41:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:07.461+0000 7fa2fc01a700 1 -- 172.21.15.5:0/1165553090 wait complete. 2023-09-23T04:41:07.868 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.6.service 2023-09-23T04:41:07.872 INFO:tasks.cephadm:Deploying osd.7 on smithi069 with /dev/vg_nvme/lv_2... 2023-09-23T04:41:07.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T04:41:07.953 INFO:journalctl@ceph.osd.6.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:41:08.154 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T04:41:07.989+0000 7fb8b9b07640 -1 osd.6 0 log_to_monitors true 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:07 smithi005 ceph-mon[97638]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:08.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:07 smithi118 ceph-mon[134297]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:08.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:08.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:07 smithi069 ceph-mon[133931]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:08.448 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:41:09.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:08 smithi005 ceph-mon[97638]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T04:41:09.155 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:08 smithi069 ceph-mon[133931]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T04:41:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:08 smithi118 ceph-mon[134297]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.010996 s, 954 MB/s 2023-09-23T04:41:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-09-23T04:41:10.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-mon[97638]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T04:41:10.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-mon[97638]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T04:41:10.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-mon[97638]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:41:10.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:10.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-mon[97638]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:10.156 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T04:41:09.902+0000 7fb8af17f700 -1 osd.6 0 waiting for initial osdmap 2023-09-23T04:41:10.156 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 04:41:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T04:41:09.918+0000 7fb8a8f70700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:41:10.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:09 smithi118 ceph-mon[134297]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T04:41:10.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:09 smithi118 ceph-mon[134297]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T04:41:10.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:09 smithi118 ceph-mon[134297]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:41:10.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:09 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:10.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:09 smithi118 ceph-mon[134297]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:09 smithi069 ceph-mon[133931]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-23T04:41:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:09 smithi069 ceph-mon[133931]: osdmap e44: 7 total, 6 up, 7 in 2023-09-23T04:41:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:09 smithi069 ceph-mon[133931]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:41:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:09 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:09 smithi069 ceph-mon[133931]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-09-23T04:41:10.402 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2023-09-23T04:41:10.955 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:41:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:10 smithi005 ceph-mon[97638]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:41:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:10 smithi005 ceph-mon[97638]: osdmap e45: 7 total, 6 up, 7 in 2023-09-23T04:41:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:10 smithi118 ceph-mon[134297]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:41:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:10 smithi118 ceph-mon[134297]: osdmap e45: 7 total, 6 up, 7 in 2023-09-23T04:41:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:11.167 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:10 smithi069 ceph-mon[133931]: from='osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:41:11.167 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:10 smithi069 ceph-mon[133931]: osdmap e45: 7 total, 6 up, 7 in 2023-09-23T04:41:11.167 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:11.167 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] boot 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi005 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 5757M 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:12.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:11 smithi118 ceph-mon[134297]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 -- 172.21.15.69:0/125333419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102090 msgr2=0x7f0dbc102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 --2- 172.21.15.69:0/125333419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102090 0x7f0dbc102510 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009990 tx=0x7f0db0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.181 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 -- 172.21.15.69:0/125333419 shutdown_connections 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 --2- 172.21.15.69:0/125333419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc102a50 0x7f0dbc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 --2- 172.21.15.69:0/125333419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc100ea0 0x7f0dbc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 --2- 172.21.15.69:0/125333419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102090 0x7f0dbc102510 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 -- 172.21.15.69:0/125333419 >> 172.21.15.69:0/125333419 conn(0x7f0dbc0fc490 msgr2=0x7f0dbc0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 -- 172.21.15.69:0/125333419 shutdown_connections 2023-09-23T04:41:12.182 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 -- 172.21.15.69:0/125333419 wait complete. 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.181+0000 7f0dc1733700 1 Processor -- start 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 -- start start 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc100ea0 0x7f0dbc196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc102090 0x7f0dbc1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:12.184 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19e700 con 0x7f0dbc100ea0 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19e840 con 0x7f0dbc102a50 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc19e9d0 con 0x7f0dbc102090 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58256/0 (socket says 172.21.15.69:58256) 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 -- 172.21.15.69:0/3580442563 learned_addr learned my addr 172.21.15.69:0/3580442563 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbaffd700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc100ea0 0x7f0dbc196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dba7fc700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc102090 0x7f0dbc1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 -- 172.21.15.69:0/3580442563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc102090 msgr2=0x7f0dbc1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc102090 0x7f0dbc1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 -- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc100ea0 msgr2=0x7f0dbc196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:12.185 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc100ea0 0x7f0dbc196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 -- 172.21.15.69:0/3580442563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00095f0 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dbb7fe700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007d70 tx=0x7f0db400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400ad40 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc19ecb0 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db40128e0 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.182+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011780 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.183+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc19f1b0 con 0x7f0dbc102a50 2023-09-23T04:41:12.186 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.184+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc107e10 con 0x7f0dbc102a50 2023-09-23T04:41:12.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.185+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0db4012a50 con 0x7f0dbc102a50 2023-09-23T04:41:12.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.185+0000 7f0da3fff700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 0x7f0da40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:12.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.185+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f0db40cc540 con 0x7f0dbc102a50 2023-09-23T04:41:12.187 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.186+0000 7f0dbaffd700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 0x7f0da40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:12.188 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.186+0000 7f0dbaffd700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 0x7f0da40a3a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005fd0 tx=0x7f0dac005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:12.190 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.190+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0db4096840 con 0x7f0dbc102a50 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] boot 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi005 2023-09-23T04:41:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 5757M 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:11 smithi069 ceph-mon[133931]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:12.391 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:12.391+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dbc02d220 con 0x7f0da40a15a0 2023-09-23T04:41:12.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: osd.6 [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] boot 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: osdmap e46: 7 total, 7 up, 7 in 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi005 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 5757M 2023-09-23T04:41:12.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:12.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:12.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:12.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:11 smithi005 ceph-mon[97638]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:13.198 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:12 smithi069 ceph-mon[133931]: osdmap e47: 7 total, 7 up, 7 in 2023-09-23T04:41:13.198 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:12 smithi069 ceph-mon[133931]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:13.198 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:13.198 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:13.198 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:13.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:12 smithi005 ceph-mon[97638]: osdmap e47: 7 total, 7 up, 7 in 2023-09-23T04:41:13.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:12 smithi005 ceph-mon[97638]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:13.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:13.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:13.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:12 smithi118 ceph-mon[134297]: osdmap e47: 7 total, 7 up, 7 in 2023-09-23T04:41:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:12 smithi118 ceph-mon[134297]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:13 smithi069 ceph-mon[133931]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:14.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:13 smithi005 ceph-mon[97638]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:14.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:13 smithi118 ceph-mon[134297]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:14 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/773341118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:14 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:14 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]': finished 2023-09-23T04:41:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:14 smithi069 ceph-mon[133931]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T04:41:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:14 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/773341118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:14 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:14 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]': finished 2023-09-23T04:41:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:14 smithi005 ceph-mon[97638]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T04:41:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:15.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:14 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/773341118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:14 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]: dispatch 2023-09-23T04:41:15.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:14 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "496c61c7-6762-44b0-8361-bc62e51fb6cb"}]': finished 2023-09-23T04:41:15.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:14 smithi118 ceph-mon[134297]: osdmap e48: 8 total, 7 up, 8 in 2023-09-23T04:41:15.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:15 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/2617297411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:15 smithi069 ceph-mon[133931]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:16.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/2617297411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:15 smithi005 ceph-mon[97638]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:16.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:15 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/2617297411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:16.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:15 smithi118 ceph-mon[134297]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:18.740 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:18 smithi069 ceph-mon[133931]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:18 smithi005 ceph-mon[97638]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:18 smithi118 ceph-mon[134297]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:20.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:20 smithi005 ceph-mon[97638]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:20.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:20 smithi118 ceph-mon[134297]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:20 smithi069 ceph-mon[133931]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T04:41:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:21.946 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T04:41:21.947 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-23T04:41:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:22.960 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:22 smithi069 ceph-mon[133931]: Deploying daemon osd.7 on smithi069 2023-09-23T04:41:22.960 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:22 smithi069 ceph-mon[133931]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:23.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:22 smithi005 ceph-mon[97638]: Deploying daemon osd.7 on smithi069 2023-09-23T04:41:23.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:22 smithi005 ceph-mon[97638]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:22 smithi118 ceph-mon[134297]: Deploying daemon osd.7 on smithi069 2023-09-23T04:41:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:22 smithi118 ceph-mon[134297]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:24.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:24 smithi069 ceph-mon[133931]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:24 smithi005 ceph-mon[97638]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:24 smithi118 ceph-mon[134297]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:26.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:26.474 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:26 smithi069 ceph-mon[133931]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:26.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:26 smithi005 ceph-mon[97638]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:26 smithi118 ceph-mon[134297]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:28.059 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.057+0000 7f0da3fff700 1 -- 172.21.15.69:0/3580442563 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0dbc02d220 con 0x7f0da40a15a0 2023-09-23T04:41:28.059 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 7 on host 'smithi069' 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 msgr2=0x7f0da40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 0x7f0da40a3a60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005fd0 tx=0x7f0dac005dd0 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 msgr2=0x7f0dbc19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007d70 tx=0x7f0db400a340 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 shutdown_connections 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0da40a15a0 0x7f0da40a3a60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0dbc102a50 0x7f0dbc19e130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc100ea0 0x7f0dbc196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 --2- 172.21.15.69:0/3580442563 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0dbc102090 0x7f0dbc1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:28.061 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.059+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 >> 172.21.15.69:0/3580442563 conn(0x7f0dbc0fc490 msgr2=0x7f0dbc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:28.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.060+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 shutdown_connections 2023-09-23T04:41:28.062 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:41:28.060+0000 7f0dc1733700 1 -- 172.21.15.69:0/3580442563 wait complete. 2023-09-23T04:41:28.442 DEBUG:teuthology.orchestra.run.smithi069:osd.7> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.7.service 2023-09-23T04:41:28.444 INFO:tasks.cephadm:Deploying osd.8 on smithi118 with /dev/vg_nvme/lv_2... 2023-09-23T04:41:28.444 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-23T04:41:28.474 INFO:journalctl@ceph.osd.7.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:28.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:28.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:28.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:28.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:28.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:29.022 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:41:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:29.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:29.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:29.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:29.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:29.664 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 04:41:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T04:41:29.297+0000 7f455d82b640 -1 osd.7 0 log_to_monitors true 2023-09-23T04:41:29.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:29 smithi118 ceph-mon[134297]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:29.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:29 smithi118 ceph-mon[134297]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:29 smithi069 ceph-mon[133931]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:29 smithi069 ceph-mon[133931]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:29 smithi005 ceph-mon[97638]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:29 smithi005 ceph-mon[97638]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-23T04:41:30.488 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-23T04:41:30.488 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-23T04:41:30.488 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2023-09-23T04:41:30.488 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2023-09-23T04:41:30.489 INFO:teuthology.orchestra.run.smithi118.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00827772 s, 1.3 GB/s 2023-09-23T04:41:30.489 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:30.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:30 smithi118 ceph-mon[134297]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:30.877 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi118:vg_nvme/lv_2 2023-09-23T04:41:30.978 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:30.978 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T04:41:30.979 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T04:41:30.979 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:30.979 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:30.979 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-mon[133931]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:30.979 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T04:41:30.673+0000 7f4552ea3700 -1 osd.7 0 waiting for initial osdmap 2023-09-23T04:41:30.979 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 04:41:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T04:41:30.681+0000 7f454e497700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: osdmap e49: 8 total, 7 up, 8 in 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: from='osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:30 smithi005 ceph-mon[97638]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:41:31.432 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: osdmap e50: 8 total, 7 up, 8 in 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:31.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:31.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:31 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:41:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: osdmap e50: 8 total, 7 up, 8 in 2023-09-23T04:41:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:31.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:31 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: osdmap e50: 8 total, 7 up, 8 in 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:31 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:32.640 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 -- 172.21.15.118:0/3839259061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad4100fb0 msgr2=0x7f7ad4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/3839259061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad4100fb0 0x7f7ad4101410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009990 tx=0x7f7ac8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 -- 172.21.15.118:0/3839259061 shutdown_connections 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/3839259061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/3839259061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/3839259061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad4100fb0 0x7f7ad4101410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 -- 172.21.15.118:0/3839259061 >> 172.21.15.118:0/3839259061 conn(0x7f7ad40fb330 msgr2=0x7f7ad40fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:32.641 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 -- 172.21.15.118:0/3839259061 shutdown_connections 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 -- 172.21.15.118:0/3839259061 wait complete. 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.639+0000 7f7ad9d92700 1 Processor -- start 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 -- start start 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad4100fb0 0x7f7ad4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41a0790 con 0x7f7ad4100fb0 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41a08d0 con 0x7f7ad4101950 2023-09-23T04:41:32.642 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41a0a90 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52410/0 (socket says 172.21.15.118:52410) 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 -- 172.21.15.118:0/1572523145 learned_addr learned my addr 172.21.15.118:0/1572523145 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad3fff700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 -- 172.21.15.118:0/1572523145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 msgr2=0x7f7ad41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 -- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad4100fb0 msgr2=0x7f7ad4199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad4100fb0 0x7f7ad4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:32.643 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 -- 172.21.15.118:0/1572523145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80095f0 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad3fff700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad37fe700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c990 tx=0x7f7ac400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4007b00 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad41a0d70 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac4007c60 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.640+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4018660 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.644 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.641+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad41a11b0 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.645 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.641+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad410b620 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.645 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.643+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f7ac400f450 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.645 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.643+0000 7f7ad0ff9700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 0x7f7abc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:32.646 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.643+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f7ac40cc4c0 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.646 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.644+0000 7f7ad2ffd700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 0x7f7abc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:32.646 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.644+0000 7f7ad2ffd700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 0x7f7abc0a3a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80098e0 tx=0x7f7ac8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:32.646 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.647+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7ac4096680 con 0x7f7ad40ffdb0 2023-09-23T04:41:32.844 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:32.845+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ad4069070 con 0x7f7abc0a15a0 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi069 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 5722M 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] boot 2023-09-23T04:41:32.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T04:41:32.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:32.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:32 smithi118 ceph-mon[134297]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi069 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 5722M 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] boot 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:32 smithi069 ceph-mon[133931]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi069 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 5722M 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: osd.7 [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] boot 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: osdmap e51: 8 total, 8 up, 8 in 2023-09-23T04:41:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:41:33.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:32 smithi005 ceph-mon[97638]: osdmap e52: 8 total, 8 up, 8 in 2023-09-23T04:41:33.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:33.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:33.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:33.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:33 smithi118 ceph-mon[134297]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T04:41:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:33 smithi069 ceph-mon[133931]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T04:41:34.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:34.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:34.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:34.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:33 smithi005 ceph-mon[97638]: osdmap e53: 8 total, 8 up, 8 in 2023-09-23T04:41:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:34 smithi069 ceph-mon[133931]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:34 smithi069 ceph-mon[133931]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:34 smithi005 ceph-mon[97638]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:34 smithi005 ceph-mon[97638]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:34 smithi118 ceph-mon[134297]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:34 smithi118 ceph-mon[134297]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/627265106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]': finished 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T04:41:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/627265106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]': finished 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T04:41:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: osdmap e54: 8 total, 8 up, 8 in 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/627265106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]: dispatch 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aea8c15-9de3-4519-9f86-93b891d108e7"}]': finished 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: osdmap e55: 9 total, 8 up, 9 in 2023-09-23T04:41:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:36 smithi069 ceph-mon[133931]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:36 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/3254004822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:36 smithi005 ceph-mon[97638]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/3254004822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:36 smithi118 ceph-mon[134297]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:36 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/3254004822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:38.941 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:38 smithi118 ceph-mon[134297]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:38 smithi069 ceph-mon[133931]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:38 smithi005 ceph-mon[97638]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:40 smithi069 ceph-mon[133931]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:41.075 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:40 smithi118 ceph-mon[134297]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:40 smithi005 ceph-mon[97638]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:41.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:41 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T04:41:41.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:41 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:41 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T04:41:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:41 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-23T04:41:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:42.967 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:42 smithi118 ceph-mon[134297]: Deploying daemon osd.8 on smithi118 2023-09-23T04:41:42.967 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:42 smithi118 ceph-mon[134297]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:42 smithi069 ceph-mon[133931]: Deploying daemon osd.8 on smithi118 2023-09-23T04:41:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:42 smithi069 ceph-mon[133931]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:42 smithi005 ceph-mon[97638]: Deploying daemon osd.8 on smithi118 2023-09-23T04:41:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:42 smithi005 ceph-mon[97638]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:44.974 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:44 smithi118 ceph-mon[134297]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:44.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:44 smithi069 ceph-mon[133931]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:44 smithi005 ceph-mon[97638]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:46.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:46 smithi118 ceph-mon[134297]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:46.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:46 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:46.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:46 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:46.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:46 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:46 smithi069 ceph-mon[133931]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:46 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:46 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:46 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:46 smithi005 ceph-mon[97638]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.460 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 8 on host 'smithi118' 2023-09-23T04:41:48.460 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.459+0000 7f7ad0ff9700 1 -- 172.21.15.118:0/1572523145 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7ad4069070 con 0x7f7abc0a15a0 2023-09-23T04:41:48.462 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 msgr2=0x7f7abc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:48.462 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 0x7f7abc0a3a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80098e0 tx=0x7f7ac8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:41:48.462 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 msgr2=0x7f7ad4198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c990 tx=0x7f7ac400cd50 comp rx=0 tx=0).stop 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 shutdown_connections 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7abc0a15a0 0x7f7abc0a3a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ad4101950 0x7f7ad41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7ad40ffdb0 0x7f7ad4198c40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 --2- 172.21.15.118:0/1572523145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ad4100fb0 0x7f7ad4199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 >> 172.21.15.118:0/1572523145 conn(0x7f7ad40fb330 msgr2=0x7f7ad4068960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.461+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 shutdown_connections 2023-09-23T04:41:48.463 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:41:48.462+0000 7f7ad9d92700 1 -- 172.21.15.118:0/1572523145 wait complete. 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.821 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:48.823 DEBUG:teuthology.orchestra.run.smithi118:osd.8> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.8.service 2023-09-23T04:41:48.825 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2023-09-23T04:41:48.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T04:41:48.853 INFO:journalctl@ceph.osd.8.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:41:48.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:48.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:48.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:41:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:41:49.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 04:41:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T04:41:49.720+0000 7fd3cc314640 -1 osd.8 0 log_to_monitors true 2023-09-23T04:41:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:50 smithi005 ceph-mon[97638]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:50 smithi005 ceph-mon[97638]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:50 smithi005 ceph-mon[97638]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:50 smithi069 ceph-mon[133931]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:50 smithi069 ceph-mon[133931]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:50 smithi069 ceph-mon[133931]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:51.000 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:50 smithi118 ceph-mon[134297]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:51.000 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:50 smithi118 ceph-mon[134297]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:51.000 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:50 smithi118 ceph-mon[134297]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0316128 s, 332 MB/s 2023-09-23T04:41:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:51 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:51.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:51.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:51.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:51.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:51 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-23T04:41:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: osdmap e56: 9 total, 8 up, 9 in 2023-09-23T04:41:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:41:52.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:41:52.163 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T04:41:51.673+0000 7fd3c198c700 -1 osd.8 0 waiting for initial osdmap 2023-09-23T04:41:52.163 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 04:41:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T04:41:51.683+0000 7fd3b9779700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:41:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:41:52.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi118 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 5722M 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:52.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi118 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 5722M 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T04:41:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:52.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi118 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 5722M 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: osdmap e57: 9 total, 8 up, 9 in 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:53.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:53 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:41:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:53 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:41:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:53 smithi005 ceph-mon[97638]: osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] boot 2023-09-23T04:41:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:53 smithi005 ceph-mon[97638]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T04:41:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:53 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:41:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:53 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:41:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:53 smithi069 ceph-mon[133931]: osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] boot 2023-09-23T04:41:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:53 smithi069 ceph-mon[133931]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T04:41:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:53 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:53 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:41:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:53 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:41:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:53 smithi118 ceph-mon[134297]: osd.8 [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] boot 2023-09-23T04:41:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:53 smithi118 ceph-mon[134297]: osdmap e58: 9 total, 9 up, 9 in 2023-09-23T04:41:54.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:53 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:41:54.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 -- 172.21.15.5:0/2518839802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 msgr2=0x7fd2c01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/2518839802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c01024d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009990 tx=0x7fd2b4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 -- 172.21.15.5:0/2518839802 shutdown_connections 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/2518839802 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 0x7fd2c0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/2518839802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/2518839802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c01024d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.189+0000 7fd2c50c2700 1 -- 172.21.15.5:0/2518839802 >> 172.21.15.5:0/2518839802 conn(0x7fd2c00fc410 msgr2=0x7fd2c00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:41:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.190+0000 7fd2c50c2700 1 -- 172.21.15.5:0/2518839802 shutdown_connections 2023-09-23T04:41:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.190+0000 7fd2c50c2700 1 -- 172.21.15.5:0/2518839802 wait complete. 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.191+0000 7fd2c50c2700 1 Processor -- start 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.191+0000 7fd2c50c2700 1 -- start start 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 0x7fd2c0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0073ed0 con 0x7fd2c0102050 2023-09-23T04:41:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0074040 con 0x7fd2c0100e60 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2c50c2700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a0c90 con 0x7fd2c0102a10 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:46608/0 (socket says 172.21.15.5:46608) 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 -- 172.21.15.5:0/1270946273 learned_addr learned my addr 172.21.15.5:0/1270946273 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2be59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45818/0 (socket says 172.21.15.5:45818) 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bf59e700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 0x7fd2c0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 -- 172.21.15.5:0/1270946273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 msgr2=0x7fd2c0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 0x7fd2c0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 -- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 msgr2=0x7fd2c0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.192+0000 7fd2bed9d700 1 -- 172.21.15.5:0/1270946273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40095f0 con 0x7fd2c0100e60 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2bed9d700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000e970 tx=0x7fd2b000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0009790 con 0x7fd2c0100e60 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01a0f70 con 0x7fd2c0100e60 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b000c7e0 con 0x7fd2c0100e60 2023-09-23T04:41:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0003c70 con 0x7fd2c0100e60 2023-09-23T04:41:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.193+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a1390 con 0x7fd2c0100e60 2023-09-23T04:41:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.194+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c010c730 con 0x7fd2c0100e60 2023-09-23T04:41:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.196+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fd2b000cc50 con 0x7fd2c0100e60 2023-09-23T04:41:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.198+0000 7fd2a7fff700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 0x7fd2a80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:41:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.198+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fd2b0014070 con 0x7fd2c0100e60 2023-09-23T04:41:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.198+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 0x7fd2a80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:41:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.198+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 0x7fd2a80a3a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400b2e0 tx=0x7fd2b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:41:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.200+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd2b0096670 con 0x7fd2c0100e60 2023-09-23T04:41:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:41:54.424+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2c0061260 con 0x7fd2a80a15a0 2023-09-23T04:41:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:54 smithi069 ceph-mon[133931]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:54 smithi069 ceph-mon[133931]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T04:41:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:54 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:54 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:54 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:54 smithi005 ceph-mon[97638]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:55.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:54 smithi005 ceph-mon[97638]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T04:41:55.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:54 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:55.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:54 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:55.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:54 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:54 smithi118 ceph-mon[134297]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-09-23T04:41:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:54 smithi118 ceph-mon[134297]: osdmap e59: 9 total, 9 up, 9 in 2023-09-23T04:41:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:54 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:41:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:54 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:41:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:54 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:41:55.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:55 smithi005 ceph-mon[97638]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:55.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:55 smithi005 ceph-mon[97638]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T04:41:55.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:55 smithi005 ceph-mon[97638]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T04:41:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:55 smithi069 ceph-mon[133931]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:55 smithi069 ceph-mon[133931]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T04:41:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:55 smithi069 ceph-mon[133931]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T04:41:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:55 smithi118 ceph-mon[134297]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:41:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:55 smithi118 ceph-mon[134297]: osdmap e60: 9 total, 9 up, 9 in 2023-09-23T04:41:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:55 smithi118 ceph-mon[134297]: osdmap e61: 9 total, 9 up, 9 in 2023-09-23T04:41:56.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:56 smithi005 ceph-mon[97638]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:41:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:56 smithi005 ceph-mon[97638]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T04:41:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:56 smithi069 ceph-mon[133931]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:41:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:56 smithi069 ceph-mon[133931]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T04:41:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:56 smithi118 ceph-mon[134297]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:41:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:56 smithi118 ceph-mon[134297]: osdmap e62: 9 total, 9 up, 9 in 2023-09-23T04:41:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:57 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]: dispatch 2023-09-23T04:41:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:57 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]': finished 2023-09-23T04:41:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:57 smithi069 ceph-mon[133931]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T04:41:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:57 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:41:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:57 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]: dispatch 2023-09-23T04:41:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:57 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]': finished 2023-09-23T04:41:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:57 smithi005 ceph-mon[97638]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T04:41:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:57 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:41:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:57 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]: dispatch 2023-09-23T04:41:58.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:57 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3083315612' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "463a1b28-e1f2-485e-a11c-16f788e17414"}]': finished 2023-09-23T04:41:58.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:57 smithi118 ceph-mon[134297]: osdmap e63: 10 total, 9 up, 10 in 2023-09-23T04:41:58.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:57 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:41:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:58 smithi069 ceph-mon[133931]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-09-23T04:41:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:41:58 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/532153224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:58 smithi005 ceph-mon[97638]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-09-23T04:41:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:41:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/532153224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:41:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:58 smithi118 ceph-mon[134297]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 147 KiB/s, 0 objects/s recovering 2023-09-23T04:41:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:41:58 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/532153224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:00.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:00 smithi069 ceph-mon[133931]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-23T04:42:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:00 smithi005 ceph-mon[97638]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-23T04:42:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:00 smithi118 ceph-mon[134297]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-23T04:42:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:02 smithi069 ceph-mon[133931]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:42:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:02 smithi005 ceph-mon[97638]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:42:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:02 smithi118 ceph-mon[134297]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-09-23T04:42:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:04 smithi069 ceph-mon[133931]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:04 smithi005 ceph-mon[97638]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:04 smithi118 ceph-mon[134297]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T04:42:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:06.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T04:42:06.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-23T04:42:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:06.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:06 smithi005 ceph-mon[97638]: Deploying daemon osd.9 on smithi005 2023-09-23T04:42:06.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:06 smithi005 ceph-mon[97638]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-23T04:42:06.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:06 smithi069 ceph-mon[133931]: Deploying daemon osd.9 on smithi005 2023-09-23T04:42:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:06 smithi069 ceph-mon[133931]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-23T04:42:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:06 smithi118 ceph-mon[134297]: Deploying daemon osd.9 on smithi005 2023-09-23T04:42:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:06 smithi118 ceph-mon[134297]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-09-23T04:42:08.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:08 smithi005 ceph-mon[97638]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:08.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:08 smithi069 ceph-mon[133931]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:08 smithi118 ceph-mon[134297]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:10 smithi069 ceph-mon[133931]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:10.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:10 smithi005 ceph-mon[97638]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:10 smithi118 ceph-mon[134297]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:12 smithi069 ceph-mon[133931]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:12.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:12.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:12 smithi005 ceph-mon[97638]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:12 smithi118 ceph-mon[134297]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.440+0000 7fd2a7fff700 1 -- 172.21.15.5:0/1270946273 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd2c0061260 con 0x7fd2a80a15a0 2023-09-23T04:42:14.441 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2023-09-23T04:42:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 msgr2=0x7fd2a80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 0x7fd2a80a3a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400b2e0 tx=0x7fd2b4005fb0 comp rx=0 tx=0).stop 2023-09-23T04:42:14.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 msgr2=0x7fd2c0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000e970 tx=0x7fd2b000ec80 comp rx=0 tx=0).stop 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 shutdown_connections 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd2a80a15a0 0x7fd2a80a3a60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2c0102a10 0x7fd2c0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2c0100e60 0x7fd2c0074bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 --2- 172.21.15.5:0/1270946273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0102050 0x7fd2c0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 >> 172.21.15.5:0/1270946273 conn(0x7fd2c00fc410 msgr2=0x7fd2c00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 shutdown_connections 2023-09-23T04:42:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:14.443+0000 7fd2c50c2700 1 -- 172.21.15.5:0/1270946273 wait complete. 2023-09-23T04:42:14.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:14 smithi005 ceph-mon[97638]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:14.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.949 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.949 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:14 smithi069 ceph-mon[133931]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:15.016 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.9.service 2023-09-23T04:42:15.020 INFO:tasks.cephadm:Deploying osd.10 on smithi069 with /dev/vg_nvme/lv_1... 2023-09-23T04:42:15.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T04:42:15.091 INFO:journalctl@ceph.osd.9.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:42:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:14 smithi118 ceph-mon[134297]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:15.404 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 04:42:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T04:42:15.164+0000 7f11f3e47640 -1 osd.9 0 log_to_monitors true 2023-09-23T04:42:15.588 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:42:15.981 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:15 smithi069 ceph-mon[133931]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T04:42:16.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:15 smithi005 ceph-mon[97638]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T04:42:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:15 smithi118 ceph-mon[134297]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-23T04:42:16.904 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T04:42:16.676+0000 7f11e94bf700 -1 osd.9 0 waiting for initial osdmap 2023-09-23T04:42:16.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T04:42:16.695+0000 7f11e52b4700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:42:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-mon[97638]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-mon[97638]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T04:42:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-mon[97638]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T04:42:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-mon[97638]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:42:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:16 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:16 smithi069 ceph-mon[133931]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:16 smithi069 ceph-mon[133931]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T04:42:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:16 smithi069 ceph-mon[133931]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T04:42:16.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:16 smithi069 ceph-mon[133931]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:42:16.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:16 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:17.094 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T04:42:17.095 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T04:42:17.095 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2023-09-23T04:42:17.095 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2023-09-23T04:42:17.095 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010975 s, 955 MB/s 2023-09-23T04:42:17.095 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2023-09-23T04:42:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:16 smithi118 ceph-mon[134297]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:16 smithi118 ceph-mon[134297]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-23T04:42:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:16 smithi118 ceph-mon[134297]: osdmap e64: 10 total, 9 up, 10 in 2023-09-23T04:42:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:16 smithi118 ceph-mon[134297]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-09-23T04:42:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:16 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:17.434 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2023-09-23T04:42:17.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:17 smithi069 ceph-mon[133931]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:42:17.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:17 smithi069 ceph-mon[133931]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T04:42:17.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:17.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:17 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:17.985 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T04:42:18.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:17 smithi005 ceph-mon[97638]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:42:18.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:17 smithi005 ceph-mon[97638]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T04:42:18.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:17 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:17 smithi118 ceph-mon[134297]: from='osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-09-23T04:42:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:17 smithi118 ceph-mon[134297]: osdmap e65: 10 total, 9 up, 10 in 2023-09-23T04:42:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:17 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] boot 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T04:42:18.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] boot 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T04:42:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: osd.9 [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] boot 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: osdmap e66: 10 total, 10 up, 10 in 2023-09-23T04:42:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:42:19.200 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 -- 172.21.15.69:0/39192740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102a40 msgr2=0x7f87f4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/39192740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102a40 0x7f87f4109360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00b0a0 tx=0x7f87ec00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 -- 172.21.15.69:0/39192740 shutdown_connections 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/39192740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102a40 0x7f87f4109360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/39192740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 0x7f87f41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/39192740 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102080 0x7f87f4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 -- 172.21.15.69:0/39192740 >> 172.21.15.69:0/39192740 conn(0x7f87f40fc4a0 msgr2=0x7f87f40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 -- 172.21.15.69:0/39192740 shutdown_connections 2023-09-23T04:42:19.201 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.200+0000 7f87f8bf6700 1 -- 172.21.15.69:0/39192740 wait complete. 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 Processor -- start 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 -- start start 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 0x7f87f4196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102a40 0x7f87f419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:19.203 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419e7e0 con 0x7f87f4100e90 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419e920 con 0x7f87f4102080 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f8bf6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419eae0 con 0x7f87f4102a40 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58402/0 (socket says 172.21.15.69:58402) 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f1d9b700 1 -- 172.21.15.69:0/289820673 learned_addr learned my addr 172.21.15.69:0/289820673 (peer_addr_for_me v2:172.21.15.69:0/0) 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f259c700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 0x7f87f4196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.201+0000 7f87f2d9d700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102a40 0x7f87f419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 -- 172.21.15.69:0/289820673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102a40 msgr2=0x7f87f419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102a40 0x7f87f419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 -- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 msgr2=0x7f87f4196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:19.204 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 0x7f87f4196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 -- 172.21.15.69:0/289820673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec009d00 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f1d9b700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c970 tx=0x7f87e400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004220 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e4004380 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f419edc0 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003b50 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.202+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f419f260 con 0x7f87f4102080 2023-09-23T04:42:19.205 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.203+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f4108470 con 0x7f87f4102080 2023-09-23T04:42:19.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.204+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f87e40076d0 con 0x7f87f4102080 2023-09-23T04:42:19.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.204+0000 7f87e37fe700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 0x7f87dc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:19.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.205+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f87e40cc650 con 0x7f87f4102080 2023-09-23T04:42:19.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.205+0000 7f87f259c700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 0x7f87dc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:19.206 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.205+0000 7f87f259c700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 0x7f87dc0a3a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f87f4197b40 tx=0x7f87e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:19.208 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.208+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f87e40963a0 con 0x7f87f4102080 2023-09-23T04:42:19.405 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:19.404+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f87f402d290 con 0x7f87dc0a15a0 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi005 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 4318M 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:19.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:19 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi005 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 4318M 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:20.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:20.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:19 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: osdmap e67: 10 total, 10 up, 10 in 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi005 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 4318M 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:20.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:20.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:20.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:20.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:19 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:20 smithi069 ceph-mon[133931]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:20 smithi069 ceph-mon[133931]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:20 smithi005 ceph-mon[97638]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:20 smithi005 ceph-mon[97638]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:20 smithi118 ceph-mon[134297]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:20 smithi118 ceph-mon[134297]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:21 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/3494082497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:21 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:21 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]': finished 2023-09-23T04:42:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:21 smithi069 ceph-mon[133931]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T04:42:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:21 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/3494082497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:21 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:21 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]': finished 2023-09-23T04:42:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:21 smithi005 ceph-mon[97638]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T04:42:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:21 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/3494082497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:21 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]: dispatch 2023-09-23T04:42:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:21 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ab3d09-beb2-4736-99b1-554ad9d15218"}]': finished 2023-09-23T04:42:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:21 smithi118 ceph-mon[134297]: osdmap e68: 11 total, 10 up, 11 in 2023-09-23T04:42:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:22.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:22 smithi118 ceph-mon[134297]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:22.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.69:0/2517856694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:22 smithi069 ceph-mon[133931]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.69:0/2517856694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:22 smithi005 ceph-mon[97638]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.69:0/2517856694' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:24 smithi069 ceph-mon[133931]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:24 smithi005 ceph-mon[97638]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:24 smithi118 ceph-mon[134297]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:26.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:26 smithi069 ceph-mon[133931]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:26.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:26 smithi005 ceph-mon[97638]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:26.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:26 smithi118 ceph-mon[134297]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:28.905 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:28 smithi069 ceph-mon[133931]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:28.905 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T04:42:28.905 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:28 smithi005 ceph-mon[97638]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T04:42:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:28 smithi118 ceph-mon[134297]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-23T04:42:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:29.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:29 smithi069 ceph-mon[133931]: Deploying daemon osd.10 on smithi069 2023-09-23T04:42:30.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:29 smithi005 ceph-mon[97638]: Deploying daemon osd.10 on smithi069 2023-09-23T04:42:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:29 smithi118 ceph-mon[134297]: Deploying daemon osd.10 on smithi069 2023-09-23T04:42:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:30 smithi069 ceph-mon[133931]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:30 smithi005 ceph-mon[97638]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:31.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:30 smithi118 ceph-mon[134297]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:32.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:32 smithi069 ceph-mon[133931]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:32 smithi005 ceph-mon[97638]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:32 smithi118 ceph-mon[134297]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.774+0000 7f87e37fe700 1 -- 172.21.15.69:0/289820673 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f87f402d290 con 0x7f87dc0a15a0 2023-09-23T04:42:34.775 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 10 on host 'smithi069' 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 msgr2=0x7f87dc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 0x7f87dc0a3a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f87f4197b40 tx=0x7f87e800b410 comp rx=0 tx=0).stop 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 msgr2=0x7f87f41971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c970 tx=0x7f87e400cc80 comp rx=0 tx=0).stop 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 shutdown_connections 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f87dc0a15a0 0x7f87dc0a3a60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f87f4102a40 0x7f87f419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87f4100e90 0x7f87f4196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 --2- 172.21.15.69:0/289820673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87f4102080 0x7f87f41971a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.776+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 >> 172.21.15.69:0/289820673 conn(0x7f87f40fc4a0 msgr2=0x7f87f41055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.777+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 shutdown_connections 2023-09-23T04:42:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2023-09-23T04:42:34.777+0000 7f87f8bf6700 1 -- 172.21.15.69:0/289820673 wait complete. 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:34 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:34 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:35.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:34 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.174 DEBUG:teuthology.orchestra.run.smithi069:osd.10> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.10.service 2023-09-23T04:42:35.176 INFO:tasks.cephadm:Deploying osd.11 on smithi118 with /dev/vg_nvme/lv_1... 2023-09-23T04:42:35.176 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-23T04:42:35.204 INFO:journalctl@ceph.osd.10.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:42:35.749 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:42:35.854 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.854 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.854 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:35.979 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.979 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:35.979 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:36.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:36.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 04:42:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T04:42:36.182+0000 7f0987713640 -1 osd.10 0 log_to_monitors true 2023-09-23T04:42:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:36 smithi069 ceph-mon[133931]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:36 smithi069 ceph-mon[133931]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:36 smithi069 ceph-mon[133931]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:36 smithi005 ceph-mon[97638]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:36 smithi005 ceph-mon[97638]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:36 smithi005 ceph-mon[97638]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:36 smithi118 ceph-mon[134297]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:36 smithi118 ceph-mon[134297]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:36 smithi118 ceph-mon[134297]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0082634 s, 1.3 GB/s 2023-09-23T04:42:37.207 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2023-09-23T04:42:37.606 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch daemon add osd smithi118:vg_nvme/lv_1 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T04:42:37.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:37 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:37.991 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T04:42:37.734+0000 7f097cd8b700 -1 osd.10 0 waiting for initial osdmap 2023-09-23T04:42:37.991 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T04:42:37.748+0000 7f0976b7c700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:42:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T04:42:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T04:42:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:42:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T04:42:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:37 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:38.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: osdmap e69: 11 total, 10 up, 11 in 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: osdmap e70: 11 total, 10 up, 11 in 2023-09-23T04:42:38.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:37 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:38.171 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:42:39.005 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:39.005 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.005 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi069 2023-09-23T04:42:39.005 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] boot 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T04:42:39.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi069 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:39.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] boot 2023-09-23T04:42:39.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T04:42:39.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi069 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: osd.10 [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] boot 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: osdmap e71: 11 total, 11 up, 11 in 2023-09-23T04:42:39.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:42:39.446 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- 172.21.15.118:0/3013258597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54101940 msgr2=0x7f9f54103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3013258597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54101940 0x7f9f54103d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009990 tx=0x7f9f48009ca0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- 172.21.15.118:0/3013258597 shutdown_connections 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3013258597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f54106ba0 0x7f9f54109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3013258597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54101940 0x7f9f54103d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3013258597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54104270 0x7f9f54106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- 172.21.15.118:0/3013258597 >> 172.21.15.118:0/3013258597 conn(0x7f9f540fb330 msgr2=0x7f9f540fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- 172.21.15.118:0/3013258597 shutdown_connections 2023-09-23T04:42:39.447 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- 172.21.15.118:0/3013258597 wait complete. 2023-09-23T04:42:39.449 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 Processor -- start 2023-09-23T04:42:39.449 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 -- start start 2023-09-23T04:42:39.449 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.446+0000 7f9f5b028700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f54101940 0x7f9f54196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:39.449 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54106ba0 0x7f9f5419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419e750 con 0x7f9f54101940 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419e890 con 0x7f9f54106ba0 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419ea20 con 0x7f9f54104270 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52560/0 (socket says 172.21.15.118:52560) 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 -- 172.21.15.118:0/3670176149 learned_addr learned my addr 172.21.15.118:0/3670176149 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 -- 172.21.15.118:0/3670176149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54106ba0 msgr2=0x7f9f5419e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54106ba0 0x7f9f5419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 -- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f54101940 msgr2=0x7f9f54196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:39.450 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f54101940 0x7f9f54196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 -- 172.21.15.118:0/3670176149 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480095f0 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f53fff700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40009b40 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4000b6e0 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000f620 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5419ed00 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.447+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5419f1b0 con 0x7f9f54104270 2023-09-23T04:42:39.451 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.448+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f540fceb0 con 0x7f9f54104270 2023-09-23T04:42:39.452 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.449+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f9f40010460 con 0x7f9f54104270 2023-09-23T04:42:39.452 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.450+0000 7f9f51ffb700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 0x7f9f440a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:39.452 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.450+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f9f400cc6f0 con 0x7f9f54104270 2023-09-23T04:42:39.452 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.450+0000 7f9f58dc4700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 0x7f9f440a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:39.452 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.451+0000 7f9f58dc4700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 0x7f9f440a3ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f480095c0 tx=0x7f9f48009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:39.455 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.454+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9f40096310 con 0x7f9f54104270 2023-09-23T04:42:39.652 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:39.651+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5402d270 con 0x7f9f440a1620 2023-09-23T04:42:40.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 4292M 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:39 smithi005 ceph-mon[97638]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T04:42:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:42:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:42:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 4292M 2023-09-23T04:42:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:39 smithi118 ceph-mon[134297]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T04:42:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:42:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:42:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 4292M 2023-09-23T04:42:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-23T04:42:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-23T04:42:40.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:40.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:39 smithi069 ceph-mon[133931]: osdmap e72: 11 total, 11 up, 11 in 2023-09-23T04:42:41.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:40 smithi118 ceph-mon[134297]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:41.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:40 smithi118 ceph-mon[134297]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:41.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:40 smithi118 ceph-mon[134297]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T04:42:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:40 smithi005 ceph-mon[97638]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:40 smithi005 ceph-mon[97638]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:40 smithi005 ceph-mon[97638]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T04:42:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:40 smithi069 ceph-mon[133931]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:40 smithi069 ceph-mon[133931]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:42:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:40 smithi069 ceph-mon[133931]: osdmap e73: 11 total, 11 up, 11 in 2023-09-23T04:42:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/411338158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]': finished 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:42 smithi005 ceph-mon[97638]: from='client.? 172.21.15.118:0/922028435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/411338158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]': finished 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:42 smithi118 ceph-mon[134297]: from='client.? 172.21.15.118:0/922028435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: osdmap e74: 11 total, 11 up, 11 in 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/411338158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]: dispatch 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6354fc3d-d3ae-491d-8592-1b9bb4336d15"}]': finished 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: osdmap e75: 12 total, 11 up, 12 in 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:42 smithi069 ceph-mon[133931]: from='client.? 172.21.15.118:0/922028435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-23T04:42:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:44 smithi005 ceph-mon[97638]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:42:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:44 smithi118 ceph-mon[134297]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:42:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:44 smithi069 ceph-mon[133931]: pgmap v225: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-23T04:42:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:46 smithi069 ceph-mon[133931]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-09-23T04:42:46.753 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:46 smithi118 ceph-mon[134297]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-09-23T04:42:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:46 smithi005 ceph-mon[97638]: pgmap v226: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-09-23T04:42:48.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:48 smithi118 ceph-mon[134297]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-09-23T04:42:48.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T04:42:48.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:48 smithi069 ceph-mon[133931]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-09-23T04:42:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T04:42:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:48 smithi005 ceph-mon[97638]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2023-09-23T04:42:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-23T04:42:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:49 smithi069 ceph-mon[133931]: Deploying daemon osd.11 on smithi118 2023-09-23T04:42:50.141 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:49 smithi118 ceph-mon[134297]: Deploying daemon osd.11 on smithi118 2023-09-23T04:42:50.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:49 smithi005 ceph-mon[97638]: Deploying daemon osd.11 on smithi118 2023-09-23T04:42:50.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:50 smithi118 ceph-mon[134297]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:50 smithi069 ceph-mon[133931]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:50 smithi005 ceph-mon[97638]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-23T04:42:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:52 smithi069 ceph-mon[133931]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:53.110 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:52 smithi118 ceph-mon[134297]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:52 smithi005 ceph-mon[97638]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:53.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:53 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:53.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:53 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:53.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:53 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:53 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:53 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:53 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:54.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:54.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:53 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:54.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:54 smithi118 ceph-mon[134297]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:54 smithi005 ceph-mon[97638]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:54 smithi069 ceph-mon[133931]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:55.420 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.420+0000 7f9f51ffb700 1 -- 172.21.15.118:0/3670176149 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9f5402d270 con 0x7f9f440a1620 2023-09-23T04:42:55.420 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 11 on host 'smithi118' 2023-09-23T04:42:55.422 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.421+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 msgr2=0x7f9f440a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:55.422 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.421+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 0x7f9f440a3ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f480095c0 tx=0x7f9f48009530 comp rx=0 tx=0).stop 2023-09-23T04:42:55.422 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.421+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 msgr2=0x7f9f54197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.421+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dd40 comp rx=0 tx=0).stop 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 shutdown_connections 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9f440a1620 0x7f9f440a3ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f54106ba0 0x7f9f5419e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f54101940 0x7f9f54196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 --2- 172.21.15.118:0/3670176149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9f54104270 0x7f9f54197100 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 >> 172.21.15.118:0/3670176149 conn(0x7f9f540fb330 msgr2=0x7f9f541025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 shutdown_connections 2023-09-23T04:42:55.423 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:42:55.422+0000 7f9f5b028700 1 -- 172.21.15.118:0/3670176149 wait complete. 2023-09-23T04:42:55.747 DEBUG:teuthology.orchestra.run.smithi118:osd.11> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.11.service 2023-09-23T04:42:55.749 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-23T04:42:55.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd stat -f json 2023-09-23T04:42:55.778 INFO:journalctl@ceph.osd.11.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:42:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:42:56.545 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.545 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.545 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-mon[134297]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:56.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:56 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:56 smithi005 ceph-mon[97638]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:56 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:56.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:56 smithi069 ceph-mon[133931]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:56.911 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 04:42:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T04:42:56.649+0000 7f9af7785640 -1 osd.11 0 log_to_monitors true 2023-09-23T04:42:57.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:57 smithi005 ceph-mon[97638]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:57.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:57 smithi005 ceph-mon[97638]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:57.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:57 smithi069 ceph-mon[133931]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:57.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:57 smithi069 ceph-mon[133931]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:57.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:57 smithi118 ceph-mon[134297]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:57 smithi118 ceph-mon[134297]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.002+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/484466085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4100e90 msgr2=0x7f0aa41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.002+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/484466085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4100e90 0x7f0aa41012b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009990 tx=0x7f0a94009ca0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/484466085 shutdown_connections 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/484466085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 0x7f0aa4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/484466085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4100e90 0x7f0aa41012b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/484466085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4102080 0x7f0aa4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/484466085 >> 172.21.15.5:0/484466085 conn(0x7f0aa40fc440 msgr2=0x7f0aa40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/484466085 shutdown_connections 2023-09-23T04:42:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.003+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/484466085 wait complete. 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 Processor -- start 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 -- start start 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 0x7f0aa4073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 0x7f0aa4073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4073ed0 con 0x7f0aa4102080 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4074040 con 0x7f0aa4102a40 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.004+0000 7f0aa8bb1700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa41a0c40 con 0x7f0aa4100e90 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55716/0 (socket says 172.21.15.5:55716) 2023-09-23T04:42:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 -- 172.21.15.5:0/853982913 learned_addr learned my addr 172.21.15.5:0/853982913 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 0x7f0aa4073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa2d9d700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 0x7f0aa4073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 -- 172.21.15.5:0/853982913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 msgr2=0x7f0aa4073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 0x7f0aa4073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 -- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 msgr2=0x7f0aa4073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 0x7f0aa4073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 -- 172.21.15.5:0/853982913 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940095f0 con 0x7f0aa4100e90 2023-09-23T04:42:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0a9bfff700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 0x7f0aa4073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.005+0000 7f0aa259c700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b620 tx=0x7f0a94005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.006+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9401d070 con 0x7f0aa4100e90 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.006+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa41a0ec0 con 0x7f0aa4100e90 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.006+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a94021410 con 0x7f0aa4100e90 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.006+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94017650 con 0x7f0aa4100e90 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.006+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa41a1270 con 0x7f0aa4100e90 2023-09-23T04:42:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.007+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa4107e70 con 0x7f0aa4100e90 2023-09-23T04:42:58.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.009+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0a94021580 con 0x7f0aa4100e90 2023-09-23T04:42:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.010+0000 7f0a9b7fe700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 0x7f0a900a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:42:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.010+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f0a940cdef0 con 0x7f0aa4100e90 2023-09-23T04:42:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.010+0000 7f0a9bfff700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 0x7f0a900a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:42:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.011+0000 7f0a9bfff700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 0x7f0a900a3a60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4100bc0 tx=0x7f0a8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:42:58.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.012+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0a94097aa0 con 0x7f0aa4100e90 2023-09-23T04:42:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.222+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0aa410c760 con 0x7f0aa4100e90 2023-09-23T04:42:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.222+0000 7f0a9b7fe700 1 -- 172.21.15.5:0/853982913 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0a9409b0c0 con 0x7f0aa4100e90 2023-09-23T04:42:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:42:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 msgr2=0x7f0a900a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 0x7f0a900a3a60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4100bc0 tx=0x7f0a8c009260 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 msgr2=0x7f0aa4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b620 tx=0x7f0a94005740 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 shutdown_connections 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a900a15a0 0x7f0a900a3a60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aa4102a40 0x7f0aa4073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0aa4100e90 0x7f0aa4074bc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 --2- 172.21.15.5:0/853982913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa4102080 0x7f0aa4073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 >> 172.21.15.5:0/853982913 conn(0x7f0aa40fc440 msgr2=0x7f0aa40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 shutdown_connections 2023-09-23T04:42:58.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:42:58.225+0000 7f0aa8bb1700 1 -- 172.21.15.5:0/853982913 wait complete. 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/853982913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:58.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:58 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1695444158,"num_in_osds":12,"osd_in_since":1695444161,"num_remapped_pgs":0} 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/853982913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:42:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:58.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:58 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.910 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T04:42:58.455+0000 7f9aecdfd700 -1 osd.11 0 waiting for initial osdmap 2023-09-23T04:42:58.911 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T04:42:58.467+0000 7f9ae5bec700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T04:42:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-23T04:42:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: osdmap e76: 12 total, 11 up, 12 in 2023-09-23T04:42:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/853982913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:42:58.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:58.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:58.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:58.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:58 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:59.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd stat -f json 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: Detected new or changed devices on smithi118 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 4292M 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:59.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:59.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:59.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:59.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:42:59 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: Detected new or changed devices on smithi118 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 4292M 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:59.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:59.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:59.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:59.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:42:59 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: Detected new or changed devices on smithi118 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 4292M 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: osdmap e77: 12 total, 11 up, 12 in 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:42:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:42:59.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:42:59.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:42:59.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:42:59 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:43:00.421 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: purged_snaps scrub starts 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: purged_snaps scrub ok 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] boot 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:43:00.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:00 smithi069 ceph-mon[133931]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: purged_snaps scrub starts 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: purged_snaps scrub ok 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] boot 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:43:00.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:00 smithi005 ceph-mon[97638]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:43:00.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: purged_snaps scrub starts 2023-09-23T04:43:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: purged_snaps scrub ok 2023-09-23T04:43:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: osd.11 [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] boot 2023-09-23T04:43:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: osdmap e78: 12 total, 12 up, 12 in 2023-09-23T04:43:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:43:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:00 smithi118 ceph-mon[134297]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-23T04:43:01.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:01 smithi069 ceph-mon[133931]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T04:43:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.816+0000 7f0424cf4700 1 -- 172.21.15.5:0/2604164180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102050 msgr2=0x7f04201024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.816+0000 7f0424cf4700 1 --2- 172.21.15.5:0/2604164180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102050 0x7f04201024d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0414009990 tx=0x7f0414009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 -- 172.21.15.5:0/2604164180 shutdown_connections 2023-09-23T04:43:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 --2- 172.21.15.5:0/2604164180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420102a10 0x7f0420109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 --2- 172.21.15.5:0/2604164180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420100e60 0x7f0420101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 --2- 172.21.15.5:0/2604164180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102050 0x7f04201024d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 -- 172.21.15.5:0/2604164180 >> 172.21.15.5:0/2604164180 conn(0x7f04200fc410 msgr2=0x7f04200fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 -- 172.21.15.5:0/2604164180 shutdown_connections 2023-09-23T04:43:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.817+0000 7f0424cf4700 1 -- 172.21.15.5:0/2604164180 wait complete. 2023-09-23T04:43:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 Processor -- start 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 -- start start 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420100e60 0x7f0420198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201a09f0 con 0x7f0420102a10 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201a0b30 con 0x7f0420102050 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.818+0000 7f0424cf4700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201a0cc0 con 0x7f0420100e60 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34392/0 (socket says 172.21.15.5:34392) 2023-09-23T04:43:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33236/0 (socket says 172.21.15.5:33236) 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 -- 172.21.15.5:0/3697132079 learned_addr learned my addr 172.21.15.5:0/3697132079 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041e59c700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420100e60 0x7f0420198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 -- 172.21.15.5:0/3697132079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420100e60 msgr2=0x7f0420198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420100e60 0x7f0420198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 -- 172.21.15.5:0/3697132079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 msgr2=0x7f04201993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 -- 172.21.15.5:0/3697132079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140095f0 con 0x7f0420102a10 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041dd9b700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.819+0000 7f041ed9d700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0418007f10 tx=0x7f041800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.820+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800dd20 con 0x7f0420102a10 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.820+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041800f040 con 0x7f0420102a10 2023-09-23T04:43:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.820+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04201a0fa0 con 0x7f0420102a10 2023-09-23T04:43:01.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.820+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180048a0 con 0x7f0420102a10 2023-09-23T04:43:01.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.820+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a13c0 con 0x7f0420102a10 2023-09-23T04:43:01.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.821+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04201934d0 con 0x7f0420102a10 2023-09-23T04:43:01.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.823+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0418003a00 con 0x7f0420102a10 2023-09-23T04:43:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.824+0000 7f040f7fe700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 0x7f04080a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.824+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f04180cc2d0 con 0x7f0420102a10 2023-09-23T04:43:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.824+0000 7f041e59c700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 0x7f04080a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.825+0000 7f041e59c700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 0x7f04080a3a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0420101bc0 tx=0x7f041000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:01.828+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f041804eec0 con 0x7f0420102a10 2023-09-23T04:43:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:01 smithi005 ceph-mon[97638]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T04:43:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:01 smithi118 ceph-mon[134297]: osdmap e79: 12 total, 12 up, 12 in 2023-09-23T04:43:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.037+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f042010c6e0 con 0x7f0420102a10 2023-09-23T04:43:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.038+0000 7f040f7fe700 1 -- 172.21.15.5:0/3697132079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0418095cc0 con 0x7f0420102a10 2023-09-23T04:43:02.038 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 msgr2=0x7f04080a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 0x7f04080a3a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0420101bc0 tx=0x7f041000b410 comp rx=0 tx=0).stop 2023-09-23T04:43:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 msgr2=0x7f04201a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0418007f10 tx=0x7f041800d3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 shutdown_connections 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f04080a15a0 0x7f04080a3a60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0420102a10 0x7f04201a0420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0420100e60 0x7f0420198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 --2- 172.21.15.5:0/3697132079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0420102050 0x7f04201993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 >> 172.21.15.5:0/3697132079 conn(0x7f04200fc410 msgr2=0x7f0420107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 shutdown_connections 2023-09-23T04:43:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:02.040+0000 7f0424cf4700 1 -- 172.21.15.5:0/3697132079 wait complete. 2023-09-23T04:43:02.503 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1695444179,"num_in_osds":12,"osd_in_since":1695444161,"num_remapped_pgs":0} 2023-09-23T04:43:02.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T04:43:02.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:02 smithi069 ceph-mon[133931]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:02.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:02 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3697132079' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:43:02.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:02 smithi005 ceph-mon[97638]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:02.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:02 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3697132079' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:43:02.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:02 smithi118 ceph-mon[134297]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:02.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:02 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3697132079' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-23T04:43:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:43:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 -- 172.21.15.5:0/4219151307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 msgr2=0x7ffa1010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/4219151307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1010dd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b0a0 tx=0x7ffa0800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 -- 172.21.15.5:0/4219151307 shutdown_connections 2023-09-23T04:43:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/4219151307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1010dd50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/4219151307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 0x7ffa10101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/4219151307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 0x7ffa10105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 -- 172.21.15.5:0/4219151307 >> 172.21.15.5:0/4219151307 conn(0x7ffa100fc910 msgr2=0x7ffa100fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 -- 172.21.15.5:0/4219151307 shutdown_connections 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.633+0000 7ffa16bbc700 1 -- 172.21.15.5:0/4219151307 wait complete. 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.634+0000 7ffa16bbc700 1 Processor -- start 2023-09-23T04:43:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.634+0000 7ffa16bbc700 1 -- start start 2023-09-23T04:43:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 0x7ffa101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 0x7ffa101997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019e260 con 0x7ffa10106180 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019e3d0 con 0x7ffa101010a0 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa16bbc700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1019e590 con 0x7ffa10101a50 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa15159700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa15159700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34414/0 (socket says 172.21.15.5:34414) 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa15159700 1 -- 172.21.15.5:0/3126803534 learned_addr learned my addr 172.21.15.5:0/3126803534 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa14958700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 0x7ffa101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.635+0000 7ffa0ffff700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 0x7ffa101997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 -- 172.21.15.5:0/3126803534 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 msgr2=0x7ffa101997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 0x7ffa101997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 -- 172.21.15.5:0/3126803534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 msgr2=0x7ffa101992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 0x7ffa101992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 -- 172.21.15.5:0/3126803534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08009d00 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa15159700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009fd0 tx=0x7ffa0800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800e040 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1019e810 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa08004e60 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08005280 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.636+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101a60d0 con 0x7ffa10106180 2023-09-23T04:43:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.637+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1004eb20 con 0x7ffa10106180 2023-09-23T04:43:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.639+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7ffa08003a00 con 0x7ffa10106180 2023-09-23T04:43:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.640+0000 7ffa0dffb700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 0x7ffa000a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.641+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffa080ce2d0 con 0x7ffa10106180 2023-09-23T04:43:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.641+0000 7ffa14958700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 0x7ffa000a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.641+0000 7ffa14958700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 0x7ffa000a3a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c60 tx=0x7ffa04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.642+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ffa08097d50 con 0x7ffa10106180 2023-09-23T04:43:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.855+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa1019a600 con 0x7ffa10106180 2023-09-23T04:43:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.856+0000 7ffa0dffb700 1 -- 172.21.15.5:0/3126803534 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18328 (secure 0 0 0) 0x7ffa08003cb0 con 0x7ffa10106180 2023-09-23T04:43:04.858 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:04.858 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":79,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:00.465274+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 msgr2=0x7ffa000a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 0x7ffa000a3a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c60 tx=0x7ffa04009260 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 msgr2=0x7ffa1019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009fd0 tx=0x7ffa0800bce0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 shutdown_connections 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ffa000a15a0 0x7ffa000a3a60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa10106180 0x7ffa1019dc50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa101010a0 0x7ffa101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 --2- 172.21.15.5:0/3126803534 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffa10101a50 0x7ffa101997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 >> 172.21.15.5:0/3126803534 conn(0x7ffa100fc910 msgr2=0x7ffa100fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 shutdown_connections 2023-09-23T04:43:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:04.859+0000 7ffa16bbc700 1 -- 172.21.15.5:0/3126803534 wait complete. 2023-09-23T04:43:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:04 smithi069 ceph-mon[133931]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:05.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:04 smithi005 ceph-mon[97638]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:04 smithi118 ceph-mon[134297]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:05.321 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T04:43:05.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-09-23T04:43:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3126803534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:43:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3126803534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3126803534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:06.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:06 smithi005 ceph-mon[97638]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:06 smithi069 ceph-mon[133931]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:06 smithi118 ceph-mon[134297]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 -- 172.21.15.5:0/1423821085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544106180 msgr2=0x7fa54410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/1423821085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544106180 0x7fa54410dd50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00b0a0 tx=0x7fa53c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 -- 172.21.15.5:0/1423821085 shutdown_connections 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/1423821085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544106180 0x7fa54410dd50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/1423821085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5441010a0 0x7fa544101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/1423821085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 0x7fa544105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.451+0000 7fa54ae2a700 1 -- 172.21.15.5:0/1423821085 >> 172.21.15.5:0/1423821085 conn(0x7fa5440fc910 msgr2=0x7fa5440fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.452+0000 7fa54ae2a700 1 -- 172.21.15.5:0/1423821085 shutdown_connections 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.452+0000 7fa54ae2a700 1 -- 172.21.15.5:0/1423821085 wait complete. 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 Processor -- start 2023-09-23T04:43:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 -- start start 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 0x7fa5441997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa544106180 0x7fa54419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419e230 con 0x7fa5441010a0 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419e3a0 con 0x7fa544101a50 2023-09-23T04:43:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa54ae2a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54419e560 con 0x7fa544106180 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa548bc6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa548bc6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34430/0 (socket says 172.21.15.5:34430) 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa548bc6700 1 -- 172.21.15.5:0/886246810 learned_addr learned my addr 172.21.15.5:0/886246810 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.453+0000 7fa543fff700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 0x7fa5441997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa5493c7700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa544106180 0x7fa54419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 -- 172.21.15.5:0/886246810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa544106180 msgr2=0x7fa54419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa544106180 0x7fa54419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 -- 172.21.15.5:0/886246810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 msgr2=0x7fa5441997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 0x7fa5441997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 -- 172.21.15.5:0/886246810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c009d00 con 0x7fa5441010a0 2023-09-23T04:43:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa548bc6700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9a0 tx=0x7fa53800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538009790 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54419e840 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa538010430 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.454+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538018680 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.455+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5441a60a0 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.456+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54404eb20 con 0x7fa5441010a0 2023-09-23T04:43:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.457+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa5380105a0 con 0x7fa5441010a0 2023-09-23T04:43:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.459+0000 7fa541ffb700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 0x7fa5340a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.459+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa538014070 con 0x7fa5441010a0 2023-09-23T04:43:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.459+0000 7fa543fff700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 0x7fa5340a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:07.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.460+0000 7fa543fff700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 0x7fa5340a3a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa54419a890 tx=0x7fa530009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.463+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa538051ec0 con 0x7fa5441010a0 2023-09-23T04:43:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.669+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa544110f60 con 0x7fa5441010a0 2023-09-23T04:43:07.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.670+0000 7fa541ffb700 1 -- 172.21.15.5:0/886246810 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5380cee60 con 0x7fa5441010a0 2023-09-23T04:43:07.671 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 msgr2=0x7fa5340a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 0x7fa5340a3a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa54419a890 tx=0x7fa530009260 comp rx=0 tx=0).stop 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 msgr2=0x7fa544199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9a0 tx=0x7fa53800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 shutdown_connections 2023-09-23T04:43:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa5340a15a0 0x7fa5340a3a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa544106180 0x7fa54419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5441010a0 0x7fa544199270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 --2- 172.21.15.5:0/886246810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa544101a50 0x7fa5441997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 >> 172.21.15.5:0/886246810 conn(0x7fa5440fc910 msgr2=0x7fa5440fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 shutdown_connections 2023-09-23T04:43:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:07.672+0000 7fa54ae2a700 1 -- 172.21.15.5:0/886246810 wait complete. 2023-09-23T04:43:07.994 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:07 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/886246810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:08.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:07 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/886246810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:08.141 INFO:tasks.cephadm:Adding mds.a on smithi005 2023-09-23T04:43:08.141 INFO:tasks.cephadm:Adding mds.d on smithi005 2023-09-23T04:43:08.141 INFO:tasks.cephadm:Adding mds.g on smithi005 2023-09-23T04:43:08.141 INFO:tasks.cephadm:Adding mds.j on smithi005 2023-09-23T04:43:08.141 INFO:tasks.cephadm:Adding mds.b on smithi069 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.e on smithi069 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.h on smithi069 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.k on smithi069 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.c on smithi118 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.f on smithi118 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.i on smithi118 2023-09-23T04:43:08.142 INFO:tasks.cephadm:Adding mds.l on smithi118 2023-09-23T04:43:08.142 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l' 2023-09-23T04:43:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:07 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/886246810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:08.721 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T04:43:08.941 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:08 smithi118 ceph-mon[134297]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:08.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:08 smithi069 ceph-mon[133931]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:08 smithi005 ceph-mon[97638]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 -- 172.21.15.118:0/2113214981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 msgr2=0x7fe35c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 --2- 172.21.15.118:0/2113214981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c101410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe358009990 tx=0x7fe358009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 -- 172.21.15.118:0/2113214981 shutdown_connections 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 --2- 172.21.15.118:0/2113214981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 --2- 172.21.15.118:0/2113214981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 0x7fe35c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 --2- 172.21.15.118:0/2113214981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c101410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 -- 172.21.15.118:0/2113214981 >> 172.21.15.118:0/2113214981 conn(0x7fe35c0fb330 msgr2=0x7fe35c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 -- 172.21.15.118:0/2113214981 shutdown_connections 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.098+0000 7fe364154700 1 -- 172.21.15.118:0/2113214981 wait complete. 2023-09-23T04:43:10.100 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 Processor -- start 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 -- start start 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 0x7fe35c195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d750 con 0x7fe35c101950 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d890 con 0x7fe35c0ffdb0 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe364154700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19da20 con 0x7fe35c100fb0 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.118:52686/0 (socket says 172.21.15.118:52686) 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 -- 172.21.15.118:0/2211527632 learned_addr learned my addr 172.21.15.118:0/2211527632 (peer_addr_for_me v2:172.21.15.118:0/0) 2023-09-23T04:43:10.101 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3626f1700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 -- 172.21.15.118:0/2211527632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 msgr2=0x7fe35c195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe361ef0700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 0x7fe35c195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 0x7fe35c195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 -- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 msgr2=0x7fe35c19d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c19d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 -- 172.21.15.118:0/2211527632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580095f0 con 0x7fe35c100fb0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3626f1700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c19d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.099+0000 7fe3616ef700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe358003790 tx=0x7fe35800fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.100+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35801d070 con 0x7fe35c100fb0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.100+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35800be70 con 0x7fe35c100fb0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.100+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c19dca0 con 0x7fe35c100fb0 2023-09-23T04:43:10.102 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.100+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580177f0 con 0x7fe35c100fb0 2023-09-23T04:43:10.103 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.100+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c19e0b0 con 0x7fe35c100fb0 2023-09-23T04:43:10.103 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.101+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c107420 con 0x7fe35c100fb0 2023-09-23T04:43:10.103 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.101+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fe358022b30 con 0x7fe35c100fb0 2023-09-23T04:43:10.104 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.102+0000 7fe34effd700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 0x7fe3480a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:10.104 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.102+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3580cdee0 con 0x7fe35c100fb0 2023-09-23T04:43:10.104 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.102+0000 7fe361ef0700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 0x7fe3480a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:10.104 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.103+0000 7fe361ef0700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 0x7fe3480a3a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe350006fd0 tx=0x7fe350008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:10.106 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.107+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe358097a00 con 0x7fe35c100fb0 2023-09-23T04:43:10.304 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.305+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe35c02d2c0 con 0x7fe3480a15a0 2023-09-23T04:43:10.313 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.313+0000 7fe34effd700 1 -- 172.21.15.118:0/2211527632 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe35c02d2c0 con 0x7fe3480a15a0 2023-09-23T04:43:10.313 INFO:teuthology.orchestra.run.smithi118.stdout:Scheduled mds.all update... 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 msgr2=0x7fe3480a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 0x7fe3480a3a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe350006fd0 tx=0x7fe350008040 comp rx=0 tx=0).stop 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 msgr2=0x7fe35c196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe358003790 tx=0x7fe35800fab0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 shutdown_connections 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe3480a15a0 0x7fe3480a3a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe35c101950 0x7fe35c19d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe35c0ffdb0 0x7fe35c195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.315 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 --2- 172.21.15.118:0/2211527632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe35c100fb0 0x7fe35c196100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:10.316 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 >> 172.21.15.118:0/2211527632 conn(0x7fe35c0fb330 msgr2=0x7fe35c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:10.316 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 shutdown_connections 2023-09-23T04:43:10.316 INFO:teuthology.orchestra.run.smithi118.stderr:2023-09-23T04:43:10.315+0000 7fe364154700 1 -- 172.21.15.118:0/2211527632 wait complete. 2023-09-23T04:43:10.674 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.a.service 2023-09-23T04:43:10.680 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.d.service 2023-09-23T04:43:10.725 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.g.service 2023-09-23T04:43:10.765 INFO:journalctl@ceph.mds.a.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:43:10.769 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.j.service 2023-09-23T04:43:10.801 INFO:journalctl@ceph.mds.d.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:43:10.804 DEBUG:teuthology.orchestra.run.smithi069:mds.b> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.b.service 2023-09-23T04:43:10.807 DEBUG:teuthology.orchestra.run.smithi069:mds.e> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.e.service 2023-09-23T04:43:10.842 INFO:journalctl@ceph.mds.g.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:43:10.844 INFO:journalctl@ceph.mds.b.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:43:10.845 DEBUG:teuthology.orchestra.run.smithi069:mds.h> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.h.service 2023-09-23T04:43:10.874 INFO:journalctl@ceph.mds.e.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:43:10.875 INFO:journalctl@ceph.mds.j.smithi005.stdout:-- Logs begin at Sat 2023-09-23 04:20:01 UTC. -- 2023-09-23T04:43:10.875 DEBUG:teuthology.orchestra.run.smithi069:mds.k> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.k.service 2023-09-23T04:43:10.902 INFO:journalctl@ceph.mds.h.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:43:10.903 DEBUG:teuthology.orchestra.run.smithi118:mds.c> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.c.service 2023-09-23T04:43:10.906 DEBUG:teuthology.orchestra.run.smithi118:mds.f> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.f.service 2023-09-23T04:43:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:10 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:10.930 INFO:journalctl@ceph.mds.k.smithi069.stdout:-- Logs begin at Sat 2023-09-23 04:15:38 UTC. -- 2023-09-23T04:43:10.930 DEBUG:teuthology.orchestra.run.smithi118:mds.i> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.i.service 2023-09-23T04:43:10.935 INFO:journalctl@ceph.mds.c.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:43:10.936 DEBUG:teuthology.orchestra.run.smithi118:mds.l> sudo journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.l.service 2023-09-23T04:43:10.957 INFO:journalctl@ceph.mds.f.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:43:10.957 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-23T04:43:10.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:43:10.961 INFO:journalctl@ceph.mds.i.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:43:10.983 INFO:journalctl@ceph.mds.l.smithi118.stdout:-- Logs begin at Sat 2023-09-23 04:16:10 UTC. -- 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:10.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:10 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:11.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:11.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:11.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:11.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:11.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:11.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:11.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:11.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:11.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:10 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 -- 172.21.15.5:0/1338074307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102070 msgr2=0x7fe8941024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 --2- 172.21.15.5:0/1338074307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102070 0x7fe8941024f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8800098f0 tx=0x7fe880009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 -- 172.21.15.5:0/1338074307 shutdown_connections 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 --2- 172.21.15.5:0/1338074307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894102a30 0x7fe894109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 --2- 172.21.15.5:0/1338074307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894100e90 0x7fe8941012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 --2- 172.21.15.5:0/1338074307 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102070 0x7fe8941024f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 -- 172.21.15.5:0/1338074307 >> 172.21.15.5:0/1338074307 conn(0x7fe8940fc450 msgr2=0x7fe8940fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 -- 172.21.15.5:0/1338074307 shutdown_connections 2023-09-23T04:43:11.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.171+0000 7fe89afff700 1 -- 172.21.15.5:0/1338074307 wait complete. 2023-09-23T04:43:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.172+0000 7fe89afff700 1 Processor -- start 2023-09-23T04:43:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 -- start start 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 0x7fe894198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941a0980 con 0x7fe894102070 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941a0ac0 con 0x7fe894100e90 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89afff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941a0c50 con 0x7fe894102a30 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe893fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89959c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe893fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45218/0 (socket says 172.21.15.5:45218) 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe893fff700 1 -- 172.21.15.5:0/1427905762 learned_addr learned my addr 172.21.15.5:0/1427905762 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe89959c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58912/0 (socket says 172.21.15.5:58912) 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe898d9b700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 0x7fe894198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.173+0000 7fe893fff700 1 -- 172.21.15.5:0/1427905762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 msgr2=0x7fe8941a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe893fff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe893fff700 1 -- 172.21.15.5:0/1427905762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 msgr2=0x7fe894198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe893fff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 0x7fe894198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe893fff700 1 -- 172.21.15.5:0/1427905762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880009520 con 0x7fe894102070 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe89959c700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe898d9b700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 0x7fe894198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe893fff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe880000c00 tx=0x7fe88000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88001c070 con 0x7fe894102070 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88000f7d0 con 0x7fe894102070 2023-09-23T04:43:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8941a0ed0 con 0x7fe894102070 2023-09-23T04:43:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800039b0 con 0x7fe894102070 2023-09-23T04:43:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.174+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8941a1240 con 0x7fe894102070 2023-09-23T04:43:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.175+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89410c700 con 0x7fe894102070 2023-09-23T04:43:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.177+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fe880020030 con 0x7fe894102070 2023-09-23T04:43:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.179+0000 7fe891ffb700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 0x7fe8840a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.179+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8800cdb20 con 0x7fe894102070 2023-09-23T04:43:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.179+0000 7fe898d9b700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 0x7fe8840a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.179+0000 7fe898d9b700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 0x7fe8840a3a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe894101be0 tx=0x7fe888009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.183+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe8800975a0 con 0x7fe894102070 2023-09-23T04:43:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.395+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8941a1430 con 0x7fe894102070 2023-09-23T04:43:11.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.396+0000 7fe891ffb700 1 -- 172.21.15.5:0/1427905762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18328 (secure 0 0 0) 0x7fe88009abc0 con 0x7fe894102070 2023-09-23T04:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":79,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:00.465274+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:11.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 msgr2=0x7fe8840a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 0x7fe8840a3a60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe894101be0 tx=0x7fe888009310 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 msgr2=0x7fe894199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe880000c00 tx=0x7fe88000b790 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 shutdown_connections 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe8840a15a0 0x7fe8840a3a60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe894102a30 0x7fe8941a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe894100e90 0x7fe894198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 --2- 172.21.15.5:0/1427905762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe894102070 0x7fe894199340 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.398+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 >> 172.21.15.5:0/1427905762 conn(0x7fe8940fc450 msgr2=0x7fe8940fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:11.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.399+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 shutdown_connections 2023-09-23T04:43:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.399+0000 7fe89afff700 1 -- 172.21.15.5:0/1427905762 wait complete. 2023-09-23T04:43:11.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T04:43:11.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 -- 172.21.15.5:0/3077098829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304073880 msgr2=0x7f0304073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3077098829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304073880 0x7f0304073d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009990 tx=0x7f02f4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 -- 172.21.15.5:0/3077098829 shutdown_connections 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3077098829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f030410b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3077098829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304074e60 0x7f03040732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3077098829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304073880 0x7f0304073d00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 -- 172.21.15.5:0/3077098829 >> 172.21.15.5:0/3077098829 conn(0x7f03040fc480 msgr2=0x7f03040fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 -- 172.21.15.5:0/3077098829 shutdown_connections 2023-09-23T04:43:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.625+0000 7f030b5ff700 1 -- 172.21.15.5:0/3077098829 wait complete. 2023-09-23T04:43:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.626+0000 7f030b5ff700 1 Processor -- start 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.626+0000 7f030b5ff700 1 -- start start 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304074e60 0x7f0304199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f03041a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041a09a0 con 0x7f0304074e60 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041a0ae0 con 0x7f0304108dd0 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030b5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041a0c70 con 0x7f0304073880 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58942/0 (socket says 172.21.15.5:58942) 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 -- 172.21.15.5:0/3342453572 learned_addr learned my addr 172.21.15.5:0/3342453572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f0308b9a700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304074e60 0x7f0304199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f0309b9c700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f03041a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 -- 172.21.15.5:0/3342453572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 msgr2=0x7f03041a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f03041a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 -- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304074e60 msgr2=0x7f0304199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304074e60 0x7f0304199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f030939b700 1 -- 172.21.15.5:0/3342453572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40095f0 con 0x7f0304073880 2023-09-23T04:43:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.627+0000 7f0309b9c700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f03041a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f030939b700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9a0 tx=0x7f030000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300009790 con 0x7f0304073880 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f030000cbd0 con 0x7f0304073880 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03041a0f50 con 0x7f0304073880 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300018680 con 0x7f0304073880 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.628+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03041a1370 con 0x7f0304073880 2023-09-23T04:43:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.629+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0304107480 con 0x7f0304073880 2023-09-23T04:43:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.631+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f030000cd40 con 0x7f0304073880 2023-09-23T04:43:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.633+0000 7f02fa7fc700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 0x7f02f00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.633+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0300014070 con 0x7f0304073880 2023-09-23T04:43:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.633+0000 7f0308b9a700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 0x7f02f00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.633+0000 7f0308b9a700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 0x7f02f00a3ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02f400b2e0 tx=0x7f02f4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.636+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0300096810 con 0x7f0304073880 2023-09-23T04:43:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.850+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f030410ec00 con 0x7f0304073880 2023-09-23T04:43:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.851+0000 7f02fa7fc700 1 -- 172.21.15.5:0/3342453572 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0300099e30 con 0x7f0304073880 2023-09-23T04:43:11.852 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:43:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 msgr2=0x7f02f00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 0x7f02f00a3ae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02f400b2e0 tx=0x7f02f4005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 msgr2=0x7f0304198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9a0 tx=0x7f030000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 shutdown_connections 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f02f00a1620 0x7f02f00a3ae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0304108dd0 0x7f03041a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0304073880 0x7f0304198e20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 --2- 172.21.15.5:0/3342453572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0304074e60 0x7f0304199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.853+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 >> 172.21.15.5:0/3342453572 conn(0x7f03040fc480 msgr2=0x7f03040fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.854+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 shutdown_connections 2023-09-23T04:43:11.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:11.854+0000 7f030b5ff700 1 -- 172.21.15.5:0/3342453572 wait complete. 2023-09-23T04:43:11.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:11 smithi118 ceph-mon[134297]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:43:11.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:11 smithi118 ceph-mon[134297]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l;count:12 2023-09-23T04:43:11.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:11 smithi118 ceph-mon[134297]: Deploying daemon mds.l on smithi118 2023-09-23T04:43:11.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:11 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1427905762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:11.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:11 smithi069 ceph-mon[133931]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:43:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:11 smithi069 ceph-mon[133931]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l;count:12 2023-09-23T04:43:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:11 smithi069 ceph-mon[133931]: Deploying daemon mds.l on smithi118 2023-09-23T04:43:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:11 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1427905762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 -- 172.21.15.5:0/91219445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 msgr2=0x7fc66c109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 --2- 172.21.15.5:0/91219445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c109050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc66400b0a0 tx=0x7fc66400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 -- 172.21.15.5:0/91219445 shutdown_connections 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 --2- 172.21.15.5:0/91219445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c109050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 --2- 172.21.15.5:0/91219445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c101980 0x7fc66c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 --2- 172.21.15.5:0/91219445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c1042b0 0x7fc66c1066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 -- 172.21.15.5:0/91219445 >> 172.21.15.5:0/91219445 conn(0x7fc66c0fb350 msgr2=0x7fc66c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 -- 172.21.15.5:0/91219445 shutdown_connections 2023-09-23T04:43:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.078+0000 7fc670fff700 1 -- 172.21.15.5:0/91219445 wait complete. 2023-09-23T04:43:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.079+0000 7fc670fff700 1 Processor -- start 2023-09-23T04:43:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.079+0000 7fc670fff700 1 -- start start 2023-09-23T04:43:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c101980 0x7fc66c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c1042b0 0x7fc66c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1a0a40 con 0x7fc66c106be0 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1a0b80 con 0x7fc66c101980 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc670fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1a0d10 con 0x7fc66c1042b0 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45270/0 (socket says 172.21.15.5:45270) 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 -- 172.21.15.5:0/658865005 learned_addr learned my addr 172.21.15.5:0/658865005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc669d9b700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c1042b0 0x7fc66c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66a59c700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c101980 0x7fc66c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 -- 172.21.15.5:0/658865005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c1042b0 msgr2=0x7fc66c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c1042b0 0x7fc66c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 -- 172.21.15.5:0/658865005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c101980 msgr2=0x7fc66c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c101980 0x7fc66c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.080+0000 7fc66ad9d700 1 -- 172.21.15.5:0/658865005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009520 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc66ad9d700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc664009fd0 tx=0x7fc66400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66400e040 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6640094a0 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc664009d00 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66401b6c0 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.081+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c1a1220 con 0x7fc66c106be0 2023-09-23T04:43:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.082+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c108760 con 0x7fc66c106be0 2023-09-23T04:43:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.084+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc66401d470 con 0x7fc66c106be0 2023-09-23T04:43:12.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.085+0000 7fc65b7fe700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 0x7fc6540a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:12.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.085+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6640cd9f0 con 0x7fc66c106be0 2023-09-23T04:43:12.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.086+0000 7fc66a59c700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 0x7fc6540a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.086+0000 7fc66a59c700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 0x7fc6540a3ae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009df0 tx=0x7fc65c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.089+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc664097470 con 0x7fc66c106be0 2023-09-23T04:43:12.149 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:11 smithi118 systemd[1]: Starting Ceph mds.l for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:11 smithi005 ceph-mon[97638]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:43:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:11 smithi005 ceph-mon[97638]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l;count:12 2023-09-23T04:43:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:11 smithi005 ceph-mon[97638]: Deploying daemon mds.l on smithi118 2023-09-23T04:43:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:11 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1427905762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.346+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc66c10c730 con 0x7fc66c106be0 2023-09-23T04:43:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.347+0000 7fc65b7fe700 1 -- 172.21.15.5:0/658865005 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fc66409aa90 con 0x7fc66c106be0 2023-09-23T04:43:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:12.348 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-23T04:43:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 msgr2=0x7fc6540a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 0x7fc6540a3ae0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009df0 tx=0x7fc65c009260 comp rx=0 tx=0).stop 2023-09-23T04:43:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 msgr2=0x7fc66c1a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc664009fd0 tx=0x7fc66400bce0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 shutdown_connections 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6540a1620 0x7fc6540a3ae0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc66c106be0 0x7fc66c1a0470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc66c101980 0x7fc66c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 --2- 172.21.15.5:0/658865005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc66c1042b0 0x7fc66c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 >> 172.21.15.5:0/658865005 conn(0x7fc66c0fb350 msgr2=0x7fc66c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 shutdown_connections 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:12.349+0000 7fc670fff700 1 -- 172.21.15.5:0/658865005 wait complete. 2023-09-23T04:43:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2023-09-23T04:43:12.410 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:12 smithi118 podman[171024]: 2023-09-23T04:43:12.410 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:12 smithi118 bash[171024]: 7aea8930cf85bd2fe95d708bd29b5b36ed5044db4ad870ac222b6acc49374cad 2023-09-23T04:43:12.410 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mds[171064]: -- 172.21.15.118:0/505265163 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ab818ed1e0 con 0x55ab818c2800 2023-09-23T04:43:12.411 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: starting mds.l at 2023-09-23T04:43:12.910 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 04:43:12 smithi118 systemd[1]: Started Ceph mds.l for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3342453572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/658865005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6832/3707604246,v1:172.21.15.118:6833/3707604246] up:boot 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: fsmap 1 up:standby 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:12.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:12.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:12 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3342453572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/658865005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6832/3707604246,v1:172.21.15.118:6833/3707604246] up:boot 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: fsmap 1 up:standby 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:43:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:12.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:12.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:12.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:12 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3342453572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/658865005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6832/3707604246,v1:172.21.15.118:6833/3707604246] up:boot 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: fsmap 1 up:standby 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:13.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:13.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:12 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:13.732 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:13 smithi069 ceph-mon[133931]: Deploying daemon mds.k on smithi069 2023-09-23T04:43:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:13 smithi005 ceph-mon[97638]: Deploying daemon mds.k on smithi069 2023-09-23T04:43:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:13 smithi118 ceph-mon[134297]: Deploying daemon mds.k on smithi069 2023-09-23T04:43:14.294 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 systemd[1]: Starting Ceph mds.k for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:14.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.582+0000 7fa044bff700 1 -- 172.21.15.5:0/896839653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102080 msgr2=0x7fa040102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.582+0000 7fa044bff700 1 --2- 172.21.15.5:0/896839653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102080 0x7fa040102500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0340098f0 tx=0x7fa034009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 -- 172.21.15.5:0/896839653 shutdown_connections 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 --2- 172.21.15.5:0/896839653 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040102a40 0x7fa040109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 --2- 172.21.15.5:0/896839653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040100ea0 0x7fa0401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 --2- 172.21.15.5:0/896839653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102080 0x7fa040102500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 -- 172.21.15.5:0/896839653 >> 172.21.15.5:0/896839653 conn(0x7fa0400fc480 msgr2=0x7fa0400fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 -- 172.21.15.5:0/896839653 shutdown_connections 2023-09-23T04:43:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.583+0000 7fa044bff700 1 -- 172.21.15.5:0/896839653 wait complete. 2023-09-23T04:43:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.584+0000 7fa044bff700 1 Processor -- start 2023-09-23T04:43:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.584+0000 7fa044bff700 1 -- start start 2023-09-23T04:43:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040100ea0 0x7fa040196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 0x7fa04019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019e7e0 con 0x7fa040102080 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019e950 con 0x7fa040102a40 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa044bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04019eb10 con 0x7fa040100ea0 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa03dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa03dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45284/0 (socket says 172.21.15.5:45284) 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa03dd9b700 1 -- 172.21.15.5:0/3544826038 learned_addr learned my addr 172.21.15.5:0/3544826038 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa03ed9d700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 0x7fa04019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.585+0000 7fa03e59c700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040100ea0 0x7fa040196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 -- 172.21.15.5:0/3544826038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040100ea0 msgr2=0x7fa040196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040100ea0 0x7fa040196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 -- 172.21.15.5:0/3544826038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 msgr2=0x7fa04019e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 0x7fa04019e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 -- 172.21.15.5:0/3544826038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa034009520 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03ed9d700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 0x7fa04019e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa03dd9b700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa034009fd0 tx=0x7fa03400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03401c070 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04019ed90 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa03400f7d0 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034003a20 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.586+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04019f1c0 con 0x7fa040102080 2023-09-23T04:43:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.587+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04010c760 con 0x7fa040102080 2023-09-23T04:43:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.589+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa034020030 con 0x7fa040102080 2023-09-23T04:43:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.591+0000 7fa02f7fe700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 0x7fa0280a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.591+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0340cda10 con 0x7fa040102080 2023-09-23T04:43:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.591+0000 7fa03e59c700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 0x7fa0280a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.592+0000 7fa03e59c700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 0x7fa0280a3a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa040197b40 tx=0x7fa030009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.595+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa034097490 con 0x7fa040102080 2023-09-23T04:43:14.621 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 podman[169525]: 2023-09-23T04:43:14.621 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 ceph-mds[169565]: -- 172.21.15.69:0/3231898331 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561598fdf1e0 con 0x561598fb4c00 2023-09-23T04:43:14.621 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 bash[169525]: 82e27fdc92c4a3f12b943953513e861c1d18ad0af478dc5e967d81c17ae7f789 2023-09-23T04:43:14.622 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: starting mds.k at 2023-09-23T04:43:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.871+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa040107ef0 con 0x7fa040102080 2023-09-23T04:43:14.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.871+0000 7fa02f7fe700 1 -- 172.21.15.5:0/3544826038 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7fa03409aab0 con 0x7fa040102080 2023-09-23T04:43:14.872 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:14.872 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-23T04:43:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.873+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 msgr2=0x7fa0280a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.873+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 0x7fa0280a3a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa040197b40 tx=0x7fa030009310 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.873+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 msgr2=0x7fa0401971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.873+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa034009fd0 tx=0x7fa03400b790 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 shutdown_connections 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa0280a15a0 0x7fa0280a3a60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa040102a40 0x7fa04019e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa040100ea0 0x7fa040196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 --2- 172.21.15.5:0/3544826038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa040102080 0x7fa0401971b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 >> 172.21.15.5:0/3544826038 conn(0x7fa0400fc480 msgr2=0x7fa0400fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 shutdown_connections 2023-09-23T04:43:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:14.874+0000 7fa044bff700 1 -- 172.21.15.5:0/3544826038 wait complete. 2023-09-23T04:43:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-09-23T04:43:14.991 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 04:43:14 smithi069 systemd[1]: Started Ceph mds.k for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:14 smithi069 ceph-mon[133931]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:14 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:14 smithi005 ceph-mon[97638]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:14 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:14 smithi118 ceph-mon[134297]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:14 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6832/827181928,v1:172.21.15.69:6833/827181928] up:boot 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: fsmap 2 up:standby 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: Deploying daemon mds.j on smithi005 2023-09-23T04:43:15.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3544826038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6832/827181928,v1:172.21.15.69:6833/827181928] up:boot 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: fsmap 2 up:standby 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: Deploying daemon mds.j on smithi005 2023-09-23T04:43:15.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:15 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3544826038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6832/827181928,v1:172.21.15.69:6833/827181928] up:boot 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: fsmap 2 up:standby 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: Deploying daemon mds.j on smithi005 2023-09-23T04:43:16.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:15 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3544826038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:16.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:16.927 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:16 smithi005 systemd[1]: Starting Ceph mds.j for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:16.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:16 smithi005 ceph-mon[97638]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:16 smithi069 ceph-mon[133931]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 -- 172.21.15.5:0/746580246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 msgr2=0x7f322010b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 --2- 172.21.15.5:0/746580246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f322010b240 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3218009f90 tx=0x7f321800b330 comp rx=0 tx=0).stop 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 -- 172.21.15.5:0/746580246 shutdown_connections 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 --2- 172.21.15.5:0/746580246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f322010b240 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 --2- 172.21.15.5:0/746580246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220074e60 0x7f32200732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 --2- 172.21.15.5:0/746580246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220073880 0x7f3220073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 -- 172.21.15.5:0/746580246 >> 172.21.15.5:0/746580246 conn(0x7f32200fc480 msgr2=0x7f32200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 -- 172.21.15.5:0/746580246 shutdown_connections 2023-09-23T04:43:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.131+0000 7f3227bff700 1 -- 172.21.15.5:0/746580246 wait complete. 2023-09-23T04:43:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 Processor -- start 2023-09-23T04:43:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 -- start start 2023-09-23T04:43:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220073880 0x7f3220198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f32201a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201a0a60 con 0x7f3220108db0 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201a0ba0 con 0x7f3220074e60 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.133+0000 7f3227bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201a0d30 con 0x7f3220073880 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322619c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f32201a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40374/0 (socket says 172.21.15.5:40374) 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322599b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220073880 0x7f3220198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 -- 172.21.15.5:0/3746935179 learned_addr learned my addr 172.21.15.5:0/3746935179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 -- 172.21.15.5:0/3746935179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220073880 msgr2=0x7f3220198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220073880 0x7f3220198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 -- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 msgr2=0x7f32201a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f32201a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 -- 172.21.15.5:0/3746935179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100095f0 con 0x7f3220074e60 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322619c700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f32201a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f322519a700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f321000ebf0 tx=0x7f321000ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.134+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210009a10 con 0x7f3220074e60 2023-09-23T04:43:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.135+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321000cc90 con 0x7f3220074e60 2023-09-23T04:43:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.135+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3218009c40 con 0x7f3220074e60 2023-09-23T04:43:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.135+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210018750 con 0x7f3220074e60 2023-09-23T04:43:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.135+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32201a1350 con 0x7f3220074e60 2023-09-23T04:43:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.136+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220107460 con 0x7f3220074e60 2023-09-23T04:43:17.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.138+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f32100188b0 con 0x7f3220074e60 2023-09-23T04:43:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.140+0000 7f3216ffd700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 0x7f320c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.140+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3210014070 con 0x7f3220074e60 2023-09-23T04:43:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.140+0000 7f322599b700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 0x7f320c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:17.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.141+0000 7f322599b700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 0x7f320c0a3ae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f321c009c60 tx=0x7f321c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:17.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.142+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3210096be0 con 0x7f3220074e60 2023-09-23T04:43:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:16 smithi118 ceph-mon[134297]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.411+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f322010eb80 con 0x7f3220074e60 2023-09-23T04:43:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.411+0000 7f3216ffd700 1 -- 172.21.15.5:0/3746935179 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f321009a200 con 0x7f3220074e60 2023-09-23T04:43:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:17.412 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-23T04:43:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 msgr2=0x7f320c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 0x7f320c0a3ae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f321c009c60 tx=0x7f321c009260 comp rx=0 tx=0).stop 2023-09-23T04:43:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 msgr2=0x7f3220199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f321000ebf0 tx=0x7f321000ef00 comp rx=0 tx=0).stop 2023-09-23T04:43:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 shutdown_connections 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f320c0a1620 0x7f320c0a3ae0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3220108db0 0x7f32201a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3220073880 0x7f3220198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 --2- 172.21.15.5:0/3746935179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3220074e60 0x7f3220199420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 >> 172.21.15.5:0/3746935179 conn(0x7f32200fc480 msgr2=0x7f32200fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 shutdown_connections 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:17.414+0000 7f3227bff700 1 -- 172.21.15.5:0/3746935179 wait complete. 2023-09-23T04:43:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-09-23T04:43:17.711 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:17 smithi005 podman[148145]: 2023-09-23T04:43:17.711 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:17 smithi005 bash[148145]: c387557dd5396a8a48216d8399952a8097b702846cc4b5dfbe283a5b8be157d9 2023-09-23T04:43:17.711 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:17 smithi005 ceph-mds[148187]: -- 172.21.15.5:0/382348437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae2705d1e0 con 0x55ae27033000 2023-09-23T04:43:17.711 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:17 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-j[148163]: starting mds.j at 2023-09-23T04:43:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:17 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3746935179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:18.154 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 04:43:17 smithi005 systemd[1]: Started Ceph mds.j for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:17 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3746935179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:17 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3746935179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6834/3475689025,v1:172.21.15.5:6835/3475689025] up:boot 2023-09-23T04:43:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: fsmap 3 up:standby 2023-09-23T04:43:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:18.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:18 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:19.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6834/3475689025,v1:172.21.15.5:6835/3475689025] up:boot 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: fsmap 3 up:standby 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:19.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:18 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6834/3475689025,v1:172.21.15.5:6835/3475689025] up:boot 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: fsmap 3 up:standby 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:19.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:19.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:19.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:18 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:19.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 -- 172.21.15.5:0/577790005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 msgr2=0x7f48e81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 --2- 172.21.15.5:0/577790005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e81024f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009990 tx=0x7f48d8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 -- 172.21.15.5:0/577790005 shutdown_connections 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 --2- 172.21.15.5:0/577790005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 --2- 172.21.15.5:0/577790005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 --2- 172.21.15.5:0/577790005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e81024f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 -- 172.21.15.5:0/577790005 >> 172.21.15.5:0/577790005 conn(0x7f48e80fc450 msgr2=0x7f48e80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:19.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 -- 172.21.15.5:0/577790005 shutdown_connections 2023-09-23T04:43:19.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.680+0000 7f48effff700 1 -- 172.21.15.5:0/577790005 wait complete. 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 Processor -- start 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 -- start start 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819e770 con 0x7f48e8102070 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819e8b0 con 0x7f48e8100e90 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48effff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819ea40 con 0x7f48e8102a30 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48ed59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:19.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48ed59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45324/0 (socket says 172.21.15.5:45324) 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48ed59a700 1 -- 172.21.15.5:0/286666599 learned_addr learned my addr 172.21.15.5:0/286666599 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.682+0000 7f48edd9b700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ee59c700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 -- 172.21.15.5:0/286666599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 msgr2=0x7f48e819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 -- 172.21.15.5:0/286666599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 msgr2=0x7f48e8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 -- 172.21.15.5:0/286666599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d80095f0 con 0x7f48e8102070 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48edd9b700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ee59c700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.683+0000 7f48ed59a700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48d8005720 tx=0x7f48d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.684+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d801d070 con 0x7f48e8102070 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.684+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e819ecc0 con 0x7f48e8102070 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.684+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d8021410 con 0x7f48e8102070 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.684+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8017650 con 0x7f48e8102070 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.684+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e819f0e0 con 0x7f48e8102070 2023-09-23T04:43:19.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.685+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e8107e90 con 0x7f48e8102070 2023-09-23T04:43:19.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.687+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f48d8021580 con 0x7f48e8102070 2023-09-23T04:43:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.688+0000 7f48deffd700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.689+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48d80ce100 con 0x7f48e8102070 2023-09-23T04:43:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.689+0000 7f48edd9b700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.689+0000 7f48edd9b700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 0x7f48d40a3a60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f48e8101be0 tx=0x7f48e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:19.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.691+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f48d8097b80 con 0x7f48e8102070 2023-09-23T04:43:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.983+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e810c9c0 con 0x7f48e8102070 2023-09-23T04:43:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.984+0000 7f48deffd700 1 -- 172.21.15.5:0/286666599 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2904 (secure 0 0 0) 0x7f48d809b1a0 con 0x7f48e8102070 2023-09-23T04:43:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-23T04:43:19.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 msgr2=0x7f48d40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:19.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 0x7f48d40a3a60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f48e8101be0 tx=0x7f48e4009260 comp rx=0 tx=0).stop 2023-09-23T04:43:19.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 msgr2=0x7f48e8197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:19.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48d8005720 tx=0x7f48d8009580 comp rx=0 tx=0).stop 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 shutdown_connections 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48d40a15a0 0x7f48d40a3a60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e8102a30 0x7f48e819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e8100e90 0x7f48e8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 --2- 172.21.15.5:0/286666599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e8102070 0x7f48e8197130 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.986+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 >> 172.21.15.5:0/286666599 conn(0x7f48e80fc450 msgr2=0x7f48e80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.987+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 shutdown_connections 2023-09-23T04:43:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:19.987+0000 7f48effff700 1 -- 172.21.15.5:0/286666599 wait complete. 2023-09-23T04:43:19.989 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2023-09-23T04:43:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:19 smithi069 ceph-mon[133931]: Deploying daemon mds.d on smithi005 2023-09-23T04:43:20.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:19 smithi005 ceph-mon[97638]: Deploying daemon mds.d on smithi005 2023-09-23T04:43:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:19 smithi118 ceph-mon[134297]: Deploying daemon mds.d on smithi005 2023-09-23T04:43:20.405 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 systemd[1]: Starting Ceph mds.d for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:20.823 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 podman[148537]: 2023-09-23T04:43:20.823 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:20 smithi005 ceph-mon[97638]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:20.823 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/286666599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:20 smithi069 ceph-mon[133931]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/286666599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:21.154 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 bash[148537]: 7ab79e94071c3e1faffafc21a4e4a89b96a1ed75d00b50f3614fea4b49be9905 2023-09-23T04:43:21.155 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 ceph-mds[148577]: -- 172.21.15.5:0/3017007151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55773a1f51e0 con 0x55773a1cac00 2023-09-23T04:43:21.155 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-d[148553]: starting mds.d at 2023-09-23T04:43:21.155 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 04:43:20 smithi005 systemd[1]: Started Ceph mds.d for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:20 smithi118 ceph-mon[134297]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/286666599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:22.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: Deploying daemon mds.e on smithi069 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:22.196 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6836/3951415099,v1:172.21.15.5:6837/3951415099] up:boot 2023-09-23T04:43:22.197 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: fsmap 4 up:standby 2023-09-23T04:43:22.197 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:21 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:43:22.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 -- 172.21.15.5:0/2276570804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 msgr2=0x7f45e81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:22.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 --2- 172.21.15.5:0/2276570804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e81024f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45d00098f0 tx=0x7f45d0009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:22.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 -- 172.21.15.5:0/2276570804 shutdown_connections 2023-09-23T04:43:22.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 --2- 172.21.15.5:0/2276570804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45e8102a30 0x7f45e8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 --2- 172.21.15.5:0/2276570804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 0x7f45e81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 --2- 172.21.15.5:0/2276570804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e81024f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 -- 172.21.15.5:0/2276570804 >> 172.21.15.5:0/2276570804 conn(0x7f45e80fc450 msgr2=0x7f45e80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:22.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 -- 172.21.15.5:0/2276570804 shutdown_connections 2023-09-23T04:43:22.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.226+0000 7f45edfff700 1 -- 172.21.15.5:0/2276570804 wait complete. 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.227+0000 7f45edfff700 1 Processor -- start 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.227+0000 7f45edfff700 1 -- start start 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 0x7f45e8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45e8102a30 0x7f45e81a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0a40 con 0x7f45e8102070 2023-09-23T04:43:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0b80 con 0x7f45e8102a30 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45edfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a0d10 con 0x7f45e8100e90 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45e77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 0x7f45e8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 -- 172.21.15.5:0/343256691 learned_addr learned my addr 172.21.15.5:0/343256691 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 -- 172.21.15.5:0/343256691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 msgr2=0x7f45e8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 0x7f45e8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 -- 172.21.15.5:0/343256691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45e8102a30 msgr2=0x7f45e81a0470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45e8102a30 0x7f45e81a0470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.228+0000 7f45dffff700 1 -- 172.21.15.5:0/343256691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0009520 con 0x7f45e8102070 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45dffff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009fd0 tx=0x7f45d000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:22.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d001c070 con 0x7f45e8102070 2023-09-23T04:43:22.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d000f7d0 con 0x7f45e8102070 2023-09-23T04:43:22.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e81a0f90 con 0x7f45e8102070 2023-09-23T04:43:22.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003ac0 con 0x7f45e8102070 2023-09-23T04:43:22.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.229+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81a1350 con 0x7f45e8102070 2023-09-23T04:43:22.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.230+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e810c750 con 0x7f45e8102070 2023-09-23T04:43:22.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.232+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f45d0020030 con 0x7f45e8102070 2023-09-23T04:43:22.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.234+0000 7f45e57fa700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 0x7f45d40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:22.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.235+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f45d00cda30 con 0x7f45e8102070 2023-09-23T04:43:22.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.235+0000 7f45e77fe700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 0x7f45d40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:22.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.235+0000 7f45e77fe700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 0x7f45d40a3ae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45e8101be0 tx=0x7f45d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.239+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f45d00974b0 con 0x7f45e8102070 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: Deploying daemon mds.e on smithi069 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6836/3951415099,v1:172.21.15.5:6837/3951415099] up:boot 2023-09-23T04:43:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: fsmap 4 up:standby 2023-09-23T04:43:22.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:21 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:43:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: Deploying daemon mds.e on smithi069 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6836/3951415099,v1:172.21.15.5:6837/3951415099] up:boot 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: fsmap 4 up:standby 2023-09-23T04:43:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:21 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:43:22.491 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:22 smithi069 systemd[1]: Starting Ceph mds.e for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.497+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e81a14e0 con 0x7f45e8102070 2023-09-23T04:43:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.498+0000 7f45e57fa700 1 -- 172.21.15.5:0/343256691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7f45d009aad0 con 0x7f45e8102070 2023-09-23T04:43:22.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:22.499 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-23T04:43:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 msgr2=0x7f45d40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 0x7f45d40a3ae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45e8101be0 tx=0x7f45d8009260 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 msgr2=0x7f45e8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009fd0 tx=0x7f45d000b790 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 shutdown_connections 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f45d40a1620 0x7f45d40a3ae0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45e8102a30 0x7f45e81a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f45e8100e90 0x7f45e8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 --2- 172.21.15.5:0/343256691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8102070 0x7f45e8199400 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.500+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 >> 172.21.15.5:0/343256691 conn(0x7f45e80fc450 msgr2=0x7f45e80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.501+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 shutdown_connections 2023-09-23T04:43:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:22.501+0000 7f45edfff700 1 -- 172.21.15.5:0/343256691 wait complete. 2023-09-23T04:43:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2023-09-23T04:43:22.972 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:22 smithi069 podman[169866]: 2023-09-23T04:43:22.973 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:22 smithi069 bash[169866]: 47d826c520c6188b1946a3b9c31f71163c1d4363531c23ec3cd626099535d17d 2023-09-23T04:43:22.973 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:22 smithi069 ceph-mds[169906]: -- 172.21.15.69:0/1022973163 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ba8e6f1e0 con 0x562ba8e45000 2023-09-23T04:43:22.973 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: starting mds.e at 2023-09-23T04:43:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/343256691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:23.241 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 04:43:23 smithi069 systemd[1]: Started Ceph mds.e for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:23.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/343256691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:23.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/343256691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: Deploying daemon mds.f on smithi118 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:24.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6834/3966429650,v1:172.21.15.69:6835/3966429650] up:boot 2023-09-23T04:43:24.261 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: fsmap 5 up:standby 2023-09-23T04:43:24.261 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:43:24.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: Deploying daemon mds.f on smithi118 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6834/3966429650,v1:172.21.15.69:6835/3966429650] up:boot 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: fsmap 5 up:standby 2023-09-23T04:43:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: Deploying daemon mds.f on smithi118 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6834/3966429650,v1:172.21.15.69:6835/3966429650] up:boot 2023-09-23T04:43:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: fsmap 5 up:standby 2023-09-23T04:43:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:43:24.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:24.661 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:24 smithi118 systemd[1]: Starting Ceph mds.f for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 -- 172.21.15.5:0/3996613826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0100ea0 msgr2=0x7f96c01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 --2- 172.21.15.5:0/3996613826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0100ea0 0x7f96c01012c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009990 tx=0x7f96b0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 -- 172.21.15.5:0/3996613826 shutdown_connections 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 --2- 172.21.15.5:0/3996613826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0102a40 0x7f96c0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 --2- 172.21.15.5:0/3996613826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0100ea0 0x7f96c01012c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 --2- 172.21.15.5:0/3996613826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102080 0x7f96c0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 -- 172.21.15.5:0/3996613826 >> 172.21.15.5:0/3996613826 conn(0x7f96c00fc480 msgr2=0x7f96c00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 -- 172.21.15.5:0/3996613826 shutdown_connections 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.734+0000 7f96c49ff700 1 -- 172.21.15.5:0/3996613826 wait complete. 2023-09-23T04:43:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.736+0000 7f96c49ff700 1 Processor -- start 2023-09-23T04:43:24.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.736+0000 7f96c49ff700 1 -- start start 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.736+0000 7f96c49ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 0x7f96c0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96c49ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96c49ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102a40 0x7f96c0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96c49ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0073ef0 con 0x7f96c0102080 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96c49ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0074060 con 0x7f96c0100ea0 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96c49ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01a0cb0 con 0x7f96c0102a40 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35688/0 (socket says 172.21.15.5:35688) 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 -- 172.21.15.5:0/799284654 learned_addr learned my addr 172.21.15.5:0/799284654 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96be59c700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 0x7f96c0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bed9d700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102a40 0x7f96c0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 -- 172.21.15.5:0/799284654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102a40 msgr2=0x7f96c0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102a40 0x7f96c0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 -- 172.21.15.5:0/799284654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 msgr2=0x7f96c0074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 0x7f96c0074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96bdd9b700 1 -- 172.21.15.5:0/799284654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00095f0 con 0x7f96c0102080 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.737+0000 7f96be59c700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 0x7f96c0074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96bdd9b700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e970 tx=0x7f96b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4009790 con 0x7f96c0102080 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c01a0f90 con 0x7f96c0102080 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b400c7e0 con 0x7f96c0102080 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4003c70 con 0x7f96c0102080 2023-09-23T04:43:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.738+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c01a1410 con 0x7f96c0102080 2023-09-23T04:43:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.739+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c010c760 con 0x7f96c0102080 2023-09-23T04:43:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.741+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f96b400cc50 con 0x7f96c0102080 2023-09-23T04:43:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.742+0000 7f96af7fe700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 0x7f96a80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.742+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96b4014070 con 0x7f96c0102080 2023-09-23T04:43:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.743+0000 7f96be59c700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 0x7f96a80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.743+0000 7f96be59c700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 0x7f96a80a3a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b2e0 tx=0x7f96b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:24.745+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f96b40967f0 con 0x7f96c0102080 2023-09-23T04:43:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.006+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96c0107ea0 con 0x7f96c0102080 2023-09-23T04:43:25.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.007+0000 7f96af7fe700 1 -- 172.21.15.5:0/799284654 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f96b4003dd0 con 0x7f96c0102080 2023-09-23T04:43:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:25.008 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 msgr2=0x7f96a80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 0x7f96a80a3a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b2e0 tx=0x7f96b0005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 msgr2=0x7f96c0073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96b400e970 tx=0x7f96b400ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 shutdown_connections 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f96a80a15a0 0x7f96a80a3a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96c0102a40 0x7f96c0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96c0100ea0 0x7f96c0074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 --2- 172.21.15.5:0/799284654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96c0102080 0x7f96c0073210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 >> 172.21.15.5:0/799284654 conn(0x7f96c00fc480 msgr2=0x7f96c00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 shutdown_connections 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:25.009+0000 7f96c49ff700 1 -- 172.21.15.5:0/799284654 wait complete. 2023-09-23T04:43:25.011 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2023-09-23T04:43:25.047 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:24 smithi118 podman[171373]: 2023-09-23T04:43:25.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:25 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/799284654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:25.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:25 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/799284654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:25.411 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:25 smithi118 ceph-mds[171413]: -- 172.21.15.118:0/2315424292 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5630042611e0 con 0x563004236800 2023-09-23T04:43:25.411 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: starting mds.f at 2023-09-23T04:43:25.411 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:25 smithi118 bash[171373]: 9dc1fcc9abff0c3060c509aa574844af8c8263e0f80f6b546ad441e4f1ccc2e8 2023-09-23T04:43:25.411 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 04:43:25 smithi118 systemd[1]: Started Ceph mds.f for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:25.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:25 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/799284654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:boot 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: fsmap 6 up:standby 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: Deploying daemon mds.c on smithi118 2023-09-23T04:43:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:26 smithi005 ceph-mon[97638]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:boot 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: fsmap 6 up:standby 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: Deploying daemon mds.c on smithi118 2023-09-23T04:43:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:26 smithi118 ceph-mon[134297]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:boot 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: fsmap 6 up:standby 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: Deploying daemon mds.c on smithi118 2023-09-23T04:43:26.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:26 smithi069 ceph-mon[133931]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:26.911 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:26 smithi118 systemd[1]: Starting Ceph mds.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:27.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 -- 172.21.15.5:0/1820612085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690073880 msgr2=0x7ff690073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 --2- 172.21.15.5:0/1820612085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690073880 0x7ff690073d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff6800098f0 tx=0x7ff680009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 -- 172.21.15.5:0/1820612085 shutdown_connections 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 --2- 172.21.15.5:0/1820612085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690108db0 0x7ff69010b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 --2- 172.21.15.5:0/1820612085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff6900732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 --2- 172.21.15.5:0/1820612085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690073880 0x7ff690073d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 -- 172.21.15.5:0/1820612085 >> 172.21.15.5:0/1820612085 conn(0x7ff6900fc480 msgr2=0x7ff6900fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:27.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 -- 172.21.15.5:0/1820612085 shutdown_connections 2023-09-23T04:43:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.243+0000 7ff697fff700 1 -- 172.21.15.5:0/1820612085 wait complete. 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 Processor -- start 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 -- start start 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690073880 0x7ff690198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690108db0 0x7ff6901a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6901a09d0 con 0x7ff690074e60 2023-09-23T04:43:27.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6901a0b10 con 0x7ff690073880 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.244+0000 7ff697fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6901a0ca0 con 0x7ff690108db0 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690108db0 0x7ff6901a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35712/0 (socket says 172.21.15.5:35712) 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 -- 172.21.15.5:0/1844956749 learned_addr learned my addr 172.21.15.5:0/1844956749 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff695d9b700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690073880 0x7ff690198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 -- 172.21.15.5:0/1844956749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690108db0 msgr2=0x7ff6901a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690108db0 0x7ff6901a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 -- 172.21.15.5:0/1844956749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690073880 msgr2=0x7ff690198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690073880 0x7ff690198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 -- 172.21.15.5:0/1844956749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0095f0 con 0x7ff690074e60 2023-09-23T04:43:27.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.245+0000 7ff69559a700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff680009fd0 tx=0x7ff68000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.246+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68001c070 con 0x7ff690074e60 2023-09-23T04:43:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.246+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68000f7d0 con 0x7ff690074e60 2023-09-23T04:43:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.246+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff680009520 con 0x7ff690074e60 2023-09-23T04:43:27.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.246+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680003a40 con 0x7ff690074e60 2023-09-23T04:43:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.246+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6901a11b0 con 0x7ff690074e60 2023-09-23T04:43:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.247+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff690107460 con 0x7ff690074e60 2023-09-23T04:43:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.249+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7ff680020030 con 0x7ff690074e60 2023-09-23T04:43:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.250+0000 7ff686ffd700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 0x7ff67c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.250+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6800cda40 con 0x7ff690074e60 2023-09-23T04:43:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.250+0000 7ff695d9b700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 0x7ff67c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.251+0000 7ff695d9b700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 0x7ff67c0a3a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005fd0 tx=0x7ff68c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.254+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff6800974c0 con 0x7ff690074e60 2023-09-23T04:43:27.267 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:26 smithi118 podman[171713]: 2023-09-23T04:43:27.267 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:27 smithi118 ceph-mds[171754]: -- 172.21.15.118:0/3909038426 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bcc5ad91e0 con 0x55bcc5aae800 2023-09-23T04:43:27.267 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: starting mds.c at 2023-09-23T04:43:27.268 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:27 smithi118 bash[171713]: dc0dea18b0f8ae87448df7c9c3da63736dc006c9ab07c318d7f808886d8ae064 2023-09-23T04:43:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.512+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff69010e930 con 0x7ff690074e60 2023-09-23T04:43:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.513+0000 7ff686ffd700 1 -- 172.21.15.5:0/1844956749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6068 (secure 0 0 0) 0x7ff68009aae0 con 0x7ff690074e60 2023-09-23T04:43:27.514 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:27.514 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.515+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 msgr2=0x7ff67c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.515+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 0x7ff67c0a3a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005fd0 tx=0x7ff68c009310 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.515+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 msgr2=0x7ff690199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.515+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff680009fd0 tx=0x7ff68000b790 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 shutdown_connections 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff67c0a15a0 0x7ff67c0a3a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff690108db0 0x7ff6901a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff690073880 0x7ff690198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 --2- 172.21.15.5:0/1844956749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff690074e60 0x7ff690199390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:27.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 >> 172.21.15.5:0/1844956749 conn(0x7ff6900fc480 msgr2=0x7ff6900fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:27.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 shutdown_connections 2023-09-23T04:43:27.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:27.516+0000 7ff697fff700 1 -- 172.21.15.5:0/1844956749 wait complete. 2023-09-23T04:43:27.518 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-09-23T04:43:27.661 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 04:43:27 smithi118 systemd[1]: Started Ceph mds.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:28.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: Deploying daemon mds.h on smithi069 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:boot 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: fsmap 7 up:standby 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1844956749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:28 smithi005 ceph-mon[97638]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:28.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: Deploying daemon mds.h on smithi069 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:boot 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: fsmap 7 up:standby 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1844956749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:28 smithi118 ceph-mon[134297]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:28.702 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.702 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.702 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:28.702 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: Deploying daemon mds.h on smithi069 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:boot 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: fsmap 7 up:standby 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1844956749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:28.703 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:28 smithi069 ceph-mon[133931]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:28.991 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:28 smithi069 systemd[1]: Starting Ceph mds.h for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:29.491 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:29 smithi069 podman[170208]: 2023-09-23T04:43:29.491 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:29 smithi069 ceph-mds[170248]: -- 172.21.15.69:0/1474834024 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557eeaa631e0 con 0x557eeaa38800 2023-09-23T04:43:29.491 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:29 smithi069 bash[170208]: f71571f1678024671620a28456d8a9241887b9159cc03db596842af2766900f1 2023-09-23T04:43:29.491 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: starting mds.h at 2023-09-23T04:43:29.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.753+0000 7fb918dff700 1 -- 172.21.15.5:0/2883339571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 msgr2=0x7fb914100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.753+0000 7fb918dff700 1 --2- 172.21.15.5:0/2883339571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914100220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb9040099c0 tx=0x7fb904009cd0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 -- 172.21.15.5:0/2883339571 shutdown_connections 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 --2- 172.21.15.5:0/2883339571 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 0x7fb914108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 --2- 172.21.15.5:0/2883339571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914100220 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 --2- 172.21.15.5:0/2883339571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 -- 172.21.15.5:0/2883339571 >> 172.21.15.5:0/2883339571 conn(0x7fb9140fb350 msgr2=0x7fb9140fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 -- 172.21.15.5:0/2883339571 shutdown_connections 2023-09-23T04:43:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.754+0000 7fb918dff700 1 -- 172.21.15.5:0/2883339571 wait complete. 2023-09-23T04:43:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 Processor -- start 2023-09-23T04:43:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 -- start start 2023-09-23T04:43:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 0x7fb9141a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141a0970 con 0x7fb9140ffe00 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141a0ab0 con 0x7fb914100ff0 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb918dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141a0c40 con 0x7fb914101990 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb91259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb911d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb91259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35730/0 (socket says 172.21.15.5:35730) 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb91259c700 1 -- 172.21.15.5:0/3169428212 learned_addr learned my addr 172.21.15.5:0/3169428212 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.755+0000 7fb912d9d700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 0x7fb9141a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 -- 172.21.15.5:0/3169428212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 msgr2=0x7fb9141a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 0x7fb9141a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 -- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 msgr2=0x7fb914198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 -- 172.21.15.5:0/3169428212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040095f0 con 0x7fb914100ff0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb911d9b700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914199330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb90800e9f0 tx=0x7fb90800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90800c9c0 con 0x7fb914100ff0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb908010430 con 0x7fb914100ff0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9141a0f20 con 0x7fb914100ff0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.756+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb908018680 con 0x7fb914100ff0 2023-09-23T04:43:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.757+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9141a13a0 con 0x7fb914100ff0 2023-09-23T04:43:29.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.758+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91410b660 con 0x7fb914100ff0 2023-09-23T04:43:29.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.760+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fb908010860 con 0x7fb914100ff0 2023-09-23T04:43:29.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.762+0000 7fb9037fe700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 0x7fb8fc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.762+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb908014070 con 0x7fb914100ff0 2023-09-23T04:43:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.762+0000 7fb91259c700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 0x7fb8fc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.763+0000 7fb91259c700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 0x7fb8fc0a3a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb904000c00 tx=0x7fb904005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:29.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:29.764+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb9080965d0 con 0x7fb914100ff0 2023-09-23T04:43:29.991 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 04:43:29 smithi069 systemd[1]: Started Ceph mds.h for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:30.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.021+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9141a1590 con 0x7fb914100ff0 2023-09-23T04:43:30.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.023+0000 7fb9037fe700 1 -- 172.21.15.5:0/3169428212 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6859 (secure 0 0 0) 0x7fb908099bf0 con 0x7fb914100ff0 2023-09-23T04:43:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:30.023 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-23T04:43:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 msgr2=0x7fb8fc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 0x7fb8fc0a3a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb904000c00 tx=0x7fb904005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 msgr2=0x7fb914199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914199330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb90800e9f0 tx=0x7fb90800ed00 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 shutdown_connections 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8fc0a15a0 0x7fb8fc0a3a60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb914101990 0x7fb9141a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9140ffe00 0x7fb914198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 --2- 172.21.15.5:0/3169428212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb914100ff0 0x7fb914199330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 >> 172.21.15.5:0/3169428212 conn(0x7fb9140fb350 msgr2=0x7fb914068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 shutdown_connections 2023-09-23T04:43:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:30.024+0000 7fb918dff700 1 -- 172.21.15.5:0/3169428212 wait complete. 2023-09-23T04:43:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6836/2958173168,v1:172.21.15.69:6837/2958173168] up:boot 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: fsmap 8 up:standby 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:30.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: Deploying daemon mds.i on smithi118 2023-09-23T04:43:30.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:30.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:30 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3169428212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:30.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6836/2958173168,v1:172.21.15.69:6837/2958173168] up:boot 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: fsmap 8 up:standby 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: Deploying daemon mds.i on smithi118 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:30.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:30 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3169428212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6836/2958173168,v1:172.21.15.69:6837/2958173168] up:boot 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: fsmap 8 up:standby 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: Deploying daemon mds.i on smithi118 2023-09-23T04:43:30.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:30.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:30 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3169428212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:31.605 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 systemd[1]: Starting Ceph mds.i for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:31.863 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 podman[172052]: 2023-09-23T04:43:32.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:32.160 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 ceph-mds[172092]: -- 172.21.15.118:0/1017665445 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562939c531e0 con 0x562939c29000 2023-09-23T04:43:32.161 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: starting mds.i at 2023-09-23T04:43:32.161 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 bash[172052]: 07775c8a2103671662787132c3beea377f9f4b1ed3288ce2940dd2e18acbde8b 2023-09-23T04:43:32.161 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 04:43:31 smithi118 systemd[1]: Started Ceph mds.i for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 -- 172.21.15.5:0/2021653108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 msgr2=0x7fdbdc0732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/2021653108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc0732b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009990 tx=0x7fdbcc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 -- 172.21.15.5:0/2021653108 shutdown_connections 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/2021653108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc108da0 0x7fdbdc10b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/2021653108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc0732b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/2021653108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc073880 0x7fdbdc073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.263+0000 7fdbe21ff700 1 -- 172.21.15.5:0/2021653108 >> 172.21.15.5:0/2021653108 conn(0x7fdbdc0fc450 msgr2=0x7fdbdc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.264+0000 7fdbe21ff700 1 -- 172.21.15.5:0/2021653108 shutdown_connections 2023-09-23T04:43:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.264+0000 7fdbe21ff700 1 -- 172.21.15.5:0/2021653108 wait complete. 2023-09-23T04:43:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 Processor -- start 2023-09-23T04:43:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 -- start start 2023-09-23T04:43:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc073880 0x7fdbdc198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc108da0 0x7fdbdc1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a0980 con 0x7fdbdc108da0 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a0ac0 con 0x7fdbdc073880 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbe21ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1a0c50 con 0x7fdbdc074e60 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbdaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbdaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42706/0 (socket says 172.21.15.5:42706) 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbdaffd700 1 -- 172.21.15.5:0/1605658601 learned_addr learned my addr 172.21.15.5:0/1605658601 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.265+0000 7fdbdb7fe700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc073880 0x7fdbdc198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdbfff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc108da0 0x7fdbdc1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 -- 172.21.15.5:0/1605658601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc073880 msgr2=0x7fdbdc198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc073880 0x7fdbdc198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 -- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc108da0 msgr2=0x7fdbdc1a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc108da0 0x7fdbdc1a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 -- 172.21.15.5:0/1605658601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0095f0 con 0x7fdbdc074e60 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbdaffd700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e970 tx=0x7fdbd000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0009790 con 0x7fdbdc074e60 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbd000eec0 con 0x7fdbdc074e60 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003cf0 con 0x7fdbdc074e60 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc1a0f30 con 0x7fdbdc074e60 2023-09-23T04:43:32.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.266+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc1a1360 con 0x7fdbdc074e60 2023-09-23T04:43:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.267+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc107450 con 0x7fdbdc074e60 2023-09-23T04:43:32.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.269+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fdbd000ccd0 con 0x7fdbdc074e60 2023-09-23T04:43:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.270+0000 7fdbd8ff9700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 0x7fdbc40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.271+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbd0014070 con 0x7fdbdc074e60 2023-09-23T04:43:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.271+0000 7fdbdb7fe700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 0x7fdbc40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.271+0000 7fdbdb7fe700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 0x7fdbc40a3a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b2e0 tx=0x7fdbcc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.274+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fdbd0096740 con 0x7fdbdc074e60 2023-09-23T04:43:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.552+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbdc10e920 con 0x7fdbdc074e60 2023-09-23T04:43:32.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.553+0000 7fdbd8ff9700 1 -- 172.21.15.5:0/1605658601 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6859 (secure 0 0 0) 0x7fdbd0099d60 con 0x7fdbdc074e60 2023-09-23T04:43:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:32.553 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 msgr2=0x7fdbc40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 0x7fdbc40a3a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00b2e0 tx=0x7fdbcc005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 msgr2=0x7fdbdc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e970 tx=0x7fdbd000ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 shutdown_connections 2023-09-23T04:43:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdbc40a15a0 0x7fdbc40a3a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbdc108da0 0x7fdbdc1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbdc073880 0x7fdbdc198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 --2- 172.21.15.5:0/1605658601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdbdc074e60 0x7fdbdc199340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 >> 172.21.15.5:0/1605658601 conn(0x7fdbdc0fc450 msgr2=0x7fdbdc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 shutdown_connections 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:32.555+0000 7fdbe21ff700 1 -- 172.21.15.5:0/1605658601 wait complete. 2023-09-23T04:43:32.556 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2023-09-23T04:43:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:32 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1605658601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:32 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1605658601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:33.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:33.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:33.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:33.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:32 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1605658601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:33.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:33 smithi005 ceph-mon[97638]: Deploying daemon mds.g on smithi005 2023-09-23T04:43:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:33 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:boot 2023-09-23T04:43:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:33 smithi005 ceph-mon[97638]: fsmap 9 up:standby 2023-09-23T04:43:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:33 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:43:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:33 smithi069 ceph-mon[133931]: Deploying daemon mds.g on smithi005 2023-09-23T04:43:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:33 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:boot 2023-09-23T04:43:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:33 smithi069 ceph-mon[133931]: fsmap 9 up:standby 2023-09-23T04:43:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:33 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:43:34.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:33 smithi118 ceph-mon[134297]: Deploying daemon mds.g on smithi005 2023-09-23T04:43:34.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:33 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:boot 2023-09-23T04:43:34.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:33 smithi118 ceph-mon[134297]: fsmap 9 up:standby 2023-09-23T04:43:34.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:33 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:43:34.322 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:34 smithi005 systemd[1]: Starting Ceph mds.g for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:34.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 -- 172.21.15.5:0/2212324379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130100ea0 msgr2=0x7f11301012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 --2- 172.21.15.5:0/2212324379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130100ea0 0x7f11301012c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1120009990 tx=0x7f1120009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 -- 172.21.15.5:0/2212324379 shutdown_connections 2023-09-23T04:43:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 --2- 172.21.15.5:0/2212324379 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130102a40 0x7f1130109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 --2- 172.21.15.5:0/2212324379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130100ea0 0x7f11301012c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 --2- 172.21.15.5:0/2212324379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102080 0x7f1130102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 -- 172.21.15.5:0/2212324379 >> 172.21.15.5:0/2212324379 conn(0x7f11300fc480 msgr2=0x7f11300fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 -- 172.21.15.5:0/2212324379 shutdown_connections 2023-09-23T04:43:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.801+0000 7f11357ff700 1 -- 172.21.15.5:0/2212324379 wait complete. 2023-09-23T04:43:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.802+0000 7f11357ff700 1 Processor -- start 2023-09-23T04:43:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 -- start start 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130100ea0 0x7f1130198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102a40 0x7f11301a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11301a09d0 con 0x7f1130102080 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11301a0b10 con 0x7f1130102a40 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f11357ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11301a0ca0 con 0x7f1130100ea0 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f112e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f112e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47702/0 (socket says 172.21.15.5:47702) 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f112e7fc700 1 -- 172.21.15.5:0/3469790891 learned_addr learned my addr 172.21.15.5:0/3469790891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f112effd700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130100ea0 0x7f1130198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.803+0000 7f112f7fe700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102a40 0x7f11301a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 -- 172.21.15.5:0/3469790891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130100ea0 msgr2=0x7f1130198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130100ea0 0x7f1130198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 -- 172.21.15.5:0/3469790891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102a40 msgr2=0x7f11301a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102a40 0x7f11301a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 -- 172.21.15.5:0/3469790891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11200095f0 con 0x7f1130102080 2023-09-23T04:43:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f112e7fc700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f112400e970 tx=0x7f112400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124009790 con 0x7f1130102080 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f112400c7e0 con 0x7f1130102080 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124003c70 con 0x7f1130102080 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11301a0f80 con 0x7f1130102080 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.804+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11301a13b0 con 0x7f1130102080 2023-09-23T04:43:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.805+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1130193550 con 0x7f1130102080 2023-09-23T04:43:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.807+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f112400cc50 con 0x7f1130102080 2023-09-23T04:43:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.809+0000 7f1117fff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 0x7f11180a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.809+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1124014070 con 0x7f1130102080 2023-09-23T04:43:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.809+0000 7f112effd700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 0x7f11180a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.810+0000 7f112effd700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 0x7f11180a3ae0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f112000b2e0 tx=0x7f1120005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:34.813+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1124096630 con 0x7f1130102080 2023-09-23T04:43:34.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:34 smithi005 ceph-mon[97638]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:34.905 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:34 smithi005 podman[149118]: 2023-09-23T04:43:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:34 smithi069 ceph-mon[133931]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.081+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f113010c760 con 0x7f1130102080 2023-09-23T04:43:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.082+0000 7f1117fff700 1 -- 172.21.15.5:0/3469790891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7652 (secure 0 0 0) 0x7f1124003dd0 con 0x7f1130102080 2023-09-23T04:43:35.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:35.082 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-09-23T04:43:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 msgr2=0x7f11180a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 0x7f11180a3ae0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f112000b2e0 tx=0x7f1120005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 msgr2=0x7f1130199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f112400e970 tx=0x7f112400ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 shutdown_connections 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f11180a1620 0x7f11180a3ae0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130102a40 0x7f11301a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1130100ea0 0x7f1130198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 --2- 172.21.15.5:0/3469790891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1130102080 0x7f1130199390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 >> 172.21.15.5:0/3469790891 conn(0x7f11300fc480 msgr2=0x7f1130107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 shutdown_connections 2023-09-23T04:43:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:35.085+0000 7f11357ff700 1 -- 172.21.15.5:0/3469790891 wait complete. 2023-09-23T04:43:35.087 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2023-09-23T04:43:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:34 smithi118 ceph-mon[134297]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:35.404 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:35 smithi005 bash[149118]: 62e76a10346a9d0a27e674caa7241acc5a22eee3c3ccfa5606c0180b925bd498 2023-09-23T04:43:35.404 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mds[149186]: -- 172.21.15.5:0/1938734166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cab06b31e0 con 0x55cab0688c00 2023-09-23T04:43:35.405 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: starting mds.g at 2023-09-23T04:43:35.405 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:35 smithi005 systemd[1]: Started Ceph mds.g for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3469790891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:35 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:36.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3469790891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:36.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:36.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:36.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:35 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3469790891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:35 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:36 smithi005 ceph-mon[97638]: Deploying daemon mds.a on smithi005 2023-09-23T04:43:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:36 smithi005 ceph-mon[97638]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:36 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:boot 2023-09-23T04:43:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:36 smithi005 ceph-mon[97638]: fsmap 10 up:standby 2023-09-23T04:43:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:36 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:43:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:36 smithi069 ceph-mon[133931]: Deploying daemon mds.a on smithi005 2023-09-23T04:43:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:36 smithi069 ceph-mon[133931]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:36 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:boot 2023-09-23T04:43:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:36 smithi069 ceph-mon[133931]: fsmap 10 up:standby 2023-09-23T04:43:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:36 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:43:37.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:36 smithi118 ceph-mon[134297]: Deploying daemon mds.a on smithi005 2023-09-23T04:43:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:36 smithi118 ceph-mon[134297]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:36 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:boot 2023-09-23T04:43:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:36 smithi118 ceph-mon[134297]: fsmap 10 up:standby 2023-09-23T04:43:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:36 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:43:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 -- 172.21.15.5:0/3950083741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 msgr2=0x7fed641012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 --2- 172.21.15.5:0/3950083741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed641012d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed54009990 tx=0x7fed54009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 -- 172.21.15.5:0/3950083741 shutdown_connections 2023-09-23T04:43:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 --2- 172.21.15.5:0/3950083741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed64109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 --2- 172.21.15.5:0/3950083741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed641012d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 --2- 172.21.15.5:0/3950083741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 -- 172.21.15.5:0/3950083741 >> 172.21.15.5:0/3950083741 conn(0x7fed640fc470 msgr2=0x7fed640fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 -- 172.21.15.5:0/3950083741 shutdown_connections 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.338+0000 7fed69bff700 1 -- 172.21.15.5:0/3950083741 wait complete. 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.339+0000 7fed69bff700 1 Processor -- start 2023-09-23T04:43:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 -- start start 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed6419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419e770 con 0x7fed64102a50 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419e8e0 con 0x7fed64100eb0 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.340+0000 7fed69bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6419eaa0 con 0x7fed64102090 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed637fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed637fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38560/0 (socket says 172.21.15.5:38560) 2023-09-23T04:43:37.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed6419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed637fe700 1 -- 172.21.15.5:0/3326132671 learned_addr learned my addr 172.21.15.5:0/3326132671 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed62ffd700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 -- 172.21.15.5:0/3326132671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 msgr2=0x7fed64197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 -- 172.21.15.5:0/3326132671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 msgr2=0x7fed64196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 -- 172.21.15.5:0/3326132671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009c40 con 0x7fed64102a50 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed637fe700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed62ffd700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:43:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed63fff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed6419e1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00efb0 tx=0x7fed5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.341+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00ca50 con 0x7fed64102a50 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.342+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed540095f0 con 0x7fed64102a50 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.342+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c00f040 con 0x7fed64102a50 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.342+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c004660 con 0x7fed64102a50 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.342+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6419f100 con 0x7fed64102a50 2023-09-23T04:43:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.343+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed64107eb0 con 0x7fed64102a50 2023-09-23T04:43:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.345+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fed5c0047c0 con 0x7fed64102a50 2023-09-23T04:43:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.346+0000 7fed60ff9700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 0x7fed4c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.346+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed5c0cc860 con 0x7fed64102a50 2023-09-23T04:43:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.346+0000 7fed637fe700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 0x7fed4c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.347+0000 7fed637fe700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 0x7fed4c0a3a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b2e0 tx=0x7fed54005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.350+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fed5c0962e0 con 0x7fed64102a50 2023-09-23T04:43:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.644+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed6402d030 con 0x7fed64102a50 2023-09-23T04:43:37.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.644+0000 7fed60ff9700 1 -- 172.21.15.5:0/3326132671 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8441 (secure 0 0 0) 0x7fed5c099900 con 0x7fed64102a50 2023-09-23T04:43:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:37.644 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 msgr2=0x7fed4c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 0x7fed4c0a3a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b2e0 tx=0x7fed54005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 msgr2=0x7fed6419e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed6419e1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00efb0 tx=0x7fed5c00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 shutdown_connections 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed4c0a15a0 0x7fed4c0a3a60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed64102a50 0x7fed6419e1a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed64100eb0 0x7fed64196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 --2- 172.21.15.5:0/3326132671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed64102090 0x7fed64197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 >> 172.21.15.5:0/3326132671 conn(0x7fed640fc470 msgr2=0x7fed640fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 shutdown_connections 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:37.647+0000 7fed69bff700 1 -- 172.21.15.5:0/3326132671 wait complete. 2023-09-23T04:43:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2023-09-23T04:43:37.706 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:37 smithi005 systemd[1]: Starting Ceph mds.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:37 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3326132671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:38.070 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:37 smithi005 podman[149537]: 2023-09-23T04:43:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:37 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3326132671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:37 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3326132671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:38.654 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:38 smithi005 bash[149537]: aec7687f445bfbaea7dcf436113130706eeb06baac87c04ff79a534f1ce0079b 2023-09-23T04:43:38.655 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mds[149577]: -- 172.21.15.5:0/2672613481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562001a491e0 con 0x562001a1e800 2023-09-23T04:43:38.655 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: starting mds.a at 2023-09-23T04:43:38.655 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 04:43:38 smithi005 systemd[1]: Started Ceph mds.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:38 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:38 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-23T04:43:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:38 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:39.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 -- 172.21.15.5:0/4154990418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102070 msgr2=0x7f4d041024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 --2- 172.21.15.5:0/4154990418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102070 0x7f4d041024f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009990 tx=0x7f4cf0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 -- 172.21.15.5:0/4154990418 shutdown_connections 2023-09-23T04:43:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 --2- 172.21.15.5:0/4154990418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102a30 0x7f4d04109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 --2- 172.21.15.5:0/4154990418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d041012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 --2- 172.21.15.5:0/4154990418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102070 0x7f4d041024f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 -- 172.21.15.5:0/4154990418 >> 172.21.15.5:0/4154990418 conn(0x7f4d040fc470 msgr2=0x7f4d040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:39.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 -- 172.21.15.5:0/4154990418 shutdown_connections 2023-09-23T04:43:39.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.891+0000 7f4d0adff700 1 -- 172.21.15.5:0/4154990418 wait complete. 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.892+0000 7f4d0adff700 1 Processor -- start 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.892+0000 7f4d0adff700 1 -- start start 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.892+0000 7f4d0adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.892+0000 7f4d0adff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102070 0x7f4d04197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 0x7f4d0419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0419e740 con 0x7f4d04102a30 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0419e880 con 0x7f4d04100e90 2023-09-23T04:43:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0adff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0419ea10 con 0x7f4d04102070 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 0x7f4d0419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38606/0 (socket says 172.21.15.5:38606) 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 -- 172.21.15.5:0/3050458929 learned_addr learned my addr 172.21.15.5:0/3050458929 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d03fff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102070 0x7f4d04197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 -- 172.21.15.5:0/3050458929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102070 msgr2=0x7f4d04197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102070 0x7f4d04197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 -- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 msgr2=0x7f4d0419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 0x7f4d0419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 -- 172.21.15.5:0/3050458929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00095f0 con 0x7f4d04100e90 2023-09-23T04:43:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d0939c700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 0x7f4d0419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.893+0000 7f4d08b9b700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800d980 tx=0x7f4cf800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.894+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8004020 con 0x7f4d04100e90 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.894+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cf800f690 con 0x7f4d04100e90 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.894+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0419ecf0 con 0x7f4d04100e90 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.894+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8003a60 con 0x7f4d04100e90 2023-09-23T04:43:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.894+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0419f1a0 con 0x7f4d04100e90 2023-09-23T04:43:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.895+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0410c700 con 0x7f4d04100e90 2023-09-23T04:43:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.897+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4cf800fb00 con 0x7f4d04100e90 2023-09-23T04:43:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.898+0000 7f4d01ffb700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 0x7f4cf40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.898+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4cf80cc610 con 0x7f4d04100e90 2023-09-23T04:43:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.898+0000 7f4d03fff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 0x7f4cf40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.899+0000 7f4d03fff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 0x7f4cf40a3ae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000b5c0 tx=0x7f4cf0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:39.900+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4cf8096080 con 0x7f4d04100e90 2023-09-23T04:43:39.941 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:39 smithi069 systemd[1]: Starting Ceph mds.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T04:43:39.941 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:39 smithi069 ceph-mon[133931]: Deploying daemon mds.b on smithi069 2023-09-23T04:43:39.941 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:39 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6840/1890134322,v1:172.21.15.5:6841/1890134322] up:boot 2023-09-23T04:43:39.942 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:39 smithi069 ceph-mon[133931]: fsmap 11 up:standby 2023-09-23T04:43:39.942 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:39 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:43:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:39 smithi005 ceph-mon[97638]: Deploying daemon mds.b on smithi069 2023-09-23T04:43:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:39 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6840/1890134322,v1:172.21.15.5:6841/1890134322] up:boot 2023-09-23T04:43:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:39 smithi005 ceph-mon[97638]: fsmap 11 up:standby 2023-09-23T04:43:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:39 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:43:40.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.160+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d04107e60 con 0x7f4d04100e90 2023-09-23T04:43:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:39 smithi118 ceph-mon[134297]: Deploying daemon mds.b on smithi069 2023-09-23T04:43:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:39 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6840/1890134322,v1:172.21.15.5:6841/1890134322] up:boot 2023-09-23T04:43:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:39 smithi118 ceph-mon[134297]: fsmap 11 up:standby 2023-09-23T04:43:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:39 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:43:40.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.161+0000 7f4d01ffb700 1 -- 172.21.15.5:0/3050458929 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9230 (secure 0 0 0) 0x7f4cf80996a0 con 0x7f4d04100e90 2023-09-23T04:43:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:40.162 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 msgr2=0x7f4cf40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 0x7f4cf40a3ae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000b5c0 tx=0x7f4cf0005fd0 comp rx=0 tx=0).stop 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 msgr2=0x7f4d04196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800d980 tx=0x7f4cf800dc90 comp rx=0 tx=0).stop 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 shutdown_connections 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4cf40a1620 0x7f4cf40a3ae0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:40.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d04102a30 0x7f4d0419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d04100e90 0x7f4d04196bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 --2- 172.21.15.5:0/3050458929 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4d04102070 0x7f4d04197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 >> 172.21.15.5:0/3050458929 conn(0x7f4d040fc470 msgr2=0x7f4d040fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 shutdown_connections 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:40.164+0000 7f4d0adff700 1 -- 172.21.15.5:0/3050458929 wait complete. 2023-09-23T04:43:40.165 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2023-09-23T04:43:40.447 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:40 smithi069 podman[170547]: 2023-09-23T04:43:40.726 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mds[170587]: -- 172.21.15.69:0/1034809583 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9524c91e0 con 0x55c95249f000 2023-09-23T04:43:40.727 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: starting mds.b at 2023-09-23T04:43:40.727 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:40 smithi069 bash[170547]: cf02c40952a743624952848c942a33f48151a903665da806c1e451c5ff773fc5 2023-09-23T04:43:40.727 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 04:43:40 smithi069 systemd[1]: Started Ceph mds.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3050458929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:40 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3050458929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:40 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3050458929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:40 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:41 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:41 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:boot 2023-09-23T04:43:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:41 smithi069 ceph-mon[133931]: fsmap 12 up:standby 2023-09-23T04:43:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:41 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:43:42.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:41 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:boot 2023-09-23T04:43:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:41 smithi005 ceph-mon[97638]: fsmap 12 up:standby 2023-09-23T04:43:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:41 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:43:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:41 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:41 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:boot 2023-09-23T04:43:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:41 smithi118 ceph-mon[134297]: fsmap 12 up:standby 2023-09-23T04:43:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:41 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:43:42.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/437027208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38073880 msgr2=0x7f8b38073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/437027208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38073880 0x7f8b38073d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009990 tx=0x7f8b2c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/437027208 shutdown_connections 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/437027208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 0x7f8b3810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/437027208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38074e60 0x7f8b380732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/437027208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38073880 0x7f8b38073d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/437027208 >> 172.21.15.5:0/437027208 conn(0x7f8b380fc480 msgr2=0x7f8b380fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/437027208 shutdown_connections 2023-09-23T04:43:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.397+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/437027208 wait complete. 2023-09-23T04:43:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.399+0000 7f8b3e5ff700 1 Processor -- start 2023-09-23T04:43:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 -- start start 2023-09-23T04:43:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 0x7f8b38198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 0x7f8b381a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b381a09b0 con 0x7f8b38108db0 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b381a0af0 con 0x7f8b38074e60 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3e5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b381a0c80 con 0x7f8b38073880 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41724/0 (socket says 172.21.15.5:41724) 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b377fe700 1 -- 172.21.15.5:0/1139904064 learned_addr learned my addr 172.21.15.5:0/1139904064 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b37fff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 0x7f8b38198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.400+0000 7f8b3cb9c700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 0x7f8b381a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 -- 172.21.15.5:0/1139904064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 msgr2=0x7f8b38198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 0x7f8b38198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 -- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 msgr2=0x7f8b381a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 0x7f8b381a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 -- 172.21.15.5:0/1139904064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0095f0 con 0x7f8b38074e60 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b37fff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 0x7f8b38198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b377fe700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0098e0 tx=0x7f8b2c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c01d070 con 0x7f8b38074e60 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b381a0f00 con 0x7f8b38074e60 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b2c0054f0 con 0x7f8b38074e60 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.401+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c004be0 con 0x7f8b38074e60 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.402+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b381a1270 con 0x7f8b38074e60 2023-09-23T04:43:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.402+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38107460 con 0x7f8b38074e60 2023-09-23T04:43:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.405+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f8b2c00fbc0 con 0x7f8b38074e60 2023-09-23T04:43:42.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.406+0000 7f8b357fa700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 0x7f8b200a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.406+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b2c0cdd20 con 0x7f8b38074e60 2023-09-23T04:43:42.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.406+0000 7f8b37fff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 0x7f8b200a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.407+0000 7f8b37fff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 0x7f8b200a3a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011490 tx=0x7f8b28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.408+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8b2c0977a0 con 0x7f8b38074e60 2023-09-23T04:43:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.677+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b3810e930 con 0x7f8b38074e60 2023-09-23T04:43:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.678+0000 7f8b357fa700 1 -- 172.21.15.5:0/1139904064 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10022 (secure 0 0 0) 0x7f8b2c09adc0 con 0x7f8b38074e60 2023-09-23T04:43:42.678 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:42.678 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-09-23T04:43:42.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 msgr2=0x7f8b200a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 0x7f8b200a3a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8b28011490 tx=0x7f8b28009260 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 msgr2=0x7f8b38199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0098e0 tx=0x7f8b2c004730 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 shutdown_connections 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8b200a15a0 0x7f8b200a3a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b38108db0 0x7f8b381a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8b38073880 0x7f8b38198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 --2- 172.21.15.5:0/1139904064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b38074e60 0x7f8b38199370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 >> 172.21.15.5:0/1139904064 conn(0x7f8b380fc480 msgr2=0x7f8b380fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 shutdown_connections 2023-09-23T04:43:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.680+0000 7f8b3e5ff700 1 -- 172.21.15.5:0/1139904064 wait complete. 2023-09-23T04:43:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2023-09-23T04:43:42.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 -- 172.21.15.5:0/3045315865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74101950 msgr2=0x7fbc74103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 --2- 172.21.15.5:0/3045315865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74101950 0x7fbc74103d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 -- 172.21.15.5:0/3045315865 shutdown_connections 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 --2- 172.21.15.5:0/3045315865 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 0x7fbc74109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 --2- 172.21.15.5:0/3045315865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74101950 0x7fbc74103d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 --2- 172.21.15.5:0/3045315865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74104280 0x7fbc74106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.912+0000 7fbc789ff700 1 -- 172.21.15.5:0/3045315865 >> 172.21.15.5:0/3045315865 conn(0x7fbc740fb360 msgr2=0x7fbc740fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.913+0000 7fbc789ff700 1 -- 172.21.15.5:0/3045315865 shutdown_connections 2023-09-23T04:43:42.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.913+0000 7fbc789ff700 1 -- 172.21.15.5:0/3045315865 wait complete. 2023-09-23T04:43:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 Processor -- start 2023-09-23T04:43:42.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 -- start start 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74104280 0x7fbc741970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 0x7fbc7419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7419e730 con 0x7fbc74104280 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7419e870 con 0x7fbc74101950 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc789ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7419ea00 con 0x7fbc74106bb0 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc7259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41732/0 (socket says 172.21.15.5:41732) 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc7259c700 1 -- 172.21.15.5:0/1820552170 learned_addr learned my addr 172.21.15.5:0/1820552170 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc72d9d700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 0x7fbc7419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.914+0000 7fbc71d9b700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74104280 0x7fbc741970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 -- 172.21.15.5:0/1820552170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 msgr2=0x7fbc7419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 0x7fbc7419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 -- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74104280 msgr2=0x7fbc741970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74104280 0x7fbc741970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 -- 172.21.15.5:0/1820552170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640095f0 con 0x7fbc74101950 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc7259c700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b680 tx=0x7fbc64005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6401d070 con 0x7fbc74101950 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc64022470 con 0x7fbc74101950 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64017650 con 0x7fbc74101950 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.915+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7419ec80 con 0x7fbc74101950 2023-09-23T04:43:42.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.916+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7419f050 con 0x7fbc74101950 2023-09-23T04:43:42.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.917+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc740fcee0 con 0x7fbc74101950 2023-09-23T04:43:42.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.919+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fbc640225e0 con 0x7fbc74101950 2023-09-23T04:43:42.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.920+0000 7fbc637fe700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 0x7fbc5c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:42.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.920+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc640ce690 con 0x7fbc74101950 2023-09-23T04:43:42.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.921+0000 7fbc71d9b700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 0x7fbc5c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.921+0000 7fbc71d9b700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 0x7fbc5c0a3a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009c60 tx=0x7fbc68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:42.922+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbc64098110 con 0x7fbc74101950 2023-09-23T04:43:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:42 smithi069 ceph-mon[133931]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:42 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1139904064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:43.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.139+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc7410c950 con 0x7fbc74101950 2023-09-23T04:43:43.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.140+0000 7fbc637fe700 1 -- 172.21.15.5:0/1820552170 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18328 (secure 0 0 0) 0x7fbc6409b730 con 0x7fbc74101950 2023-09-23T04:43:43.140 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:43.140 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":79,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:00.465274+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 msgr2=0x7fbc5c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 0x7fbc5c0a3a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009c60 tx=0x7fbc68009260 comp rx=0 tx=0).stop 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 msgr2=0x7fbc74196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b680 tx=0x7fbc64005740 comp rx=0 tx=0).stop 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 shutdown_connections 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbc5c0a15a0 0x7fbc5c0a3a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbc74106bb0 0x7fbc7419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc74101950 0x7fbc74196bb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 --2- 172.21.15.5:0/1820552170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc74104280 0x7fbc741970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 >> 172.21.15.5:0/1820552170 conn(0x7fbc740fb360 msgr2=0x7fbc741025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 shutdown_connections 2023-09-23T04:43:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.143+0000 7fbc789ff700 1 -- 172.21.15.5:0/1820552170 wait complete. 2023-09-23T04:43:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:42 smithi005 ceph-mon[97638]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:42 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1139904064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:42 smithi118 ceph-mon[134297]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:42 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1139904064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:43.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T04:43:43.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:43:43.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 -- 172.21.15.5:0/2192760121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38073820 msgr2=0x7fed38073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/2192760121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38073820 0x7fed38073ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009990 tx=0x7fed2c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 -- 172.21.15.5:0/2192760121 shutdown_connections 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/2192760121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38108dd0 0x7fed3810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/2192760121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/2192760121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38073820 0x7fed38073ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 -- 172.21.15.5:0/2192760121 >> 172.21.15.5:0/2192760121 conn(0x7fed380fc490 msgr2=0x7fed380fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.378+0000 7fed3d5ff700 1 -- 172.21.15.5:0/2192760121 shutdown_connections 2023-09-23T04:43:43.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.379+0000 7fed3d5ff700 1 -- 172.21.15.5:0/2192760121 wait complete. 2023-09-23T04:43:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.379+0000 7fed3d5ff700 1 Processor -- start 2023-09-23T04:43:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 -- start start 2023-09-23T04:43:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38108dd0 0x7fed381a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381a0980 con 0x7fed38108dd0 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381a0ac0 con 0x7fed38074e00 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed3d5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381a0c50 con 0x7fed38073820 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed36ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed367fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed36ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45344/0 (socket says 172.21.15.5:45344) 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed36ffd700 1 -- 172.21.15.5:0/362005242 learned_addr learned my addr 172.21.15.5:0/362005242 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.380+0000 7fed377fe700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38108dd0 0x7fed381a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 -- 172.21.15.5:0/362005242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 msgr2=0x7fed38199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 -- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38108dd0 msgr2=0x7fed381a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38108dd0 0x7fed381a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 -- 172.21.15.5:0/362005242 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009520 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed367fc700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed36ffd700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eb30 tx=0x7fed2800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28009880 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed280044e0 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c0095f0 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28003e90 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.381+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed381a11f0 con 0x7fed38073820 2023-09-23T04:43:43.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.383+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed381074d0 con 0x7fed38073820 2023-09-23T04:43:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.385+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fed28004000 con 0x7fed38073820 2023-09-23T04:43:43.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.386+0000 7fed1ffff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.386+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed28014070 con 0x7fed38073820 2023-09-23T04:43:43.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.387+0000 7fed367fc700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.388+0000 7fed367fc700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed2c006010 tx=0x7fed2c005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.391+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fed280d1070 con 0x7fed38073820 2023-09-23T04:43:43.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.612+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed3802d1a0 con 0x7fed38073820 2023-09-23T04:43:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.612+0000 7fed1ffff700 1 -- 172.21.15.5:0/362005242 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed28096960 con 0x7fed38073820 2023-09-23T04:43:43.612 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:43:43.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 msgr2=0x7fed200a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fed2c006010 tx=0x7fed2c005a90 comp rx=0 tx=0).stop 2023-09-23T04:43:43.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 msgr2=0x7fed38198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed2800eb30 tx=0x7fed2800ee40 comp rx=0 tx=0).stop 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 shutdown_connections 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed38108dd0 0x7fed381a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed38073820 0x7fed38198e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 --2- 172.21.15.5:0/362005242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed38074e00 0x7fed38199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 >> 172.21.15.5:0/362005242 conn(0x7fed380fc490 msgr2=0x7fed380fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:43.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 shutdown_connections 2023-09-23T04:43:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.615+0000 7fed3d5ff700 1 -- 172.21.15.5:0/362005242 wait complete. 2023-09-23T04:43:43.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T04:43:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 -- 172.21.15.5:0/4088449880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 msgr2=0x7fa968101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 --2- 172.21.15.5:0/4088449880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968101290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9580098f0 tx=0x7fa958009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 -- 172.21.15.5:0/4088449880 shutdown_connections 2023-09-23T04:43:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 --2- 172.21.15.5:0/4088449880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102a10 0x7fa968109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 --2- 172.21.15.5:0/4088449880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968101290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 --2- 172.21.15.5:0/4088449880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102050 0x7fa9681024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 -- 172.21.15.5:0/4088449880 >> 172.21.15.5:0/4088449880 conn(0x7fa9680fc410 msgr2=0x7fa9680fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 -- 172.21.15.5:0/4088449880 shutdown_connections 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.846+0000 7fa96dbff700 1 -- 172.21.15.5:0/4088449880 wait complete. 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.848+0000 7fa96dbff700 1 Processor -- start 2023-09-23T04:43:43.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.848+0000 7fa96dbff700 1 -- start start 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.848+0000 7fa96dbff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.848+0000 7fa96dbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102050 0x7fa9681993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa96dbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 0x7fa9681a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa96dbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a09e0 con 0x7fa968102050 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa96dbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0b20 con 0x7fa968102a10 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa96dbff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0cb0 con 0x7fa968100e70 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45370/0 (socket says 172.21.15.5:45370) 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 -- 172.21.15.5:0/3840519102 learned_addr learned my addr 172.21.15.5:0/3840519102 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa966ffd700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102050 0x7fa9681993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa967fff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 0x7fa9681a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 -- 172.21.15.5:0/3840519102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 msgr2=0x7fa9681a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 0x7fa9681a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 -- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102050 msgr2=0x7fa9681993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102050 0x7fa9681993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 -- 172.21.15.5:0/3840519102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009520 con 0x7fa968100e70 2023-09-23T04:43:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa967fff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 0x7fa9681a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.849+0000 7fa9677fe700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fd0 tx=0x7fa95800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.850+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95801c070 con 0x7fa968100e70 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.850+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa95800f7d0 con 0x7fa968100e70 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.850+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958003b00 con 0x7fa968100e70 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.850+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681a0f30 con 0x7fa968100e70 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.850+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681a12a0 con 0x7fa968100e70 2023-09-23T04:43:43.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.851+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96810c730 con 0x7fa968100e70 2023-09-23T04:43:43.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.853+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa958020030 con 0x7fa968100e70 2023-09-23T04:43:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.854+0000 7fa964ff9700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 0x7fa9500a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.854+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9580cdd10 con 0x7fa968100e70 2023-09-23T04:43:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.855+0000 7fa966ffd700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 0x7fa9500a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.855+0000 7fa966ffd700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 0x7fa9500a3ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa968100b90 tx=0x7fa95c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:43.856+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa9580978b0 con 0x7fa968100e70 2023-09-23T04:43:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1820552170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/362005242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:43 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1820552170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/362005242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:43 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.085+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa9681a1490 con 0x7fa968100e70 2023-09-23T04:43:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.085+0000 7fa964ff9700 1 -- 172.21.15.5:0/3840519102 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa95809aed0 con 0x7fa968100e70 2023-09-23T04:43:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:44.085 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-09-23T04:43:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T04:43:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2023-09-23T04:43:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2023-09-23T04:43:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:43:44.086 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 msgr2=0x7fa9500a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 0x7fa9500a3ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa968100b90 tx=0x7fa95c009260 comp rx=0 tx=0).stop 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 msgr2=0x7fa968198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fd0 tx=0x7fa95800b790 comp rx=0 tx=0).stop 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 shutdown_connections 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9500a1620 0x7fa9500a3ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa968102a10 0x7fa9681a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa968100e70 0x7fa968198e60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 --2- 172.21.15.5:0/3840519102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa968102050 0x7fa9681993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 >> 172.21.15.5:0/3840519102 conn(0x7fa9680fc410 msgr2=0x7fa9680fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 shutdown_connections 2023-09-23T04:43:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.088+0000 7fa96dbff700 1 -- 172.21.15.5:0/3840519102 wait complete. 2023-09-23T04:43:44.106 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T04:43:44.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1820552170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/362005242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:43:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:43 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 -- 172.21.15.5:0/234514758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04104360 msgr2=0x7fbe04106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 --2- 172.21.15.5:0/234514758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04104360 0x7fbe04106750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009990 tx=0x7fbdf8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 -- 172.21.15.5:0/234514758 shutdown_connections 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 --2- 172.21.15.5:0/234514758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04106c90 0x7fbe04109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 --2- 172.21.15.5:0/234514758 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04101a30 0x7fbe04103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 --2- 172.21.15.5:0/234514758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04104360 0x7fbe04106750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 -- 172.21.15.5:0/234514758 >> 172.21.15.5:0/234514758 conn(0x7fbe040fb3a0 msgr2=0x7fbe040fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 -- 172.21.15.5:0/234514758 shutdown_connections 2023-09-23T04:43:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.336+0000 7fbe093ff700 1 -- 172.21.15.5:0/234514758 wait complete. 2023-09-23T04:43:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 Processor -- start 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 -- start start 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 0x7fbe04199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04106c90 0x7fbe041a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041a0900 con 0x7fbe04101a30 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041a0a70 con 0x7fbe04106c90 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.337+0000 7fbe093ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041a0c30 con 0x7fbe04104360 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 0x7fbe04199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43434/0 (socket says 172.21.15.5:43434) 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe02ffd700 1 -- 172.21.15.5:0/3058813524 learned_addr learned my addr 172.21.15.5:0/3058813524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe037fe700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04106c90 0x7fbe041a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 -- 172.21.15.5:0/3058813524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04106c90 msgr2=0x7fbe041a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04106c90 0x7fbe041a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 -- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 msgr2=0x7fbe04198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 -- 172.21.15.5:0/3058813524 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf80095f0 con 0x7fbe04104360 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe02ffd700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbe027fc700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 0x7fbe04199300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800b680 tx=0x7fbdf8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.338+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf801d070 con 0x7fbe04104360 2023-09-23T04:43:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.339+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf8021410 con 0x7fbe04104360 2023-09-23T04:43:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.339+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe041a0eb0 con 0x7fbe04104360 2023-09-23T04:43:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.339+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8017650 con 0x7fbe04104360 2023-09-23T04:43:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.339+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe041a1220 con 0x7fbe04104360 2023-09-23T04:43:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.340+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe04108810 con 0x7fbe04104360 2023-09-23T04:43:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.342+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fbdf80217a0 con 0x7fbe04104360 2023-09-23T04:43:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.343+0000 7fbdebfff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 0x7fbdec0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.343+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbdf8021e40 con 0x7fbe04104360 2023-09-23T04:43:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.343+0000 7fbe02ffd700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 0x7fbdec0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.344+0000 7fbe02ffd700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 0x7fbdec0a3a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe0419a290 tx=0x7fbdf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.345+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbdf8097b50 con 0x7fbe04104360 2023-09-23T04:43:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:44.583+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbe0410c970 con 0x7fbe04104360 2023-09-23T04:43:45.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3840519102' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:45.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:45 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3840519102' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T04:43:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:45.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:45.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:45 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3840519102' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-23T04:43:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:45.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:45.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:45 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.567+0000 7fbdebfff700 1 -- 172.21.15.5:0/3058813524 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbdf809b170 con 0x7fbe04104360 2023-09-23T04:43:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 msgr2=0x7fbdec0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 0x7fbdec0a3a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe0419a290 tx=0x7fbdf4009260 comp rx=0 tx=0).stop 2023-09-23T04:43:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 msgr2=0x7fbe04199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 0x7fbe04199300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800b680 tx=0x7fbdf8009580 comp rx=0 tx=0).stop 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 shutdown_connections 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdec0a15a0 0x7fbdec0a3a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe04106c90 0x7fbe041a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04101a30 0x7fbe04198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 --2- 172.21.15.5:0/3058813524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbe04104360 0x7fbe04199300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 >> 172.21.15.5:0/3058813524 conn(0x7fbe040fb3a0 msgr2=0x7fbe041000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 shutdown_connections 2023-09-23T04:43:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.572+0000 7fbe093ff700 1 -- 172.21.15.5:0/3058813524 wait complete. 2023-09-23T04:43:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2023-09-23T04:43:45.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-23T04:43:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 -- 172.21.15.5:0/4130081800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73401000d0 msgr2=0x7f7340100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 --2- 172.21.15.5:0/4130081800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73401000d0 0x7f7340100550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f732c009990 tx=0x7f732c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 -- 172.21.15.5:0/4130081800 shutdown_connections 2023-09-23T04:43:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 --2- 172.21.15.5:0/4130081800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f7340100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 --2- 172.21.15.5:0/4130081800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73400ff770 0x7f73400ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 --2- 172.21.15.5:0/4130081800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73401000d0 0x7f7340100550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 -- 172.21.15.5:0/4130081800 >> 172.21.15.5:0/4130081800 conn(0x7f73400fb350 msgr2=0x7f73400fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 -- 172.21.15.5:0/4130081800 shutdown_connections 2023-09-23T04:43:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.809+0000 7f7346fff700 1 -- 172.21.15.5:0/4130081800 wait complete. 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.810+0000 7f7346fff700 1 Processor -- start 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.810+0000 7f7346fff700 1 -- start start 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73401000d0 0x7f7340199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f73401a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73401a0950 con 0x7f73400ff770 2023-09-23T04:43:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73401a0a90 con 0x7f7340100a90 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7346fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73401a0c20 con 0x7f73401000d0 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7344d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7344d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43466/0 (socket says 172.21.15.5:43466) 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7344d9b700 1 -- 172.21.15.5:0/3965893245 learned_addr learned my addr 172.21.15.5:0/3965893245 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f73401a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f733ffff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73401000d0 0x7f7340199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 -- 172.21.15.5:0/3965893245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73401000d0 msgr2=0x7f7340199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73401000d0 0x7f7340199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 -- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 msgr2=0x7f7340198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f734559c700 1 -- 172.21.15.5:0/3965893245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7338009c40 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.811+0000 7f7344d9b700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f734559c700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f73401a0380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f733800efb0 tx=0x7f733800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338003a00 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f733800f040 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0095f0 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338004540 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.812+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73401a11d0 con 0x7f7340100a90 2023-09-23T04:43:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.813+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734010a550 con 0x7f7340100a90 2023-09-23T04:43:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.816+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f7338003bf0 con 0x7f7340100a90 2023-09-23T04:43:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.817+0000 7f733dffb700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 0x7f73300a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.817+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f73380cc830 con 0x7f7340100a90 2023-09-23T04:43:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.817+0000 7f7344d9b700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 0x7f73300a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.818+0000 7f7344d9b700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 0x7f73300a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7334011550 tx=0x7f7334009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:45.819+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7338096140 con 0x7f7340100a90 2023-09-23T04:43:46.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.035+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f734010e930 con 0x7f7340100a90 2023-09-23T04:43:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.583+0000 7f733dffb700 1 -- 172.21.15.5:0/3965893245 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7338099760 con 0x7f7340100a90 2023-09-23T04:43:46.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.598+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 msgr2=0x7f73300a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:46.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.598+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 0x7f73300a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7334011550 tx=0x7f7334009260 comp rx=0 tx=0).stop 2023-09-23T04:43:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.598+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 msgr2=0x7f73401a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.598+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f73401a0380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f733800efb0 tx=0x7f733800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 shutdown_connections 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73300a15a0 0x7f73300a3a60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7340100a90 0x7f73401a0380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73400ff770 0x7f7340198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 --2- 172.21.15.5:0/3965893245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73401000d0 0x7f7340199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.601+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 >> 172.21.15.5:0/3965893245 conn(0x7f73400fb350 msgr2=0x7f73400fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.602+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 shutdown_connections 2023-09-23T04:43:46.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.602+0000 7f7346fff700 1 -- 172.21.15.5:0/3965893245 wait complete. 2023-09-23T04:43:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2023-09-23T04:43:46.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T04:43:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.845+0000 7f394ddff700 1 -- 172.21.15.5:0/902775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948073800 msgr2=0x7f3948073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.845+0000 7f394ddff700 1 --2- 172.21.15.5:0/902775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948073800 0x7f3948073c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f393c009990 tx=0x7f393c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.845+0000 7f394ddff700 1 -- 172.21.15.5:0/902775567 shutdown_connections 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.845+0000 7f394ddff700 1 --2- 172.21.15.5:0/902775567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f394810b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.846+0000 7f394ddff700 1 --2- 172.21.15.5:0/902775567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948074de0 0x7f3948073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.846+0000 7f394ddff700 1 --2- 172.21.15.5:0/902775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948073800 0x7f3948073c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.846+0000 7f394ddff700 1 -- 172.21.15.5:0/902775567 >> 172.21.15.5:0/902775567 conn(0x7f39480fc470 msgr2=0x7f39480fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.846+0000 7f394ddff700 1 -- 172.21.15.5:0/902775567 shutdown_connections 2023-09-23T04:43:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.846+0000 7f394ddff700 1 -- 172.21.15.5:0/902775567 wait complete. 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 Processor -- start 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 -- start start 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948074de0 0x7f3948199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f39481a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481a0960 con 0x7f3948074de0 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481a0aa0 con 0x7f3948108df0 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f394ddff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481a0c30 con 0x7f3948073800 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.847+0000 7f39477fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45428/0 (socket says 172.21.15.5:45428) 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f3947fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f39481a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f3946ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948074de0 0x7f3948199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 -- 172.21.15.5:0/2995336391 learned_addr learned my addr 172.21.15.5:0/2995336391 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 -- 172.21.15.5:0/2995336391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 msgr2=0x7f39481a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f39481a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 -- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948074de0 msgr2=0x7f3948199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948074de0 0x7f3948199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 -- 172.21.15.5:0/2995336391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009520 con 0x7f3948073800 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f3947fff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f39481a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f39477fe700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f393800eb30 tx=0x7f393800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009880 con 0x7f3948073800 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.848+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39380044e0 con 0x7f3948073800 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.849+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938003d80 con 0x7f3948073800 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.849+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3940009c40 con 0x7f3948073800 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.849+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c0095f0 con 0x7f3948073800 2023-09-23T04:43:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.850+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39481074f0 con 0x7f3948073800 2023-09-23T04:43:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.852+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f3938003ee0 con 0x7f3948073800 2023-09-23T04:43:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.853+0000 7f3944ff9700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 0x7f39300a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.853+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f3938014070 con 0x7f3948073800 2023-09-23T04:43:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.854+0000 7f3946ffd700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 0x7f39300a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.854+0000 7f3946ffd700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 0x7f39300a3a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f393c00b2e0 tx=0x7f393c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:46.857+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3938096910 con 0x7f3948073800 2023-09-23T04:43:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3058813524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T04:43:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T04:43:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: pgmap v261: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3965893245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:46 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3058813524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: pgmap v261: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3965893245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:46.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:46 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3058813524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: osdmap e80: 12 total, 12 up, 12 in 2023-09-23T04:43:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: pgmap v261: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:46.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3965893245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:46.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:46 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-23T04:43:47.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.146+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f394810eb50 con 0x7f3948073800 2023-09-23T04:43:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.593+0000 7f3944ff9700 1 -- 172.21.15.5:0/2995336391 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3938004190 con 0x7f3948073800 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.602+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 msgr2=0x7f39300a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.602+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 0x7f39300a3a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f393c00b2e0 tx=0x7f393c005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.602+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 msgr2=0x7f3948198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.602+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f393800eb30 tx=0x7f393800ee40 comp rx=0 tx=0).stop 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 shutdown_connections 2023-09-23T04:43:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f39300a15a0 0x7f39300a3a60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3948108df0 0x7f39481a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3948073800 0x7f3948198de0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f393800eb30 tx=0x7f393800ee40 comp rx=0 tx=0).stop 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 --2- 172.21.15.5:0/2995336391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3948074de0 0x7f3948199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.603+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 >> 172.21.15.5:0/2995336391 conn(0x7f39480fc470 msgr2=0x7f39480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.604+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 shutdown_connections 2023-09-23T04:43:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.604+0000 7f394ddff700 1 -- 172.21.15.5:0/2995336391 wait complete. 2023-09-23T04:43:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2023-09-23T04:43:47.629 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-23T04:43:47.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-23T04:43:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 -- 172.21.15.5:0/2534840090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100f10 msgr2=0x7f3b50101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 --2- 172.21.15.5:0/2534840090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100f10 0x7f3b50101330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009990 tx=0x7f3b40009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 -- 172.21.15.5:0/2534840090 shutdown_connections 2023-09-23T04:43:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 --2- 172.21.15.5:0/2534840090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 0x7f3b50109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 --2- 172.21.15.5:0/2534840090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50100f10 0x7f3b50101330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 --2- 172.21.15.5:0/2534840090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50102060 0x7f3b501024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 -- 172.21.15.5:0/2534840090 >> 172.21.15.5:0/2534840090 conn(0x7f3b500fc480 msgr2=0x7f3b500fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 -- 172.21.15.5:0/2534840090 shutdown_connections 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.855+0000 7f3b569ff700 1 -- 172.21.15.5:0/2534840090 wait complete. 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 Processor -- start 2023-09-23T04:43:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 -- start start 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 0x7f3b501971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 0x7f3b5019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5019e7b0 con 0x7f3b50102060 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5019e920 con 0x7f3b50102a20 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.857+0000 7f3b569ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5019eae0 con 0x7f3b50100f10 2023-09-23T04:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 0x7f3b501971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45448/0 (socket says 172.21.15.5:45448) 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 0x7f3b501971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43494/0 (socket says 172.21.15.5:43494) 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 -- 172.21.15.5:0/23547176 learned_addr learned my addr 172.21.15.5:0/23547176 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b54f9c700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 0x7f3b5019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 -- 172.21.15.5:0/23547176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 msgr2=0x7f3b5019e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 0x7f3b5019e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 -- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 msgr2=0x7f3b501971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 0x7f3b501971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 -- 172.21.15.5:0/23547176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b400095f0 con 0x7f3b50100f10 2023-09-23T04:43:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.858+0000 7f3b4ffff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b40004740 tx=0x7f3b40009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.859+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40021070 con 0x7f3b50100f10 2023-09-23T04:43:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.859+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4000be90 con 0x7f3b50100f10 2023-09-23T04:43:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.859+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40026470 con 0x7f3b50100f10 2023-09-23T04:43:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.859+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b50073050 con 0x7f3b50100f10 2023-09-23T04:43:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.859+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b50073450 con 0x7f3b50100f10 2023-09-23T04:43:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.860+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50108490 con 0x7f3b50100f10 2023-09-23T04:43:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.861+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f3b40004830 con 0x7f3b50100f10 2023-09-23T04:43:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.863+0000 7f3b4d7fa700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 0x7f3b380a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.863+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b400d2700 con 0x7f3b50100f10 2023-09-23T04:43:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.863+0000 7f3b4f7fe700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 0x7f3b380a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.864+0000 7f3b4f7fe700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 0x7f3b380a3ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009c60 tx=0x7f3b44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:47.866+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3b4005fec0 con 0x7f3b50100f10 2023-09-23T04:43:47.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:47.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-23T04:43:47.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T04:43:47.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2995336391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:47 smithi005 ceph-mon[97638]: daemon mds.g assigned to filesystem cephfs as rank 0 2023-09-23T04:43:47.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:47.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-23T04:43:47.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T04:43:47.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2995336391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:47 smithi118 ceph-mon[134297]: daemon mds.g assigned to filesystem cephfs as rank 0 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: osdmap e81: 12 total, 12 up, 12 in 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2995336391' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-23T04:43:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:47 smithi069 ceph-mon[133931]: daemon mds.g assigned to filesystem cephfs as rank 0 2023-09-23T04:43:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.098+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3b5010c760 con 0x7f3b50100f10 2023-09-23T04:43:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.574+0000 7f3b4d7fa700 1 -- 172.21.15.5:0/23547176 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3b5010c760 con 0x7f3b50100f10 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 msgr2=0x7f3b380a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 0x7f3b380a3ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009c60 tx=0x7f3b44009260 comp rx=0 tx=0).stop 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 msgr2=0x7f3b50196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b40004740 tx=0x7f3b40009580 comp rx=0 tx=0).stop 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 shutdown_connections 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3b380a1620 0x7f3b380a3ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3b50102a20 0x7f3b5019e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3b50100f10 0x7f3b50196c70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 --2- 172.21.15.5:0/23547176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b50102060 0x7f3b501971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 >> 172.21.15.5:0/23547176 conn(0x7f3b500fc480 msgr2=0x7f3b501055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 shutdown_connections 2023-09-23T04:43:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.577+0000 7f3b569ff700 1 -- 172.21.15.5:0/23547176 wait complete. 2023-09-23T04:43:48.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 -- 172.21.15.5:0/2528541939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4101fa0 msgr2=0x7ff8c4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/2528541939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4101fa0 0x7ff8c4102420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098f0 tx=0x7ff8b4009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 -- 172.21.15.5:0/2528541939 shutdown_connections 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/2528541939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 0x7ff8c4109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/2528541939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4100e50 0x7ff8c4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/2528541939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4101fa0 0x7ff8c4102420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 -- 172.21.15.5:0/2528541939 >> 172.21.15.5:0/2528541939 conn(0x7ff8c40fc3e0 msgr2=0x7ff8c40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:48.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 -- 172.21.15.5:0/2528541939 shutdown_connections 2023-09-23T04:43:48.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.816+0000 7ff8cbfff700 1 -- 172.21.15.5:0/2528541939 wait complete. 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 Processor -- start 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 -- start start 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4101fa0 0x7ff8c41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 0x7ff8c419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419e6e0 con 0x7ff8c4102960 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419e820 con 0x7ff8c4100e50 2023-09-23T04:43:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8cbfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419e9b0 con 0x7ff8c4101fa0 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8c9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.817+0000 7ff8c9d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41848/0 (socket says 172.21.15.5:41848) 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 -- 172.21.15.5:0/4274648687 learned_addr learned my addr 172.21.15.5:0/4274648687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8ca59c700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 0x7ff8c419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c959a700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4101fa0 0x7ff8c41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 -- 172.21.15.5:0/4274648687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4101fa0 msgr2=0x7ff8c41970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4101fa0 0x7ff8c41970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 -- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 msgr2=0x7ff8c419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 0x7ff8c419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 -- 172.21.15.5:0/4274648687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009520 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8c9d9b700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e9c0 tx=0x7ff8c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004a30 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c000cc80 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c419ec90 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.818+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00054e0 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.819+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c419f1a0 con 0x7ff8c4100e50 2023-09-23T04:43:48.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.820+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c4108430 con 0x7ff8c4100e50 2023-09-23T04:43:48.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.822+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7ff8c0004b90 con 0x7ff8c4100e50 2023-09-23T04:43:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.823+0000 7ff8baffd700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 0x7ff8b00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.823+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7ff8c0014070 con 0x7ff8c4100e50 2023-09-23T04:43:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.823+0000 7ff8c959a700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 0x7ff8b00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.824+0000 7ff8c959a700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 0x7ff8b00a3a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005e90 tx=0x7ff8b4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:48.825+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff8c0096590 con 0x7ff8c4100e50 2023-09-23T04:43:48.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: daemon mds.g is now active in filesystem cephfs as rank 0 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: pgmap v264: 97 pgs: 4 active+clean, 22 creating+peering, 71 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/23547176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:48 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: daemon mds.g is now active in filesystem cephfs as rank 0 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: pgmap v264: 97 pgs: 4 active+clean, 22 creating+peering, 71 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/23547176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:48.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:48 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:48.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: osdmap e82: 12 total, 12 up, 12 in 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:creating} 11 up:standby 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: daemon mds.g is now active in filesystem cephfs as rank 0 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: pgmap v264: 97 pgs: 4 active+clean, 22 creating+peering, 71 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/23547176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:48.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:48 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-23T04:43:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:49.036+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff8c402d230 con 0x7ff8c4100e50 2023-09-23T04:43:49.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:active 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4274648687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:49.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:49 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:active 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:49.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:49.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4274648687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:49.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:49 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: osdmap e83: 12 total, 12 up, 12 in 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] up:active 2023-09-23T04:43:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4274648687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:49.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:49 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-23T04:43:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.711+0000 7ff8baffd700 1 -- 172.21.15.5:0/4274648687 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff8c0094e60 con 0x7ff8c4100e50 2023-09-23T04:43:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 msgr2=0x7ff8b00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 0x7ff8b00a3a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005e90 tx=0x7ff8b4019040 comp rx=0 tx=0).stop 2023-09-23T04:43:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 msgr2=0x7ff8c4196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000e9c0 tx=0x7ff8c000ecd0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 shutdown_connections 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff8b00a15a0 0x7ff8b00a3a60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8c4102960 0x7ff8c419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8c4100e50 0x7ff8c4196b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 --2- 172.21.15.5:0/4274648687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8c4101fa0 0x7ff8c41970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.717+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 >> 172.21.15.5:0/4274648687 conn(0x7ff8c40fc3e0 msgr2=0x7ff8c4105570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.718+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 shutdown_connections 2023-09-23T04:43:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.718+0000 7ff8cbfff700 1 -- 172.21.15.5:0/4274648687 wait complete. 2023-09-23T04:43:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2023-09-23T04:43:50.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 -- 172.21.15.5:0/3204795565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 msgr2=0x7f1d78102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 --2- 172.21.15.5:0/3204795565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78102510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009990 tx=0x7f1d64009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 -- 172.21.15.5:0/3204795565 shutdown_connections 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 --2- 172.21.15.5:0/3204795565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 --2- 172.21.15.5:0/3204795565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d781012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 --2- 172.21.15.5:0/3204795565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78102510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 -- 172.21.15.5:0/3204795565 >> 172.21.15.5:0/3204795565 conn(0x7f1d780fc470 msgr2=0x7f1d780fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 -- 172.21.15.5:0/3204795565 shutdown_connections 2023-09-23T04:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.949+0000 7f1d7edff700 1 -- 172.21.15.5:0/3204795565 wait complete. 2023-09-23T04:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.950+0000 7f1d7edff700 1 Processor -- start 2023-09-23T04:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 -- start start 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78073ef0 con 0x7f1d78102a50 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78074060 con 0x7f1d78102090 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.951+0000 7f1d7edff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d781a0c20 con 0x7f1d78100eb0 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57418/0 (socket says 172.21.15.5:57418) 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 -- 172.21.15.5:0/320791128 learned_addr learned my addr 172.21.15.5:0/320791128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:50.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d77fff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 -- 172.21.15.5:0/320791128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 msgr2=0x7f1d78073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 -- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 msgr2=0x7f1d78073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 -- 172.21.15.5:0/320791128 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d640095f0 con 0x7f1d78100eb0 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7d39c700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:50.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7cb9b700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00e970 tx=0x7f1d6c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c009790 con 0x7f1d78100eb0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d781a0f00 con 0x7f1d78100eb0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6c00c7e0 con 0x7f1d78100eb0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.952+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c003c70 con 0x7f1d78100eb0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.953+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d781a1330 con 0x7f1d78100eb0 2023-09-23T04:43:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.953+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d781084f0 con 0x7f1d78100eb0 2023-09-23T04:43:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.956+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f1d6c00cc50 con 0x7f1d78100eb0 2023-09-23T04:43:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.957+0000 7f1d75ffb700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 0x7f1d680a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.957+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f1d6c014070 con 0x7f1d78100eb0 2023-09-23T04:43:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.957+0000 7f1d77fff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 0x7f1d680a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.958+0000 7f1d77fff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 0x7f1d680a3ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b680 tx=0x7f1d64005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:50.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:50.959+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1d6c096780 con 0x7f1d78100eb0 2023-09-23T04:43:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:50 smithi069 ceph-mon[133931]: pgmap v266: 97 pgs: 50 active+clean, 22 creating+peering, 25 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2023-09-23T04:43:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:50 smithi069 ceph-mon[133931]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T04:43:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:50 smithi005 ceph-mon[97638]: pgmap v266: 97 pgs: 50 active+clean, 22 creating+peering, 25 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2023-09-23T04:43:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:50 smithi005 ceph-mon[97638]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T04:43:51.155 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:43:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T04:43:50.706+0000 7fd91f0a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T04:43:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:50 smithi118 ceph-mon[134297]: pgmap v266: 97 pgs: 50 active+clean, 22 creating+peering, 25 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2023-09-23T04:43:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:50 smithi118 ceph-mon[134297]: osdmap e84: 12 total, 12 up, 12 in 2023-09-23T04:43:51.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.167+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1d7802d230 con 0x7f1d78100eb0 2023-09-23T04:43:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.712+0000 7f1d75ffb700 1 -- 172.21.15.5:0/320791128 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1d6c003dd0 con 0x7f1d78100eb0 2023-09-23T04:43:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.719+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 msgr2=0x7f1d680a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.719+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 0x7f1d680a3ae0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b680 tx=0x7f1d64005fd0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.719+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 msgr2=0x7f1d78074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.719+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00e970 tx=0x7f1d6c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 shutdown_connections 2023-09-23T04:43:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1d680a1620 0x7f1d680a3ae0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102a50 0x7f1d78073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1d78100eb0 0x7f1d78074bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 --2- 172.21.15.5:0/320791128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d78102090 0x7f1d78073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.728+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 >> 172.21.15.5:0/320791128 conn(0x7f1d780fc470 msgr2=0x7f1d78105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.731+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 shutdown_connections 2023-09-23T04:43:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.733+0000 7f1d7edff700 1 -- 172.21.15.5:0/320791128 wait complete. 2023-09-23T04:43:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2023-09-23T04:43:51.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-23T04:43:51.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 -- 172.21.15.5:0/1324904182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144102090 msgr2=0x7fb144102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:51.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 --2- 172.21.15.5:0/1324904182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144102090 0x7fb144102510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 -- 172.21.15.5:0/1324904182 shutdown_connections 2023-09-23T04:43:51.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 --2- 172.21.15.5:0/1324904182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102a50 0x7fb144109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 --2- 172.21.15.5:0/1324904182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144100eb0 0x7fb1441012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 --2- 172.21.15.5:0/1324904182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144102090 0x7fb144102510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 -- 172.21.15.5:0/1324904182 >> 172.21.15.5:0/1324904182 conn(0x7fb1440fc470 msgr2=0x7fb1440fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:51.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 -- 172.21.15.5:0/1324904182 shutdown_connections 2023-09-23T04:43:51.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.964+0000 7fb149bff700 1 -- 172.21.15.5:0/1324904182 wait complete. 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.965+0000 7fb149bff700 1 Processor -- start 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.965+0000 7fb149bff700 1 -- start start 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.965+0000 7fb149bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.965+0000 7fb149bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 0x7fb144197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb149bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144102a50 0x7fb14419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb149bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419e770 con 0x7fb144100eb0 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb149bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419e8e0 con 0x7fb144102090 2023-09-23T04:43:51.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb149bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14419eaa0 con 0x7fb144102a50 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52496/0 (socket says 172.21.15.5:52496) 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 -- 172.21.15.5:0/3306330961 learned_addr learned my addr 172.21.15.5:0/3306330961 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb142ffd700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 0x7fb144197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb143fff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144102a50 0x7fb14419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 -- 172.21.15.5:0/3306330961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144102a50 msgr2=0x7fb14419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144102a50 0x7fb14419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 -- 172.21.15.5:0/3306330961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 msgr2=0x7fb144197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 0x7fb144197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 -- 172.21.15.5:0/3306330961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380095f0 con 0x7fb144100eb0 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb142ffd700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 0x7fb144197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:43:51.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.966+0000 7fb1437fe700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e970 tx=0x7fb13400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.967+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134009790 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.967+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1340044e0 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.967+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134003e30 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.967+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14419ed80 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.967+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14419f180 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.968+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144107eb0 con 0x7fb144100eb0 2023-09-23T04:43:51.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.970+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fb134003f90 con 0x7fb144100eb0 2023-09-23T04:43:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.971+0000 7fb140ff9700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 0x7fb12c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.971+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb134014070 con 0x7fb144100eb0 2023-09-23T04:43:51.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.972+0000 7fb142ffd700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 0x7fb12c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:51.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.972+0000 7fb142ffd700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 0x7fb12c0a3a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb138005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:51.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:51.974+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb1340968c0 con 0x7fb144100eb0 2023-09-23T04:43:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:51 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T04:43:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:51 smithi069 ceph-mon[133931]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T04:43:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:51 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/320791128' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:51 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:52.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:51 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T04:43:52.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:51 smithi005 ceph-mon[97638]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T04:43:52.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:51 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/320791128' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:52.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:51 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:52.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:51 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-23T04:43:52.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:51 smithi118 ceph-mon[134297]: osdmap e85: 12 total, 12 up, 12 in 2023-09-23T04:43:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:51 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/320791128' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:51 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-23T04:43:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.236+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb14410ca20 con 0x7fb144100eb0 2023-09-23T04:43:52.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.720+0000 7fb140ff9700 1 -- 172.21.15.5:0/3306330961 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb14410ca20 con 0x7fb144100eb0 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 msgr2=0x7fb12c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 0x7fb12c0a3a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb138005fd0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 msgr2=0x7fb144196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e970 tx=0x7fb13400ed30 comp rx=0 tx=0).stop 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 shutdown_connections 2023-09-23T04:43:52.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb12c0a15a0 0x7fb12c0a3a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb144102a50 0x7fb14419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb144100eb0 0x7fb144196c30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 --2- 172.21.15.5:0/3306330961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb144102090 0x7fb144197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 >> 172.21.15.5:0/3306330961 conn(0x7fb1440fc470 msgr2=0x7fb1440fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 shutdown_connections 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.723+0000 7fb149bff700 1 -- 172.21.15.5:0/3306330961 wait complete. 2023-09-23T04:43:52.724 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2023-09-23T04:43:52.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 -- 172.21.15.5:0/2865178473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0074e60 msgr2=0x7f2bd00732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/2865178473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0074e60 0x7f2bd00732b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009990 tx=0x7f2bc4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 -- 172.21.15.5:0/2865178473 shutdown_connections 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/2865178473 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0108da0 0x7f2bd010b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/2865178473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0074e60 0x7f2bd00732b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/2865178473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 0x7f2bd0073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 -- 172.21.15.5:0/2865178473 >> 172.21.15.5:0/2865178473 conn(0x7f2bd00fc450 msgr2=0x7f2bd00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 -- 172.21.15.5:0/2865178473 shutdown_connections 2023-09-23T04:43:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.952+0000 7f2bd6dff700 1 -- 172.21.15.5:0/2865178473 wait complete. 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.953+0000 7f2bd6dff700 1 Processor -- start 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 -- start start 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 0x7f2bd0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0074e60 0x7f2bd0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01a09a0 con 0x7f2bd0108da0 2023-09-23T04:43:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01a0ae0 con 0x7f2bd0073880 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd6dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01a0c70 con 0x7f2bd0074e60 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd539c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd539c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52514/0 (socket says 172.21.15.5:52514) 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd539c700 1 -- 172.21.15.5:0/4135580558 learned_addr learned my addr 172.21.15.5:0/4135580558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bd4b9b700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 0x7f2bd0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.954+0000 7f2bcffff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0074e60 0x7f2bd0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 -- 172.21.15.5:0/4135580558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0074e60 msgr2=0x7f2bd0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0074e60 0x7f2bd0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 -- 172.21.15.5:0/4135580558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 msgr2=0x7f2bd0198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 0x7f2bd0198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 -- 172.21.15.5:0/4135580558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40095f0 con 0x7f2bd0108da0 2023-09-23T04:43:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd539c700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007ee0 tx=0x7f2bc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8017070 con 0x7f2bd0108da0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc800f040 con 0x7f2bd0108da0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd01a0f50 con 0x7f2bd0108da0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8005580 con 0x7f2bd0108da0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.955+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd01a13d0 con 0x7f2bd0108da0 2023-09-23T04:43:52.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.956+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0107450 con 0x7f2bd0108da0 2023-09-23T04:43:52.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.958+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f2bc800db20 con 0x7f2bd0108da0 2023-09-23T04:43:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.960+0000 7f2bcdffb700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 0x7f2bc00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.960+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2bc80cc7a0 con 0x7f2bd0108da0 2023-09-23T04:43:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.960+0000 7f2bd4b9b700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 0x7f2bc00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.961+0000 7f2bd4b9b700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 0x7f2bc00a3ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b5c0 tx=0x7f2bc4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:52.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:52.964+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2bc8095b40 con 0x7f2bd0108da0 2023-09-23T04:43:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:52 smithi069 ceph-mon[133931]: pgmap v269: 129 pgs: 81 active+clean, 26 creating+peering, 22 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2023-09-23T04:43:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:52 smithi069 ceph-mon[133931]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:52 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-23T04:43:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:52 smithi069 ceph-mon[133931]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T04:43:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:52 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-23T04:43:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:52 smithi005 ceph-mon[97638]: pgmap v269: 129 pgs: 81 active+clean, 26 creating+peering, 22 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2023-09-23T04:43:53.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:52 smithi005 ceph-mon[97638]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:53.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:52 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-23T04:43:53.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:52 smithi005 ceph-mon[97638]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T04:43:53.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:52 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-23T04:43:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:52 smithi118 ceph-mon[134297]: pgmap v269: 129 pgs: 81 active+clean, 26 creating+peering, 22 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 10 op/s 2023-09-23T04:43:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:52 smithi118 ceph-mon[134297]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-23T04:43:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:52 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-23T04:43:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:52 smithi118 ceph-mon[134297]: osdmap e86: 12 total, 12 up, 12 in 2023-09-23T04:43:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:52 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-23T04:43:53.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.224+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd010e920 con 0x7f2bd0108da0 2023-09-23T04:43:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.225+0000 7f2bcdffb700 1 -- 172.21.15.5:0/4135580558 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11240 (secure 0 0 0) 0x7f2bc8099160 con 0x7f2bd0108da0 2023-09-23T04:43:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:53.225 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:43:52.237323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 msgr2=0x7f2bc00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 0x7f2bc00a3ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b5c0 tx=0x7f2bc4005fd0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 msgr2=0x7f2bd01a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007ee0 tx=0x7f2bc800d3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 shutdown_connections 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2bc00a1620 0x7f2bc00a3ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd0108da0 0x7f2bd01a03d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bd0073880 0x7f2bd0198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 --2- 172.21.15.5:0/4135580558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2bd0074e60 0x7f2bd0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 >> 172.21.15.5:0/4135580558 conn(0x7f2bd00fc450 msgr2=0x7f2bd00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 shutdown_connections 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.227+0000 7f2bd6dff700 1 -- 172.21.15.5:0/4135580558 wait complete. 2023-09-23T04:43:53.228 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2023-09-23T04:43:53.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:43:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 -- 172.21.15.5:0/3930835026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c106bb0 msgr2=0x7fea6c109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 --2- 172.21.15.5:0/3930835026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c106bb0 0x7fea6c109020 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b0a0 tx=0x7fea6400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 -- 172.21.15.5:0/3930835026 shutdown_connections 2023-09-23T04:43:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 --2- 172.21.15.5:0/3930835026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c106bb0 0x7fea6c109020 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 --2- 172.21.15.5:0/3930835026 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c101950 0x7fea6c103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 --2- 172.21.15.5:0/3930835026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c104280 0x7fea6c106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 -- 172.21.15.5:0/3930835026 >> 172.21.15.5:0/3930835026 conn(0x7fea6c0fb360 msgr2=0x7fea6c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 -- 172.21.15.5:0/3930835026 shutdown_connections 2023-09-23T04:43:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.448+0000 7fea741ff700 1 -- 172.21.15.5:0/3930835026 wait complete. 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.449+0000 7fea741ff700 1 Processor -- start 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.449+0000 7fea741ff700 1 -- start start 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c104280 0x7fea6c0ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c106bb0 0x7fea6c0ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1005e0 con 0x7fea6c101950 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c100750 con 0x7fea6c106bb0 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea741ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1a0d20 con 0x7fea6c104280 2023-09-23T04:43:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52526/0 (socket says 172.21.15.5:52526) 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 -- 172.21.15.5:0/630996251 learned_addr learned my addr 172.21.15.5:0/630996251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea7179a700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c104280 0x7fea6c0ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea7279c700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c106bb0 0x7fea6c0ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 -- 172.21.15.5:0/630996251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c104280 msgr2=0x7fea6c0ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c104280 0x7fea6c0ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 -- 172.21.15.5:0/630996251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c106bb0 msgr2=0x7fea6c0ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c106bb0 0x7fea6c0ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.450+0000 7fea71f9b700 1 -- 172.21.15.5:0/630996251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea64009d00 con 0x7fea6c101950 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea71f9b700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e9a0 tx=0x7fea6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68009790 con 0x7fea6c101950 2023-09-23T04:43:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea68010430 con 0x7fea6c101950 2023-09-23T04:43:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1a0fa0 con 0x7fea6c101950 2023-09-23T04:43:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68018680 con 0x7fea6c101950 2023-09-23T04:43:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.451+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c1a13d0 con 0x7fea6c101950 2023-09-23T04:43:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.452+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c0fcee0 con 0x7fea6c101950 2023-09-23T04:43:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.454+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fea680105a0 con 0x7fea6c101950 2023-09-23T04:43:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.455+0000 7fea62ffd700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 0x7fea580a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.456+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fea68014070 con 0x7fea6c101950 2023-09-23T04:43:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.456+0000 7fea7179a700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 0x7fea580a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.456+0000 7fea7179a700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 0x7fea580a3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009c60 tx=0x7fea5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.457+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fea68096870 con 0x7fea6c101950 2023-09-23T04:43:53.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.672+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea6c10c970 con 0x7fea6c101950 2023-09-23T04:43:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.673+0000 7fea62ffd700 1 -- 172.21.15.5:0/630996251 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fea68099e90 con 0x7fea6c101950 2023-09-23T04:43:53.674 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:53.674 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 msgr2=0x7fea580a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 0x7fea580a3a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009c60 tx=0x7fea5c009260 comp rx=0 tx=0).stop 2023-09-23T04:43:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 msgr2=0x7fea6c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e9a0 tx=0x7fea6800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 shutdown_connections 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fea580a15a0 0x7fea580a3a60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea6c106bb0 0x7fea6c0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea6c101950 0x7fea6c1012c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 --2- 172.21.15.5:0/630996251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fea6c104280 0x7fea6c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.676+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 >> 172.21.15.5:0/630996251 conn(0x7fea6c0fb360 msgr2=0x7fea6c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.677+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 shutdown_connections 2023-09-23T04:43:53.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.677+0000 7fea741ff700 1 -- 172.21.15.5:0/630996251 wait complete. 2023-09-23T04:43:53.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:43:53.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 -- 172.21.15.5:0/1632306604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204102a30 msgr2=0x7f6204109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 --2- 172.21.15.5:0/1632306604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204102a30 0x7f6204109350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b0a0 tx=0x7f61fc00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 -- 172.21.15.5:0/1632306604 shutdown_connections 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 --2- 172.21.15.5:0/1632306604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204102a30 0x7f6204109350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 --2- 172.21.15.5:0/1632306604 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204100e90 0x7f62041012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 --2- 172.21.15.5:0/1632306604 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f62041024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 -- 172.21.15.5:0/1632306604 >> 172.21.15.5:0/1632306604 conn(0x7f62040fc450 msgr2=0x7f62040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 -- 172.21.15.5:0/1632306604 shutdown_connections 2023-09-23T04:43:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.903+0000 7f620c5ff700 1 -- 172.21.15.5:0/1632306604 wait complete. 2023-09-23T04:43:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.904+0000 7f620c5ff700 1 Processor -- start 2023-09-23T04:43:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.904+0000 7f620c5ff700 1 -- start start 2023-09-23T04:43:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f6204199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204102a30 0x7f62041a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62041a0a80 con 0x7f6204100e90 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62041a0bc0 con 0x7f6204102a30 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620c5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62041a0d50 con 0x7f6204102070 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52544/0 (socket says 172.21.15.5:52544) 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620a39b700 1 -- 172.21.15.5:0/268096092 learned_addr learned my addr 172.21.15.5:0/268096092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f6209b9a700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f6204199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620ab9c700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204102a30 0x7f62041a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.905+0000 7f620a39b700 1 -- 172.21.15.5:0/268096092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 msgr2=0x7f6204199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620a39b700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f6204199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620a39b700 1 -- 172.21.15.5:0/268096092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204102a30 msgr2=0x7f62041a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620a39b700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204102a30 0x7f62041a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620a39b700 1 -- 172.21.15.5:0/268096092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc009d00 con 0x7f6204100e90 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f6209b9a700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f6204199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620a39b700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f620000e970 tx=0x7f620000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200009790 con 0x7f6204100e90 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620000c7e0 con 0x7f6204100e90 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62041a0fd0 con 0x7f6204100e90 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200003c70 con 0x7f6204100e90 2023-09-23T04:43:53.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.906+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62041a14a0 con 0x7f6204100e90 2023-09-23T04:43:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.907+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620410c750 con 0x7f6204100e90 2023-09-23T04:43:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.909+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f620000cc50 con 0x7f6204100e90 2023-09-23T04:43:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.911+0000 7f61fb7fe700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 0x7f61f00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.911+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6200014070 con 0x7f6204100e90 2023-09-23T04:43:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.911+0000 7f6209b9a700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 0x7f61f00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.912+0000 7f6209b9a700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 0x7f61f00a3ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6204100bb0 tx=0x7f61f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:53.913+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6200096750 con 0x7f6204100e90 2023-09-23T04:43:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:53 smithi069 ceph-mon[133931]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T04:43:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:53 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-23T04:43:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:53 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:53 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4135580558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:53 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/630996251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.125+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62041a16f0 con 0x7f6204100e90 2023-09-23T04:43:54.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.126+0000 7f61fb7fe700 1 -- 172.21.15.5:0/268096092 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f6200003dd0 con 0x7f6204100e90 2023-09-23T04:43:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:54.127 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 msgr2=0x7f61f00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 0x7f61f00a3ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6204100bb0 tx=0x7f61f4009260 comp rx=0 tx=0).stop 2023-09-23T04:43:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 msgr2=0x7f6204198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f620000e970 tx=0x7f620000ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 shutdown_connections 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61f00a1620 0x7f61f00a3ae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6204102a30 0x7f62041a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6204100e90 0x7f6204198f00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 --2- 172.21.15.5:0/268096092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6204102070 0x7f6204199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 >> 172.21.15.5:0/268096092 conn(0x7f62040fc450 msgr2=0x7f62040fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 shutdown_connections 2023-09-23T04:43:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.129+0000 7f620c5ff700 1 -- 172.21.15.5:0/268096092 wait complete. 2023-09-23T04:43:54.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:53 smithi005 ceph-mon[97638]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T04:43:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:53 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-23T04:43:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:53 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:53 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4135580558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:53 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/630996251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:43:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:53 smithi118 ceph-mon[134297]: osdmap e87: 12 total, 12 up, 12 in 2023-09-23T04:43:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:53 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3306330961' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-23T04:43:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:53 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:43:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:53 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4135580558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:53 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/630996251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 -- 172.21.15.5:0/1853868405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c102080 msgr2=0x7fdb0c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 --2- 172.21.15.5:0/1853868405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c102080 0x7fdb0c102500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009990 tx=0x7fdaf8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 -- 172.21.15.5:0/1853868405 shutdown_connections 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 --2- 172.21.15.5:0/1853868405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 --2- 172.21.15.5:0/1853868405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 --2- 172.21.15.5:0/1853868405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c102080 0x7fdb0c102500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 -- 172.21.15.5:0/1853868405 >> 172.21.15.5:0/1853868405 conn(0x7fdb0c0fc480 msgr2=0x7fdb0c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 -- 172.21.15.5:0/1853868405 shutdown_connections 2023-09-23T04:43:54.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.363+0000 7fdb12bff700 1 -- 172.21.15.5:0/1853868405 wait complete. 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.365+0000 7fdb12bff700 1 Processor -- start 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.365+0000 7fdb12bff700 1 -- start start 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.365+0000 7fdb12bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb12bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c102080 0x7fdb0c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb12bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb12bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19e780 con 0x7fdb0c100ea0 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb12bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19e8f0 con 0x7fdb0c102a40 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb12bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c19eab0 con 0x7fdb0c102080 2023-09-23T04:43:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52576/0 (socket says 172.21.15.5:52576) 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 -- 172.21.15.5:0/382209340 learned_addr learned my addr 172.21.15.5:0/382209340 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1119c700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb0bfff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c102080 0x7fdb0c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 -- 172.21.15.5:0/382209340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c102080 msgr2=0x7fdb0c197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c102080 0x7fdb0c197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 -- 172.21.15.5:0/382209340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 msgr2=0x7fdb0c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1099b700 1 -- 172.21.15.5:0/382209340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf80095f0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.366+0000 7fdb1119c700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb1099b700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e970 tx=0x7fdb0000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:54.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00009790 con 0x7fdb0c100ea0 2023-09-23T04:43:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0000c7e0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c19ed90 con 0x7fdb0c100ea0 2023-09-23T04:43:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00003c70 con 0x7fdb0c100ea0 2023-09-23T04:43:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.367+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c19f1e0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.368+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c107ea0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.370+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fdb0000cc50 con 0x7fdb0c100ea0 2023-09-23T04:43:54.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.371+0000 7fdb09ffb700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 0x7fdafc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.372+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdb00014070 con 0x7fdb0c100ea0 2023-09-23T04:43:54.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.372+0000 7fdb0bfff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 0x7fdafc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.372+0000 7fdb0bfff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 0x7fdafc0a3a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b2e0 tx=0x7fdaf8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:54.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.375+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fdb000966a0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.636+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb0c10c760 con 0x7fdb0c100ea0 2023-09-23T04:43:54.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.636+0000 7fdb09ffb700 1 -- 172.21.15.5:0/382209340 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11240 (secure 0 0 0) 0x7fdb00003dd0 con 0x7fdb0c100ea0 2023-09-23T04:43:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:54.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24529,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:43:52.237323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24577},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 msgr2=0x7fdafc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 0x7fdafc0a3a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b2e0 tx=0x7fdaf8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 msgr2=0x7fdb0c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e970 tx=0x7fdb0000ec80 comp rx=0 tx=0).stop 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 shutdown_connections 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fdafc0a15a0 0x7fdafc0a3a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb0c102a40 0x7fdb0c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb0c100ea0 0x7fdb0c196c40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 --2- 172.21.15.5:0/382209340 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdb0c102080 0x7fdb0c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 >> 172.21.15.5:0/382209340 conn(0x7fdb0c0fc480 msgr2=0x7fdb0c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 shutdown_connections 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.639+0000 7fdb12bff700 1 -- 172.21.15.5:0/382209340 wait complete. 2023-09-23T04:43:54.640 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2023-09-23T04:43:54.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:43:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/1976519118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102080 msgr2=0x7f0bf8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/1976519118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102080 0x7f0bf8102500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009990 tx=0x7f0bec009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/1976519118 shutdown_connections 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/1976519118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102a40 0x7f0bf8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/1976519118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 0x7f0bf81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/1976519118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102080 0x7f0bf8102500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/1976519118 >> 172.21.15.5:0/1976519118 conn(0x7f0bf80fc480 msgr2=0x7f0bf80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/1976519118 shutdown_connections 2023-09-23T04:43:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.861+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/1976519118 wait complete. 2023-09-23T04:43:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.862+0000 7f0bfc9ff700 1 Processor -- start 2023-09-23T04:43:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 -- start start 2023-09-23T04:43:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 0x7f0bf8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102a40 0x7f0bf8073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8073ef0 con 0x7f0bf8102080 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8074060 con 0x7f0bf8102a40 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bfc9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a0c80 con 0x7f0bf8100ea0 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bf5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bf5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52588/0 (socket says 172.21.15.5:52588) 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bf5d9b700 1 -- 172.21.15.5:0/3536804790 learned_addr learned my addr 172.21.15.5:0/3536804790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bf659c700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 0x7f0bf8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.863+0000 7f0bf6d9d700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102a40 0x7f0bf8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 -- 172.21.15.5:0/3536804790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 msgr2=0x7f0bf8074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 0x7f0bf8074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 -- 172.21.15.5:0/3536804790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102a40 msgr2=0x7f0bf8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102a40 0x7f0bf8073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 -- 172.21.15.5:0/3536804790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0095f0 con 0x7f0bf8102080 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bf5d9b700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005720 tx=0x7f0bec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec01d070 con 0x7f0bf8102080 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bec021410 con 0x7f0bf8102080 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf81a0f00 con 0x7f0bf8102080 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec017650 con 0x7f0bf8102080 2023-09-23T04:43:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.864+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81a1270 con 0x7f0bf8102080 2023-09-23T04:43:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.865+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf8107ea0 con 0x7f0bf8102080 2023-09-23T04:43:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.867+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0bec021580 con 0x7f0bf8102080 2023-09-23T04:43:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.869+0000 7f0be77fe700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 0x7f0be00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.869+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0bec0ce2d0 con 0x7f0bf8102080 2023-09-23T04:43:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.869+0000 7f0bf659c700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 0x7f0be00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.869+0000 7f0bf659c700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 0x7f0be00a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8101bf0 tx=0x7f0be8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:54.872+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0bec097700 con 0x7f0bf8102080 2023-09-23T04:43:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:54 smithi069 ceph-mon[133931]: pgmap v272: 129 pgs: 102 active+clean, 13 creating+peering, 14 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-09-23T04:43:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:54 smithi069 ceph-mon[133931]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:54 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T04:43:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:54 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/268096092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:54 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/382209340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.083+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bf810c760 con 0x7f0bf8102080 2023-09-23T04:43:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.085+0000 7f0be77fe700 1 -- 172.21.15.5:0/3536804790 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f0bec09ad20 con 0x7f0bf8102080 2023-09-23T04:43:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:43:55.085 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 msgr2=0x7f0be00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 0x7f0be00a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8101bf0 tx=0x7f0be8009260 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 msgr2=0x7f0bf8073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005720 tx=0x7f0bec009580 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 shutdown_connections 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0be00a15a0 0x7f0be00a3a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bf8102a40 0x7f0bf8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0bf8100ea0 0x7f0bf8074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 --2- 172.21.15.5:0/3536804790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8102080 0x7f0bf8073210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 >> 172.21.15.5:0/3536804790 conn(0x7f0bf80fc480 msgr2=0x7f0bf80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 shutdown_connections 2023-09-23T04:43:55.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.087+0000 7f0bfc9ff700 1 -- 172.21.15.5:0/3536804790 wait complete. 2023-09-23T04:43:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:54 smithi005 ceph-mon[97638]: pgmap v272: 129 pgs: 102 active+clean, 13 creating+peering, 14 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-09-23T04:43:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:54 smithi005 ceph-mon[97638]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:54 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T04:43:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:54 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/268096092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:54 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/382209340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:55.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T04:43:47.148877+0000', 'modified': '2023-09-23T04:43:52.237323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24577}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24577': {'gid': 24577, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6839/3661239153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 3661239153}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 3661239153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-23T04:43:55.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T04:43:55.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-09-23T04:43:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:54 smithi118 ceph-mon[134297]: pgmap v272: 129 pgs: 102 active+clean, 13 creating+peering, 14 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-09-23T04:43:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:54 smithi118 ceph-mon[134297]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-23T04:43:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:54 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T04:43:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:54 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/268096092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:54 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/382209340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:43:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 -- 172.21.15.5:0/2865847336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0073800 msgr2=0x7f4ec0073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/2865847336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0073800 0x7f4ec0073c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098f0 tx=0x7f4ea8009c00 comp rx=0 tx=0).stop 2023-09-23T04:43:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 -- 172.21.15.5:0/2865847336 shutdown_connections 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/2865847336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0108df0 0x7f4ec010b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/2865847336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0074de0 0x7f4ec0073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/2865847336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0073800 0x7f4ec0073c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 -- 172.21.15.5:0/2865847336 >> 172.21.15.5:0/2865847336 conn(0x7f4ec00fc470 msgr2=0x7f4ec00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 -- 172.21.15.5:0/2865847336 shutdown_connections 2023-09-23T04:43:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.311+0000 7f4ec57ff700 1 -- 172.21.15.5:0/2865847336 wait complete. 2023-09-23T04:43:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.312+0000 7f4ec57ff700 1 Processor -- start 2023-09-23T04:43:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.312+0000 7f4ec57ff700 1 -- start start 2023-09-23T04:43:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.312+0000 7f4ec57ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.312+0000 7f4ec57ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0074de0 0x7f4ec0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.312+0000 7f4ec57ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 0x7f4ec01a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ec57ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a09b0 con 0x7f4ec0073800 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ec57ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a0af0 con 0x7f4ec0108df0 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ec57ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01a0c80 con 0x7f4ec0074de0 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52606/0 (socket says 172.21.15.5:52606) 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 -- 172.21.15.5:0/839280409 learned_addr learned my addr 172.21.15.5:0/839280409 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 0x7f4ec01a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0074de0 0x7f4ec0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 -- 172.21.15.5:0/839280409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0074de0 msgr2=0x7f4ec0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0074de0 0x7f4ec0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 -- 172.21.15.5:0/839280409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 msgr2=0x7f4ec01a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 0x7f4ec01a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 -- 172.21.15.5:0/839280409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8009520 con 0x7f4ec0073800 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 0x7f4ec01a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.313+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000e9c0 tx=0x7f4eb000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.314+0000 7f4ebcff9700 1 -- 172.21.15.5:0/839280409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004a30 con 0x7f4ec0073800 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.314+0000 7f4ebcff9700 1 -- 172.21.15.5:0/839280409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb000cc80 con 0x7f4ec0073800 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.314+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec01a0f60 con 0x7f4ec0073800 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.314+0000 7f4ebcff9700 1 -- 172.21.15.5:0/839280409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00054e0 con 0x7f4ec0073800 2023-09-23T04:43:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.314+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec01a1380 con 0x7f4ec0073800 2023-09-23T04:43:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.317+0000 7f4ebcff9700 1 -- 172.21.15.5:0/839280409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4eb0004b90 con 0x7f4ec0073800 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.318+0000 7f4ed9199b80 1 Processor -- start 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.318+0000 7f4ebcff9700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 0x7f4eac0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 -- start start 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebcff9700 1 -- 172.21.15.5:0/839280409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4eb0014070 con 0x7f4ec0073800 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 0x7f4eac0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea21b410 con 0x558eea2b3eb0 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea20cbd0 con 0x558eea2b3550 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea20a4f0 con 0x558eea2b4d50 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:46706/0 (socket says 172.21.15.5:46706) 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52614/0 (socket says 172.21.15.5:52614) 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebeffd700 1 -- 172.21.15.5:0/2216841780 learned_addr learned my addr 172.21.15.5:0/2216841780 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.319+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 -- 172.21.15.5:0/2216841780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 msgr2=0x558eea2b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 0x7f4eac0a3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8003e80 tx=0x7f4eb800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 -- 172.21.15.5:0/2216841780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 msgr2=0x558eea2b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 -- 172.21.15.5:0/2216841780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eea268880 con 0x558eea2b3550 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a20a0 tx=0x7f4eb0099c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b3736705ee6d3a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4eb57fa700 1 -- 172.21.15.5:0/2216841780 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009e170 con 0x558eea2b3550 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4eb57fa700 1 -- 172.21.15.5:0/2216841780 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb009a240 con 0x558eea2b3550 2023-09-23T04:43:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.320+0000 7f4eb57fa700 1 -- 172.21.15.5:0/2216841780 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009a700 con 0x558eea2b3550 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- 172.21.15.5:0/2216841780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 msgr2=0x558eea2b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a20a0 tx=0x7f4eb0099c70 comp rx=0 tx=0).stop 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- 172.21.15.5:0/2216841780 shutdown_connections 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2b3970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/2216841780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- 172.21.15.5:0/2216841780 >> 172.21.15.5:0/2216841780 conn(0x558eea227210 msgr2=0x558eea2ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- 172.21.15.5:0/2216841780 shutdown_connections 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- 172.21.15.5:0/2216841780 wait complete. 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.321+0000 7f4ed9199b80 1 -- start start 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 Processor -- start 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b4d50 0x558eea2c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea205de0 con 0x558eea2b4d50 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea1a26c0 con 0x558eea2b3550 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea1a2360 con 0x558eea2b3eb0 2023-09-23T04:43:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ed9199b80 10 client.0 _ll_register_callbacks cb 0x558ee9fff7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ebf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b4d50 0x558eea2c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:46716/0 (socket says 172.21.15.5:46716) 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.322+0000 7f4ebeffd700 1 -- 172.21.15.5:0/1533029273 learned_addr learned my addr 172.21.15.5:0/1533029273 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 -- 172.21.15.5:0/1533029273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 msgr2=0x558eea2bd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2bd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 -- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b4d50 msgr2=0x558eea2c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b4d50 0x558eea2c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 -- 172.21.15.5:0/1533029273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec01a1380 con 0x558eea2b3550 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0096cb0 tx=0x7f4eb0009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74350b96bc811e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009e170 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558eea2a6200 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4eb00a25d0 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb00985e0 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ed9199b80 10 client.24662 fetch_fsmap learned FSMap version 17 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.323+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x558eea030d90 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558eea2a6200 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4eb00991a0 con 0x558eea2b3550 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 10 client.24662 fetch_fsmap finished waiting for FSMap version 17 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 10 client.24662 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 20 client.24662 populate_metadata read hostname 'smithi005' 2023-09-23T04:43:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea2c8a40 0x558eea2c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 4 client.24662 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eea1fedc0 con 0x558eea2c8a40 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.324+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea2c8a40 0x558eea2c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.325+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea2c8a40 0x558eea2c77c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d28d2f571c14d16 server_cookie=3cc6ae9b3cc8cd17 in_seq=0 out_seq=0 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.325+0000 7f4ea67fc700 10 client.24662 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1533029273 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558eea1fedc0 con 0x558eea2c8a40 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ea67fc700 10 client.24662 handle_command_reply: tid=0 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 client.24662 shutdown 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 msgr2=0x558eea2bd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0096cb0 tx=0x7f4eb0009f00 comp rx=0 tx=0).stop 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 shutdown_connections 2023-09-23T04:43:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea2c8a40 0x558eea2c77c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b4d50 0x558eea2c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2bd310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ea67fc700 0 client.24662 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1533029273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 >> 172.21.15.5:0/1533029273 conn(0x558eea227210 msgr2=0x558eea2ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 shutdown_connections 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1533029273 wait complete. 2023-09-23T04:43:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.326+0000 7f4ed9199b80 20 client.24662 trim_cache size 0 max 16384 2023-09-23T04:43:55.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 Processor -- start 2023-09-23T04:43:55.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 -- start start 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b4d50 0x558eea2c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea0868d0 con 0x558eea2b3550 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea2a8aa0 con 0x558eea2b4d50 2023-09-23T04:43:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.345+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea2a85f0 con 0x558eea2b3eb0 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52644/0 (socket says 172.21.15.5:52644) 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 -- 172.21.15.5:0/533374613 learned_addr learned my addr 172.21.15.5:0/533374613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b4d50 0x558eea2c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 -- 172.21.15.5:0/533374613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 msgr2=0x558eea2c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 -- 172.21.15.5:0/533374613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b4d50 msgr2=0x558eea2c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b4d50 0x558eea2c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 -- 172.21.15.5:0/533374613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eea019c10 con 0x558eea2b3550 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0096740 tx=0x7f4eb009f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf7b2a52a072464 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4eb5ffb700 1 -- 172.21.15.5:0/533374613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000cdf0 con 0x558eea2b3550 2023-09-23T04:43:55.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4eb5ffb700 1 -- 172.21.15.5:0/533374613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb00a25d0 con 0x558eea2b3550 2023-09-23T04:43:55.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.346+0000 7f4eb5ffb700 1 -- 172.21.15.5:0/533374613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb009f280 con 0x558eea2b3550 2023-09-23T04:43:55.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- 172.21.15.5:0/533374613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 msgr2=0x558eea2c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0096740 tx=0x7f4eb009f150 comp rx=0 tx=0).stop 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- 172.21.15.5:0/533374613 shutdown_connections 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b4d50 0x558eea2c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3550 0x558eea2c77c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/533374613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b3eb0 0x558eea2c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- 172.21.15.5:0/533374613 >> 172.21.15.5:0/533374613 conn(0x558eea227210 msgr2=0x558eea2bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- 172.21.15.5:0/533374613 shutdown_connections 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- 172.21.15.5:0/533374613 wait complete. 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.347+0000 7f4ed9199b80 1 -- start start 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 Processor -- start 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea2c4990 con 0x558eea2b3eb0 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea2c4b00 con 0x558eea2b3550 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eea2c4cc0 con 0x558eea2b4d50 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ed9199b80 10 client.0 _ll_register_callbacks cb 0x7f4eb00091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52658/0 (socket says 172.21.15.5:52658) 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4eb7fff700 1 -- 172.21.15.5:0/1707905595 learned_addr learned my addr 172.21.15.5:0/1707905595 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.348+0000 7f4ebf7fe700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 -- 172.21.15.5:0/1707905595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 msgr2=0x558eea2c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 -- 172.21.15.5:0/1707905595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 msgr2=0x558eea2beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 -- 172.21.15.5:0/1707905595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558eea030d90 con 0x558eea2b3eb0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4eb7fff700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800fa40 tx=0x7f4ea800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3512d059f9ea72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8027d50 con 0x558eea2b3eb0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ea800b710 con 0x558eea2b3eb0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.349+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eb009a240 con 0x558eea2b3eb0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ea800b8d0 con 0x558eea2b3eb0 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 10 client.14661 fetch_fsmap learned FSMap version 17 2023-09-23T04:43:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x558eea057200 con 0x558eea2b3eb0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb009a240 con 0x558eea2b3eb0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4ea8009920 con 0x558eea2b3eb0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 10 client.14661 fetch_fsmap finished waiting for FSMap version 17 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 10 client.14661 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 20 client.14661 populate_metadata read hostname 'smithi005' 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea3215a0 0x558eea31da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 4 client.14661 mds_command: new command op to 24577 tid=0[{"prefix": "status"}] 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558eea1feae0 con 0x558eea3215a0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.350+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea3215a0 0x558eea31da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ebeffd700 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea3215a0 0x558eea31da20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a111aaa29d27d9 server_cookie=3186e1bc56816c17 in_seq=0 out_seq=0 2023-09-23T04:43:55.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ea67fc700 10 client.14661 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ea67fc700 1 -- 172.21.15.5:0/1707905595 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558eea1feae0 con 0x558eea3215a0 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ea67fc700 10 client.14661 handle_command_reply: tid=0 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 client.14661 shutdown 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 msgr2=0x558eea2bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800fa40 tx=0x7f4ea800f8f0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 shutdown_connections 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558eea3215a0 0x558eea31da20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558eea2b4d50 0x558eea2c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558eea2b3550 0x558eea2beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 --2- 172.21.15.5:0/1707905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558eea2b3eb0 0x558eea2bf530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ea67fc700 0 client.14661 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:43:55.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 >> 172.21.15.5:0/1707905595 conn(0x558eea227210 msgr2=0x558eea2c77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.351+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 shutdown_connections 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ed9199b80 1 -- 172.21.15.5:0/1707905595 wait complete. 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ed9199b80 20 client.14661 trim_cache size 0 max 16384 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24577, 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:43:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 7.7623575730000001, 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 17, 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 20.297188813000002 2023-09-23T04:43:55.358 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:43:55.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 msgr2=0x7f4eac0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 0x7f4eac0a3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8003e80 tx=0x7f4eb800b040 comp rx=0 tx=0).stop 2023-09-23T04:43:55.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 msgr2=0x7f4ec0198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:55.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000e9c0 tx=0x7f4eb000ecd0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 shutdown_connections 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4eac0a15a0 0x7f4eac0a3a60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ec0108df0 0x7f4ec01a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec0073800 0x7f4ec0198e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 --2- 172.21.15.5:0/839280409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ec0074de0 0x7f4ec0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 >> 172.21.15.5:0/839280409 conn(0x7f4ec00fc470 msgr2=0x7f4ec00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 shutdown_connections 2023-09-23T04:43:55.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:55.352+0000 7f4ec57ff700 1 -- 172.21.15.5:0/839280409 wait complete. 2023-09-23T04:43:55.370 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T04:43:55.370 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:43:55.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph fs ls 2023-09-23T04:43:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:55 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3536804790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:56.104 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:43:56.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:55 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3536804790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:55 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3536804790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:43:56.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:56 smithi005 ceph-mon[97638]: pgmap v273: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s wr, 2 op/s 2023-09-23T04:43:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:56 smithi069 ceph-mon[133931]: pgmap v273: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s wr, 2 op/s 2023-09-23T04:43:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:56 smithi118 ceph-mon[134297]: pgmap v273: 129 pgs: 116 active+clean, 13 creating+peering; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 345 B/s wr, 2 op/s 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.525+0000 7f4432f6e700 1 -- 172.21.15.5:0/1137616615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1032f0 msgr2=0x7f442c1036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.525+0000 7f4432f6e700 1 --2- 172.21.15.5:0/1137616615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1032f0 0x7f442c1036d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4420009990 tx=0x7f4420009ca0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 -- 172.21.15.5:0/1137616615 shutdown_connections 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 --2- 172.21.15.5:0/1137616615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c1083d0 0x7f442c10ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 --2- 172.21.15.5:0/1137616615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1032f0 0x7f442c1036d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 --2- 172.21.15.5:0/1137616615 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c103ca0 0x7f442c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 -- 172.21.15.5:0/1137616615 >> 172.21.15.5:0/1137616615 conn(0x7f442c0feb60 msgr2=0x7f442c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 -- 172.21.15.5:0/1137616615 shutdown_connections 2023-09-23T04:43:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.526+0000 7f4432f6e700 1 -- 172.21.15.5:0/1137616615 wait complete. 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 Processor -- start 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 -- start start 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c1032f0 0x7f442c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c103ca0 0x7f442c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c19e180 con 0x7f442c1083d0 2023-09-23T04:43:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c19e2f0 con 0x7f442c103ca0 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f4432f6e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c19e4b0 con 0x7f442c1032f0 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.527+0000 7f443150b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52672/0 (socket says 172.21.15.5:52672) 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 -- 172.21.15.5:0/3531927931 learned_addr learned my addr 172.21.15.5:0/3531927931 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f4430d0a700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c1032f0 0x7f442c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f442bfff700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c103ca0 0x7f442c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 -- 172.21.15.5:0/3531927931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c1032f0 msgr2=0x7f442c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c1032f0 0x7f442c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 -- 172.21.15.5:0/3531927931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c103ca0 msgr2=0x7f442c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c103ca0 0x7f442c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 -- 172.21.15.5:0/3531927931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200095f0 con 0x7f442c1083d0 2023-09-23T04:43:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f443150b700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f442400ef30 tx=0x7f442400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.528+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424018070 con 0x7f442c1083d0 2023-09-23T04:43:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.529+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c19e730 con 0x7f442c1083d0 2023-09-23T04:43:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.529+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442400f040 con 0x7f442c1083d0 2023-09-23T04:43:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.529+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400cd00 con 0x7f442c1083d0 2023-09-23T04:43:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.529+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c1a5ff0 con 0x7f442c1083d0 2023-09-23T04:43:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.530+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442c04eb20 con 0x7f442c1083d0 2023-09-23T04:43:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.532+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4424003a00 con 0x7f442c1083d0 2023-09-23T04:43:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.533+0000 7f4429ffb700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 0x7f441c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:43:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.533+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44240ccb50 con 0x7f442c1083d0 2023-09-23T04:43:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.533+0000 7f4430d0a700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 0x7f441c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:43:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.534+0000 7f4430d0a700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 0x7f441c0a3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f442000b2e0 tx=0x7f4420005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:43:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.537+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4424095f80 con 0x7f442c1083d0 2023-09-23T04:43:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.791+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f442c19a520 con 0x7f442c1083d0 2023-09-23T04:43:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.792+0000 7f4429ffb700 1 -- 172.21.15.5:0/3531927931 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f44240995a0 con 0x7f442c1083d0 2023-09-23T04:43:57.793 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T04:43:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 msgr2=0x7f441c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 0x7f441c0a3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f442000b2e0 tx=0x7f4420005fd0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 msgr2=0x7f442c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:43:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f442400ef30 tx=0x7f442400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 shutdown_connections 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f441c0a1620 0x7f441c0a3ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f442c1083d0 0x7f442c19db70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f442c1032f0 0x7f442c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 --2- 172.21.15.5:0/3531927931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f442c103ca0 0x7f442c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 >> 172.21.15.5:0/3531927931 conn(0x7f442c0feb60 msgr2=0x7f442c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 shutdown_connections 2023-09-23T04:43:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:43:57.794+0000 7f4432f6e700 1 -- 172.21.15.5:0/3531927931 wait complete. 2023-09-23T04:43:58.282 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-23T04:43:58.282 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T04:43:58.283 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-23T04:43:58.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:58.283 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:43:58.328 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41781sec preferred_lft 41781sec 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2023-09-23T04:43:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:43:58.330 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:43:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-09-23T04:43:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-09-23T04:43:58.330 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T04:43:58.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:58.330 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:43:58.330 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:43:58.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2023-09-23T04:43:58.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2023-09-23T04:43:58.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2023-09-23T04:43:58.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T04:43:58.331 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:43:58.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:58 smithi005 ceph-mon[97638]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-09-23T04:43:58.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:43:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3531927931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:43:58.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:58.817 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T04:43:58.889 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-09-23T04:43:58.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:58.892 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-09-23T04:43:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-09-23T04:43:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T04:43:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-23T04:43:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-23T04:43:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T04:43:58.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-23T04:43:58.948 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:43:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:58 smithi069 ceph-mon[133931]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-09-23T04:43:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:43:58 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3531927931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:43:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:58 smithi118 ceph-mon[134297]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2023-09-23T04:43:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:43:58 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3531927931' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:43:59.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:59.679 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-09-23T04:43:59.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:59.723 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2023-09-23T04:43:59.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:59.808 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:43:59.808 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:43:59.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:43:59.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-23T04:43:59.808 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:43:59.995 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-23T04:43:59.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:43:59.995 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:43:59.995 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:43:59.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-23T04:43:59.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T04:43:59.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-23T04:43:59.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-23T04:43:59.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-23T04:43:59.996 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:44:00.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:00.266 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:44:00.266 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:44:00.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2023-09-23T04:44:00.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T04:44:00.266 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:44:00.441 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-23T04:44:00.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T04:44:00.442 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:00.485 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-23T04:44:00.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T04:44:00.486 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:00.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-09-23T04:44:00.679 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-09-23T04:44:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/systemd 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/perf_event 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/pids 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/devices 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/blkio 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/memory 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/freezer 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/cpuset 2023-09-23T04:44:00.725 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/hugetlb 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T04:44:00.726 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b9b12ed9a9675756398c01c798a6a045b210c5bc96a25f14ee5094d6775b4a04/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/24f25012bc38e573cdb8845c578de1db32d9d29ad3a5c289ff6387d62067fba8/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/832e5e69fb0ede7891a43bed46ca18d241229af141dce7e0be9cc8a928cc3d58/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f8522220dd0625668b683ef7fe0ae97d169166af79afdc5d8d0a63df7c2694c9/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ef4639c4f68c15d01a53462586b846fbcaad933e39f0467c8cab33f9947db678/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/4600bab91f22accac71b625898083ebeb4956d0a78c736d6e21e185625325044/merged 2023-09-23T04:44:00.727 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/56328f78487dafec3fc7754d4a7b6dfee95af6ae64a120487ea9bbcfb92a9b3b/merged 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/efa8c0ae26e31a4cb4d4ba7889418b5b6c7e7d4d88544efa68136a05b751b092/merged 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/557655226d71ff5d11ad8f1dc304d2db23fafc59a46feb4c0c1e6277fd762e9d/merged 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/83c5ba2a76e0c732da705c63df967bc6c58223aa1bda8c4ef36b924fa80b8be7/merged 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:44:00.728 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:44:00.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:00.729 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-09-23T04:44:00.813 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T04:44:00.813 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-23T04:44:00.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-09-23T04:44:00.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:00 smithi005 ceph-mon[97638]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:00.930 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-09-23T04:44:00.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/systemd 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/perf_event 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/pids 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/devices 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/blkio 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/memory 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/freezer 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/cpuset 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup/hugetlb 2023-09-23T04:44:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-09-23T04:44:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b9b12ed9a9675756398c01c798a6a045b210c5bc96a25f14ee5094d6775b4a04/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/24f25012bc38e573cdb8845c578de1db32d9d29ad3a5c289ff6387d62067fba8/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/832e5e69fb0ede7891a43bed46ca18d241229af141dce7e0be9cc8a928cc3d58/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f8522220dd0625668b683ef7fe0ae97d169166af79afdc5d8d0a63df7c2694c9/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ef4639c4f68c15d01a53462586b846fbcaad933e39f0467c8cab33f9947db678/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/4600bab91f22accac71b625898083ebeb4956d0a78c736d6e21e185625325044/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/56328f78487dafec3fc7754d4a7b6dfee95af6ae64a120487ea9bbcfb92a9b3b/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/efa8c0ae26e31a4cb4d4ba7889418b5b6c7e7d4d88544efa68136a05b751b092/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/557655226d71ff5d11ad8f1dc304d2db23fafc59a46feb4c0c1e6277fd762e9d/merged 2023-09-23T04:44:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/83c5ba2a76e0c732da705c63df967bc6c58223aa1bda8c4ef36b924fa80b8be7/merged 2023-09-23T04:44:00.980 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-09-23T04:44:00.980 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:44:00.980 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:44:00.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:00.981 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-09-23T04:44:00.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:00 smithi069 ceph-mon[133931]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:01.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2023-09-23T04:44:01.051+0000 7f52291af5c0 -1 init, newargv = 0x5611629c5e40 newargc=15 2023-09-23T04:44:01.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[152035]: starting ceph client 2023-09-23T04:44:01.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[152035]: starting fuse 2023-09-23T04:44:01.069 INFO:teuthology.orchestra.run.smithi005.stdout:121 2023-09-23T04:44:01.069 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-09-23T04:44:01.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> import re 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> import os 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2023-09-23T04:44:01.070 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> return f 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> 2023-09-23T04:44:01.071 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2023-09-23T04:44:01.072 DEBUG:teuthology.orchestra.run.smithi005:> ' 2023-09-23T04:44:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:00 smithi118 ceph-mon[134297]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.152035.asok 2023-09-23T04:44:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T04:44:01.268 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.152035.asok 2023-09-23T04:44:01.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:01.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.152035.asok status 2023-09-23T04:44:01.499 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "152035", 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T04:44:01.500 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24718, 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 24718 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-09-23T04:44:01.501 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3451446776 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3451446776 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.24718 192.168.0.1:0/3451446776", 2023-09-23T04:44:01.502 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/3451446776", 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 17, 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 87, 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2023-09-23T04:44:01.503 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:44:01.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:01.514 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2023-09-23T04:44:01.514 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:01.559 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2023-09-23T04:44:01.559 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:01.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:01.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:01.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:01.672 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-23T04:44:01.787 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2023-09-23T04:44:01.787 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-23T04:44:01.787 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:01.788 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:44:01.788 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T04:44:01.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:44:02.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 -- 172.21.15.5:0/876178572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102070 msgr2=0x7f31a81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/876178572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102070 0x7f31a81024f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f319c009990 tx=0x7f319c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 -- 172.21.15.5:0/876178572 shutdown_connections 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/876178572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8102a30 0x7f31a8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/876178572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8100e90 0x7f31a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/876178572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102070 0x7f31a81024f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 -- 172.21.15.5:0/876178572 >> 172.21.15.5:0/876178572 conn(0x7f31a80fc450 msgr2=0x7f31a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 -- 172.21.15.5:0/876178572 shutdown_connections 2023-09-23T04:44:02.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.075+0000 7f31ad7ff700 1 -- 172.21.15.5:0/876178572 wait complete. 2023-09-23T04:44:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.076+0000 7f31ad7ff700 1 Processor -- start 2023-09-23T04:44:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 -- start start 2023-09-23T04:44:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8100e90 0x7f31a8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8102070 0x7f31a8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8073ef0 con 0x7f31a8102a30 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8074060 con 0x7f31a8102070 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31ad7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a81a0c50 con 0x7f31a8100e90 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38010/0 (socket says 172.21.15.5:38010) 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31a77fe700 1 -- 172.21.15.5:0/1471527687 learned_addr learned my addr 172.21.15.5:0/1471527687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31a6ffd700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8100e90 0x7f31a8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.077+0000 7f31a67fc700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8102070 0x7f31a8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 -- 172.21.15.5:0/1471527687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8100e90 msgr2=0x7f31a8074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8100e90 0x7f31a8074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 -- 172.21.15.5:0/1471527687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8102070 msgr2=0x7f31a8073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8102070 0x7f31a8073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 -- 172.21.15.5:0/1471527687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0095f0 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31a77fe700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f31a000ef30 tx=0x7f31a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a0018070 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31a000f040 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a81a0ed0 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000cd00 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.078+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a81a1300 con 0x7f31a8102a30 2023-09-23T04:44:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.080+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a8107e90 con 0x7f31a8102a30 2023-09-23T04:44:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.081+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f31a0003a00 con 0x7f31a8102a30 2023-09-23T04:44:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.084+0000 7f318ffff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 0x7f31900a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.084+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f31a00ccc30 con 0x7f31a8102a30 2023-09-23T04:44:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.084+0000 7f31a6ffd700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 0x7f31900a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.084+0000 7f31a6ffd700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 0x7f31900a3a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31a8101be0 tx=0x7f3198009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.088+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f31a00d3080 con 0x7f31a8102a30 2023-09-23T04:44:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.302+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a810c750 con 0x7f31a8102a30 2023-09-23T04:44:02.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.303+0000 7f318ffff700 1 -- 172.21.15.5:0/1471527687 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f31a00960c0 con 0x7f31a8102a30 2023-09-23T04:44:02.305 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:02.305 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 msgr2=0x7f31900a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 0x7f31900a3a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31a8101be0 tx=0x7f3198009260 comp rx=0 tx=0).stop 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 msgr2=0x7f31a8073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f31a000ef30 tx=0x7f31a000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 shutdown_connections 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31900a15a0 0x7f31900a3a60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a8102a30 0x7f31a8073750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a8100e90 0x7f31a8074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 --2- 172.21.15.5:0/1471527687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a8102070 0x7f31a8073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 >> 172.21.15.5:0/1471527687 conn(0x7f31a80fc450 msgr2=0x7f31a80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 shutdown_connections 2023-09-23T04:44:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.306+0000 7f31ad7ff700 1 -- 172.21.15.5:0/1471527687 wait complete. 2023-09-23T04:44:02.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:44:02.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:44:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 -- 172.21.15.5:0/2989445175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878100eb0 msgr2=0x7f48781012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 --2- 172.21.15.5:0/2989445175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878100eb0 0x7f48781012d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f487400b0a0 tx=0x7f487400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 -- 172.21.15.5:0/2989445175 shutdown_connections 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 --2- 172.21.15.5:0/2989445175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f4878109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 --2- 172.21.15.5:0/2989445175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878100eb0 0x7f48781012d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 --2- 172.21.15.5:0/2989445175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878102090 0x7f4878102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.531+0000 7f487fdff700 1 -- 172.21.15.5:0/2989445175 >> 172.21.15.5:0/2989445175 conn(0x7f48780fc470 msgr2=0x7f48780fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.532+0000 7f487fdff700 1 -- 172.21.15.5:0/2989445175 shutdown_connections 2023-09-23T04:44:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.532+0000 7f487fdff700 1 -- 172.21.15.5:0/2989445175 wait complete. 2023-09-23T04:44:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 Processor -- start 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 -- start start 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878100eb0 0x7f4878196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f487819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819e8f0 con 0x7f4878102090 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819ea60 con 0x7f4878100eb0 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487fdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819ec20 con 0x7f4878102a50 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38018/0 (socket says 172.21.15.5:38018) 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487d39a700 1 -- 172.21.15.5:0/3377988837 learned_addr learned my addr 172.21.15.5:0/3377988837 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487db9b700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878100eb0 0x7f4878196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.533+0000 7f487e39c700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f487819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 -- 172.21.15.5:0/3377988837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 msgr2=0x7f487819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f487819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 -- 172.21.15.5:0/3377988837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878100eb0 msgr2=0x7f4878196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878100eb0 0x7f4878196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 -- 172.21.15.5:0/3377988837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009d00 con 0x7f4878102090 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487e39c700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f487819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487d39a700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f486800e970 tx=0x7f486800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868009790 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f486800eec0 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487819ef00 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868003cf0 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.534+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487819f330 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.535+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487810c770 con 0x7f4878102090 2023-09-23T04:44:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.537+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f486800ccd0 con 0x7f4878102090 2023-09-23T04:44:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.539+0000 7f486effd700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 0x7f48640a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.539+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4868014070 con 0x7f4878102090 2023-09-23T04:44:02.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.539+0000 7f487db9b700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 0x7f48640a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.539+0000 7f487db9b700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 0x7f48640a3ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48740062a0 tx=0x7f4874006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.543+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f48680966e0 con 0x7f4878102090 2023-09-23T04:44:02.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.759+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4878107e80 con 0x7f4878102090 2023-09-23T04:44:02.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.759+0000 7f486effd700 1 -- 172.21.15.5:0/3377988837 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4868099d00 con 0x7f4878102090 2023-09-23T04:44:02.760 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 msgr2=0x7f48640a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 0x7f48640a3ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48740062a0 tx=0x7f4874006210 comp rx=0 tx=0).stop 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 msgr2=0x7f48781972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f486800e970 tx=0x7f486800ec80 comp rx=0 tx=0).stop 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 shutdown_connections 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48640a1620 0x7f48640a3ae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4878102a50 0x7f487819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4878100eb0 0x7f4878196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 --2- 172.21.15.5:0/3377988837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4878102090 0x7f48781972f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 >> 172.21.15.5:0/3377988837 conn(0x7f48780fc470 msgr2=0x7f48780fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 shutdown_connections 2023-09-23T04:44:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.762+0000 7f487fdff700 1 -- 172.21.15.5:0/3377988837 wait complete. 2023-09-23T04:44:02.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:44:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 -- 172.21.15.5:0/187125450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0074e60 msgr2=0x7f54e00732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 --2- 172.21.15.5:0/187125450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0074e60 0x7f54e00732b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 -- 172.21.15.5:0/187125450 shutdown_connections 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 --2- 172.21.15.5:0/187125450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0108dc0 0x7f54e010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 --2- 172.21.15.5:0/187125450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0074e60 0x7f54e00732b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 --2- 172.21.15.5:0/187125450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0073880 0x7f54e0073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 -- 172.21.15.5:0/187125450 >> 172.21.15.5:0/187125450 conn(0x7f54e00fc470 msgr2=0x7f54e00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 -- 172.21.15.5:0/187125450 shutdown_connections 2023-09-23T04:44:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.985+0000 7f54e6fff700 1 -- 172.21.15.5:0/187125450 wait complete. 2023-09-23T04:44:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.986+0000 7f54e6fff700 1 Processor -- start 2023-09-23T04:44:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 -- start start 2023-09-23T04:44:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0073880 0x7f54e0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0074e60 0x7f54e0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e01a0940 con 0x7f54e0108dc0 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e01a0a80 con 0x7f54e0074e60 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e6fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e01a0c10 con 0x7f54e0073880 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e559c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e4d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0073880 0x7f54e0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e559c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38032/0 (socket says 172.21.15.5:38032) 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54e559c700 1 -- 172.21.15.5:0/2919860652 learned_addr learned my addr 172.21.15.5:0/2919860652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.987+0000 7f54dffff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0074e60 0x7f54e0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 -- 172.21.15.5:0/2919860652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0073880 msgr2=0x7f54e0198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0073880 0x7f54e0198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 -- 172.21.15.5:0/2919860652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0074e60 msgr2=0x7f54e0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0074e60 0x7f54e0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 -- 172.21.15.5:0/2919860652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d40095f0 con 0x7f54e0108dc0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e559c700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54d800ef30 tx=0x7f54d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8018070 con 0x7f54e0108dc0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e01a0e90 con 0x7f54e0108dc0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d800f040 con 0x7f54e0108dc0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cd00 con 0x7f54e0108dc0 2023-09-23T04:44:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.988+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e01a12c0 con 0x7f54e0108dc0 2023-09-23T04:44:02.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:02 smithi069 ceph-mon[133931]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:02.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:02 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1471527687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.989+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e0107470 con 0x7f54e0108dc0 2023-09-23T04:44:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.991+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f54d8003a00 con 0x7f54e0108dc0 2023-09-23T04:44:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.993+0000 7f54ddffb700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 0x7f54d00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.993+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54d80ccc90 con 0x7f54e0108dc0 2023-09-23T04:44:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.993+0000 7f54e4d9b700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 0x7f54d00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.993+0000 7f54e4d9b700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 0x7f54d00a3a60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b2e0 tx=0x7f54d4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:02.997+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f54d80960c0 con 0x7f54e0108dc0 2023-09-23T04:44:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:02 smithi005 ceph-mon[97638]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:02 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1471527687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:02 smithi118 ceph-mon[134297]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:02 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1471527687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:03.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.209+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54e002d1a0 con 0x7f54e0108dc0 2023-09-23T04:44:03.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.210+0000 7f54ddffb700 1 -- 172.21.15.5:0/2919860652 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54d80996e0 con 0x7f54e0108dc0 2023-09-23T04:44:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:44:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 msgr2=0x7f54d00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 0x7f54d00a3a60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b2e0 tx=0x7f54d4005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 msgr2=0x7f54e01a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54d800ef30 tx=0x7f54d800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 shutdown_connections 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f54d00a15a0 0x7f54d00a3a60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0108dc0 0x7f54e01a0370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54e0073880 0x7f54e0198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 --2- 172.21.15.5:0/2919860652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e0074e60 0x7f54e0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 >> 172.21.15.5:0/2919860652 conn(0x7f54e00fc470 msgr2=0x7f54e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 shutdown_connections 2023-09-23T04:44:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.212+0000 7f54e6fff700 1 -- 172.21.15.5:0/2919860652 wait complete. 2023-09-23T04:44:03.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:44:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 -- 172.21.15.5:0/4196024287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c100f10 msgr2=0x7f095c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 --2- 172.21.15.5:0/4196024287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c100f10 0x7f095c101330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0958009990 tx=0x7f0958009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 -- 172.21.15.5:0/4196024287 shutdown_connections 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 --2- 172.21.15.5:0/4196024287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 --2- 172.21.15.5:0/4196024287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c100f10 0x7f095c101330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 --2- 172.21.15.5:0/4196024287 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c102060 0x7f095c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 -- 172.21.15.5:0/4196024287 >> 172.21.15.5:0/4196024287 conn(0x7f095c0fc480 msgr2=0x7f095c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.436+0000 7f09641ff700 1 -- 172.21.15.5:0/4196024287 shutdown_connections 2023-09-23T04:44:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.437+0000 7f09641ff700 1 -- 172.21.15.5:0/4196024287 wait complete. 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.437+0000 7f09641ff700 1 Processor -- start 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 -- start start 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c100f10 0x7f095c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c102060 0x7f095c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a0970 con 0x7f095c102a20 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a0ab0 con 0x7f095c102060 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f09641ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1a0c40 con 0x7f095c100f10 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f096279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f096279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38050/0 (socket says 172.21.15.5:38050) 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f096279c700 1 -- 172.21.15.5:0/632215501 learned_addr learned my addr 172.21.15.5:0/632215501 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f0961f9b700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c100f10 0x7f095c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.438+0000 7f096179a700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c102060 0x7f095c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 -- 172.21.15.5:0/632215501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c100f10 msgr2=0x7f095c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c100f10 0x7f095c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 -- 172.21.15.5:0/632215501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c102060 msgr2=0x7f095c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c102060 0x7f095c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 -- 172.21.15.5:0/632215501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580095f0 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f096279c700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095400ef30 tx=0x7f095400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954018070 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c1a0ec0 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095400f040 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400cd00 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.439+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c1a1340 con 0x7f095c102a20 2023-09-23T04:44:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.440+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c193550 con 0x7f095c102a20 2023-09-23T04:44:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.442+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0954003a00 con 0x7f095c102a20 2023-09-23T04:44:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.444+0000 7f0952ffd700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 0x7f09480a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.444+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f09540ccbf0 con 0x7f095c102a20 2023-09-23T04:44:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.444+0000 7f0961f9b700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 0x7f09480a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.445+0000 7f0961f9b700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 0x7f09480a3ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f095800b2e0 tx=0x7f0958005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.448+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0954096020 con 0x7f095c102a20 2023-09-23T04:44:03.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.667+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f095c02d180 con 0x7f095c102a20 2023-09-23T04:44:03.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.667+0000 7f0952ffd700 1 -- 172.21.15.5:0/632215501 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0954099640 con 0x7f095c102a20 2023-09-23T04:44:03.668 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 msgr2=0x7f09480a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 0x7f09480a3ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f095800b2e0 tx=0x7f0958005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 msgr2=0x7f095c1a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095400ef30 tx=0x7f095400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 shutdown_connections 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f09480a1620 0x7f09480a3ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f095c102a20 0x7f095c1a03a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f095c100f10 0x7f095c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 --2- 172.21.15.5:0/632215501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f095c102060 0x7f095c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 >> 172.21.15.5:0/632215501 conn(0x7f095c0fc480 msgr2=0x7f095c1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:03.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 shutdown_connections 2023-09-23T04:44:03.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.670+0000 7f09641ff700 1 -- 172.21.15.5:0/632215501 wait complete. 2023-09-23T04:44:03.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:44:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 -- 172.21.15.5:0/1749446906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b34104300 msgr2=0x7f5b341066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 --2- 172.21.15.5:0/1749446906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b34104300 0x7f5b341066f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009990 tx=0x7f5b20009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 -- 172.21.15.5:0/1749446906 shutdown_connections 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 --2- 172.21.15.5:0/1749446906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34106c30 0x7f5b34109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 --2- 172.21.15.5:0/1749446906 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b341019d0 0x7f5b34103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 --2- 172.21.15.5:0/1749446906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b34104300 0x7f5b341066f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 -- 172.21.15.5:0/1749446906 >> 172.21.15.5:0/1749446906 conn(0x7f5b340fb3a0 msgr2=0x7f5b340fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 -- 172.21.15.5:0/1749446906 shutdown_connections 2023-09-23T04:44:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.896+0000 7f5b3afff700 1 -- 172.21.15.5:0/1749446906 wait complete. 2023-09-23T04:44:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.898+0000 7f5b3afff700 1 Processor -- start 2023-09-23T04:44:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.898+0000 7f5b3afff700 1 -- start start 2023-09-23T04:44:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34104300 0x7f5b340ff980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b34106c30 0x7f5b340ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34100660 con 0x7f5b341019d0 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b341007d0 con 0x7f5b34104300 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3afff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b341a0c20 con 0x7f5b34106c30 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38072/0 (socket says 172.21.15.5:38072) 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 -- 172.21.15.5:0/3205676620 learned_addr learned my addr 172.21.15.5:0/3205676620 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b33fff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34104300 0x7f5b340ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b3959c700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b34106c30 0x7f5b340ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 -- 172.21.15.5:0/3205676620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b34106c30 msgr2=0x7f5b340ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b34106c30 0x7f5b340ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 -- 172.21.15.5:0/3205676620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34104300 msgr2=0x7f5b340ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34104300 0x7f5b340ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.899+0000 7f5b38d9b700 1 -- 172.21.15.5:0/3205676620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200095f0 con 0x7f5b341019d0 2023-09-23T04:44:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b38d9b700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800d950 tx=0x7f5b2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800f890 con 0x7f5b341019d0 2023-09-23T04:44:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b28010460 con 0x7f5b341019d0 2023-09-23T04:44:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b341a0f00 con 0x7f5b341019d0 2023-09-23T04:44:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800e610 con 0x7f5b341019d0 2023-09-23T04:44:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.900+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b341a1320 con 0x7f5b341019d0 2023-09-23T04:44:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.901+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b340fcf20 con 0x7f5b341019d0 2023-09-23T04:44:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.903+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f5b280108d0 con 0x7f5b341019d0 2023-09-23T04:44:03.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.904+0000 7f5b31ffb700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 0x7f5b240a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:03.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.905+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b280cceb0 con 0x7f5b341019d0 2023-09-23T04:44:03.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.905+0000 7f5b33fff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 0x7f5b240a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:03.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.905+0000 7f5b33fff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 0x7f5b240a3a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b2e0 tx=0x7f5b20005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:03.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:03.908+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5b28096250 con 0x7f5b341019d0 2023-09-23T04:44:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:03 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3377988837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:03 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2919860652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:03 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/632215501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.120+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b3402d2a0 con 0x7f5b341019d0 2023-09-23T04:44:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.121+0000 7f5b31ffb700 1 -- 172.21.15.5:0/3205676620 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b280160a0 con 0x7f5b341019d0 2023-09-23T04:44:04.121 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:44:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 msgr2=0x7f5b240a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 0x7f5b240a3a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b2e0 tx=0x7f5b20005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 msgr2=0x7f5b34101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800d950 tx=0x7f5b2800dc60 comp rx=0 tx=0).stop 2023-09-23T04:44:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 shutdown_connections 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5b240a15a0 0x7f5b240a3a60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5b34106c30 0x7f5b340ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b341019d0 0x7f5b34101340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 --2- 172.21.15.5:0/3205676620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b34104300 0x7f5b340ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 >> 172.21.15.5:0/3205676620 conn(0x7f5b340fb3a0 msgr2=0x7f5b34102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 shutdown_connections 2023-09-23T04:44:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.123+0000 7f5b3afff700 1 -- 172.21.15.5:0/3205676620 wait complete. 2023-09-23T04:44:04.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:03 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3377988837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:03 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2919860652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:03 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/632215501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T04:44:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:03 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3377988837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:03 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2919860652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:03 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/632215501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 -- 172.21.15.5:0/1514625876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 msgr2=0x7f7e2c103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 --2- 172.21.15.5:0/1514625876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c103e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009990 tx=0x7f7e1c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 -- 172.21.15.5:0/1514625876 shutdown_connections 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 --2- 172.21.15.5:0/1514625876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 --2- 172.21.15.5:0/1514625876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c103e20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 --2- 172.21.15.5:0/1514625876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 0x7f7e2c106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 -- 172.21.15.5:0/1514625876 >> 172.21.15.5:0/1514625876 conn(0x7f7e2c0fb3a0 msgr2=0x7f7e2c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 -- 172.21.15.5:0/1514625876 shutdown_connections 2023-09-23T04:44:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.347+0000 7f7e309ff700 1 -- 172.21.15.5:0/1514625876 wait complete. 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 Processor -- start 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 -- start start 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 0x7f7e2c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1a0960 con 0x7f7e2c101a30 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1a0aa0 con 0x7f7e2c104360 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.348+0000 7f7e309ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1a0c30 con 0x7f7e2c106c90 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49926/0 (socket says 172.21.15.5:49926) 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 -- 172.21.15.5:0/514250503 learned_addr learned my addr 172.21.15.5:0/514250503 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e29d9b700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 0x7f7e2c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 -- 172.21.15.5:0/514250503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 msgr2=0x7f7e2c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 0x7f7e2c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 -- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 msgr2=0x7f7e2c198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 -- 172.21.15.5:0/514250503 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c0095f0 con 0x7f7e2c106c90 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e2ad9d700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400ef30 tx=0x7f7e2400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.349+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24018070 con 0x7f7e2c106c90 2023-09-23T04:44:04.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.350+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e2400f040 con 0x7f7e2c106c90 2023-09-23T04:44:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.350+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c1a0eb0 con 0x7f7e2c106c90 2023-09-23T04:44:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.350+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cd00 con 0x7f7e2c106c90 2023-09-23T04:44:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.350+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c1a12d0 con 0x7f7e2c106c90 2023-09-23T04:44:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.351+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c108810 con 0x7f7e2c106c90 2023-09-23T04:44:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.353+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f7e24003a00 con 0x7f7e2c106c90 2023-09-23T04:44:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.354+0000 7f7e1b7fe700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 0x7f7e140a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.355+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7e240cd5e0 con 0x7f7e2c106c90 2023-09-23T04:44:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.355+0000 7f7e2a59c700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 0x7f7e140a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.355+0000 7f7e2a59c700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 0x7f7e140a3a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b2e0 tx=0x7f7e1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:04.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.357+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7e24096a10 con 0x7f7e2c106c90 2023-09-23T04:44:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.584+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e2c02d230 con 0x7f7e2c106c90 2023-09-23T04:44:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.585+0000 7f7e1b7fe700 1 -- 172.21.15.5:0/514250503 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+580 (secure 0 0 0) 0x7f7e2409a030 con 0x7f7e2c106c90 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3461224331 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1525019998 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2022747688 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2627772518 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3431143820 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:04.586 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1934869339 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:04.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.587+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 msgr2=0x7f7e140a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.587+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 0x7f7e140a3a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b2e0 tx=0x7f7e1c005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.587+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 msgr2=0x7f7e2c1a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.587+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400ef30 tx=0x7f7e2400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 shutdown_connections 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e140a15a0 0x7f7e140a3a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7e2c106c90 0x7f7e2c1a0390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e2c101a30 0x7f7e2c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 --2- 172.21.15.5:0/514250503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e2c104360 0x7f7e2c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 >> 172.21.15.5:0/514250503 conn(0x7f7e2c0fb3a0 msgr2=0x7f7e2c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:04.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 shutdown_connections 2023-09-23T04:44:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.588+0000 7f7e309ff700 1 -- 172.21.15.5:0/514250503 wait complete. 2023-09-23T04:44:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2023-09-23T04:44:04.607 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:44:04.607 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T04:44:04.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 -- 172.21.15.5:0/224969004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8100ea0 msgr2=0x7f4ac81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/224969004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8100ea0 0x7f4ac81012c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009990 tx=0x7f4ab8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 -- 172.21.15.5:0/224969004 shutdown_connections 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/224969004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102a40 0x7f4ac8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/224969004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8100ea0 0x7f4ac81012c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/224969004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 0x7f4ac8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 -- 172.21.15.5:0/224969004 >> 172.21.15.5:0/224969004 conn(0x7f4ac80fc480 msgr2=0x7f4ac80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 -- 172.21.15.5:0/224969004 shutdown_connections 2023-09-23T04:44:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.893+0000 7f4ace9ff700 1 -- 172.21.15.5:0/224969004 wait complete. 2023-09-23T04:44:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.894+0000 7f4ace9ff700 1 Processor -- start 2023-09-23T04:44:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 -- start start 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 0x7f4ac8196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 0x7f4ac81970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819e710 con 0x7f4ac8102a40 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819e850 con 0x7f4ac8100ea0 2023-09-23T04:44:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ace9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819e9e0 con 0x7f4ac8102080 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4accf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4ac7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 0x7f4ac8196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4accf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38112/0 (socket says 172.21.15.5:38112) 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.895+0000 7f4accf9c700 1 -- 172.21.15.5:0/3085450551 learned_addr learned my addr 172.21.15.5:0/3085450551 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ac77fe700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 0x7f4ac81970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 -- 172.21.15.5:0/3085450551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 msgr2=0x7f4ac81970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 0x7f4ac81970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 -- 172.21.15.5:0/3085450551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 msgr2=0x7f4ac8196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 0x7f4ac8196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 -- 172.21.15.5:0/3085450551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80095f0 con 0x7f4ac8102a40 2023-09-23T04:44:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 0x7f4ac8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4accf9c700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007d70 tx=0x7f4ac000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000ac90 con 0x7f4ac8102a40 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac819ecc0 con 0x7f4ac8102a40 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ac000adf0 con 0x7f4ac8102a40 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.896+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00115e0 con 0x7f4ac8102a40 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.897+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac819f180 con 0x7f4ac8102a40 2023-09-23T04:44:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.898+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac8107e50 con 0x7f4ac8102a40 2023-09-23T04:44:04.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.900+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4ac0012420 con 0x7f4ac8102a40 2023-09-23T04:44:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.901+0000 7f4ac57fa700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 0x7f4ab00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.901+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ac00cd340 con 0x7f4ac8102a40 2023-09-23T04:44:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.901+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 0x7f4ab00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.902+0000 7f4ac7fff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 0x7f4ab00a3a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b2e0 tx=0x7f4ab8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:04.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:04 smithi005 ceph-mon[97638]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:04.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:04 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3205676620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:04 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/514250503' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:04.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:04.905+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4ac00966e0 con 0x7f4ac8102a40 2023-09-23T04:44:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:04 smithi069 ceph-mon[133931]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:04 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3205676620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:04 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/514250503' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.119+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac810c760 con 0x7f4ac8102a40 2023-09-23T04:44:05.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.121+0000 7f4ac57fa700 1 -- 172.21.15.5:0/3085450551 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f4ac0099d00 con 0x7f4ac8102a40 2023-09-23T04:44:05.122 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:05.122 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.123+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 msgr2=0x7f4ab00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.123+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 0x7f4ab00a3a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b2e0 tx=0x7f4ab8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.123+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 msgr2=0x7f4ac819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.123+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0007d70 tx=0x7f4ac000a340 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 shutdown_connections 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f4ab00a15a0 0x7f4ab00a3a60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac8102a40 0x7f4ac819e140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8100ea0 0x7f4ac8196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 --2- 172.21.15.5:0/3085450551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4ac8102080 0x7f4ac81970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 >> 172.21.15.5:0/3085450551 conn(0x7f4ac80fc480 msgr2=0x7f4ac80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 shutdown_connections 2023-09-23T04:44:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.124+0000 7f4ace9ff700 1 -- 172.21.15.5:0/3085450551 wait complete. 2023-09-23T04:44:05.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:44:05.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:44:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:04 smithi118 ceph-mon[134297]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-23T04:44:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3205676620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/514250503' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:05.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 -- 172.21.15.5:0/300449883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 msgr2=0x7f81d8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 --2- 172.21.15.5:0/300449883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d8109340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009f90 tx=0x7f81d000b330 comp rx=0 tx=0).stop 2023-09-23T04:44:05.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 -- 172.21.15.5:0/300449883 shutdown_connections 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 --2- 172.21.15.5:0/300449883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d8109340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 --2- 172.21.15.5:0/300449883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8100f10 0x7f81d8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 --2- 172.21.15.5:0/300449883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8102060 0x7f81d81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 -- 172.21.15.5:0/300449883 >> 172.21.15.5:0/300449883 conn(0x7f81d80fc480 msgr2=0x7f81d80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 -- 172.21.15.5:0/300449883 shutdown_connections 2023-09-23T04:44:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.349+0000 7f81de9ff700 1 -- 172.21.15.5:0/300449883 wait complete. 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.350+0000 7f81de9ff700 1 Processor -- start 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.350+0000 7f81de9ff700 1 -- start start 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 0x7f81d8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 0x7f81d8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81a0830 con 0x7f81d8102060 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81a09a0 con 0x7f81d8100f10 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81de9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81a0b60 con 0x7f81d8102a20 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 0x7f81d8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 0x7f81d8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38134/0 (socket says 172.21.15.5:38134) 2023-09-23T04:44:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d77fe700 1 -- 172.21.15.5:0/1915705137 learned_addr learned my addr 172.21.15.5:0/1915705137 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 0x7f81d8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81dcf9c700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 -- 172.21.15.5:0/1915705137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 msgr2=0x7f81d81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 -- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 msgr2=0x7f81d8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 0x7f81d8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81d7fff700 1 -- 172.21.15.5:0/1915705137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0009c40 con 0x7f81d8100f10 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.351+0000 7f81dcf9c700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81d7fff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 0x7f81d8198cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81c800e9c0 tx=0x7f81c800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c800c9c0 con 0x7f81d8100f10 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d81a0e40 con 0x7f81d8100f10 2023-09-23T04:44:05.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81c8010430 con 0x7f81d8100f10 2023-09-23T04:44:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8018680 con 0x7f81d8100f10 2023-09-23T04:44:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.352+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d81a1270 con 0x7f81d8100f10 2023-09-23T04:44:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.353+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d810c6a0 con 0x7f81d8100f10 2023-09-23T04:44:05.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.355+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f81c8010860 con 0x7f81d8100f10 2023-09-23T04:44:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.357+0000 7f81d57fa700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 0x7f81c00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.357+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81c8014070 con 0x7f81d8100f10 2023-09-23T04:44:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.357+0000 7f81d77fe700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 0x7f81c00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.357+0000 7f81d77fe700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 0x7f81c00a3a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009d20 tx=0x7f81cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.359+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f81c8096710 con 0x7f81d8100f10 2023-09-23T04:44:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.571+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81d81a1460 con 0x7f81d8100f10 2023-09-23T04:44:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.572+0000 7f81d57fa700 1 -- 172.21.15.5:0/1915705137 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81c8099d30 con 0x7f81d8100f10 2023-09-23T04:44:05.573 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:44:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 msgr2=0x7f81c00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 0x7f81c00a3a60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009d20 tx=0x7f81cc009260 comp rx=0 tx=0).stop 2023-09-23T04:44:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 msgr2=0x7f81d8198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 0x7f81d8198cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81c800e9c0 tx=0x7f81c800ecd0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 shutdown_connections 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81c00a15a0 0x7f81c00a3a60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81d8102a20 0x7f81d81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81d8100f10 0x7f81d8198cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 --2- 172.21.15.5:0/1915705137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81d8102060 0x7f81d8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 >> 172.21.15.5:0/1915705137 conn(0x7f81d80fc480 msgr2=0x7f81d8107980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.574+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 shutdown_connections 2023-09-23T04:44:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.575+0000 7f81de9ff700 1 -- 172.21.15.5:0/1915705137 wait complete. 2023-09-23T04:44:05.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 -- 172.21.15.5:0/1965281237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 msgr2=0x7f31a4109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1965281237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a4109340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f319c00b0a0 tx=0x7f319c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 -- 172.21.15.5:0/1965281237 shutdown_connections 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1965281237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a4109340 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1965281237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 0x7f31a4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1965281237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 0x7f31a41024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 -- 172.21.15.5:0/1965281237 >> 172.21.15.5:0/1965281237 conn(0x7f31a40fc480 msgr2=0x7f31a40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 -- 172.21.15.5:0/1965281237 shutdown_connections 2023-09-23T04:44:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.800+0000 7f31a8dff700 1 -- 172.21.15.5:0/1965281237 wait complete. 2023-09-23T04:44:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 Processor -- start 2023-09-23T04:44:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 -- start start 2023-09-23T04:44:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 0x7f31a4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 0x7f31a4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a419e860 con 0x7f31a4102a20 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a419e9d0 con 0x7f31a4102060 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.801+0000 7f31a8dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a419eb90 con 0x7f31a4100f10 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 0x7f31a4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38142/0 (socket says 172.21.15.5:38142) 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 -- 172.21.15.5:0/1618357691 learned_addr learned my addr 172.21.15.5:0/1618357691 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a1d9b700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 0x7f31a4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 -- 172.21.15.5:0/1618357691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 msgr2=0x7f31a4196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 0x7f31a4196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 -- 172.21.15.5:0/1618357691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 msgr2=0x7f31a4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 0x7f31a4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 -- 172.21.15.5:0/1618357691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c009d00 con 0x7f31a4102a20 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a2d9d700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f319c00d010 tx=0x7f319c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c00e040 con 0x7f31a4102a20 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a419ee10 con 0x7f31a4102a20 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.802+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f319c003e10 con 0x7f31a4102a20 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.803+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c0052a0 con 0x7f31a4102a20 2023-09-23T04:44:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.803+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a419f180 con 0x7f31a4102a20 2023-09-23T04:44:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.804+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a41083e0 con 0x7f31a4102a20 2023-09-23T04:44:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.806+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f319c01d480 con 0x7f31a4102a20 2023-09-23T04:44:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.807+0000 7f31937fe700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 0x7f318c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.807+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f319c0ce7f0 con 0x7f31a4102a20 2023-09-23T04:44:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.807+0000 7f31a259c700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 0x7f318c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.808+0000 7f31a259c700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 0x7f318c0a3a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31a41981f0 tx=0x7f3194009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:05.809+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f319c097c20 con 0x7f31a4102a20 2023-09-23T04:44:05.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3085450551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:05.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1915705137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3085450551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1915705137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.022+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31a410c9b0 con 0x7f31a4102a20 2023-09-23T04:44:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.023+0000 7f31937fe700 1 -- 172.21.15.5:0/1618357691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f319c09b240 con 0x7f31a4102a20 2023-09-23T04:44:06.023 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:44:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 msgr2=0x7f318c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 0x7f318c0a3a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31a41981f0 tx=0x7f3194009260 comp rx=0 tx=0).stop 2023-09-23T04:44:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 msgr2=0x7f31a419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f319c00d010 tx=0x7f319c00bc80 comp rx=0 tx=0).stop 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 shutdown_connections 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f318c0a15a0 0x7f318c0a3a60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31a4102a20 0x7f31a419e290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31a4100f10 0x7f31a4196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 --2- 172.21.15.5:0/1618357691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31a4102060 0x7f31a4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 >> 172.21.15.5:0/1618357691 conn(0x7f31a40fc480 msgr2=0x7f31a4105520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.025+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 shutdown_connections 2023-09-23T04:44:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.026+0000 7f31a8dff700 1 -- 172.21.15.5:0/1618357691 wait complete. 2023-09-23T04:44:06.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:44:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3085450551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1915705137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:44:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 -- 172.21.15.5:0/3913837091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 msgr2=0x7fc44c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 --2- 172.21.15.5:0/3913837091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c1024e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc438009990 tx=0x7fc438009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 -- 172.21.15.5:0/3913837091 shutdown_connections 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 --2- 172.21.15.5:0/3913837091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c102a20 0x7fc44c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 --2- 172.21.15.5:0/3913837091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c100f10 0x7fc44c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 --2- 172.21.15.5:0/3913837091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c1024e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 -- 172.21.15.5:0/3913837091 >> 172.21.15.5:0/3913837091 conn(0x7fc44c0fc480 msgr2=0x7fc44c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 -- 172.21.15.5:0/3913837091 shutdown_connections 2023-09-23T04:44:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.249+0000 7fc452bff700 1 -- 172.21.15.5:0/3913837091 wait complete. 2023-09-23T04:44:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.250+0000 7fc452bff700 1 Processor -- start 2023-09-23T04:44:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 -- start start 2023-09-23T04:44:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c100f10 0x7fc44c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1a0960 con 0x7fc44c102a20 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1a0aa0 con 0x7fc44c102060 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc452bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1a0c30 con 0x7fc44c100f10 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc45119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc45119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38146/0 (socket says 172.21.15.5:38146) 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc45119c700 1 -- 172.21.15.5:0/1926530376 learned_addr learned my addr 172.21.15.5:0/1926530376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc45099b700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c100f10 0x7fc44c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.251+0000 7fc44bfff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 -- 172.21.15.5:0/1926530376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c100f10 msgr2=0x7fc44c198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c100f10 0x7fc44c198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 -- 172.21.15.5:0/1926530376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 msgr2=0x7fc44c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 -- 172.21.15.5:0/1926530376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380095f0 con 0x7fc44c102a20 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc45119c700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc44400ef30 tx=0x7fc44400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444018070 con 0x7fc44c102a20 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44400f040 con 0x7fc44c102a20 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c1a0eb0 con 0x7fc44c102a20 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400cd00 con 0x7fc44c102a20 2023-09-23T04:44:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.252+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1a1330 con 0x7fc44c102a20 2023-09-23T04:44:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.253+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c10c760 con 0x7fc44c102a20 2023-09-23T04:44:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.255+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc444003a00 con 0x7fc44c102a20 2023-09-23T04:44:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.257+0000 7fc449ffb700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 0x7fc43c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.257+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc4440ccbf0 con 0x7fc44c102a20 2023-09-23T04:44:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.257+0000 7fc45099b700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 0x7fc43c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.258+0000 7fc45099b700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 0x7fc43c0a3ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc440009c60 tx=0x7fc440009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.261+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc444096020 con 0x7fc44c102a20 2023-09-23T04:44:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.472+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc44c1a1520 con 0x7fc44c102a20 2023-09-23T04:44:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.472+0000 7fc449ffb700 1 -- 172.21.15.5:0/1926530376 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc444099640 con 0x7fc44c102a20 2023-09-23T04:44:06.473 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 msgr2=0x7fc43c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 0x7fc43c0a3ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc440009c60 tx=0x7fc440009260 comp rx=0 tx=0).stop 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 msgr2=0x7fc44c1a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc44400ef30 tx=0x7fc44400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 shutdown_connections 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc43c0a1620 0x7fc43c0a3ae0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c102a20 0x7fc44c1a0390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc44c100f10 0x7fc44c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 --2- 172.21.15.5:0/1926530376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc44c102060 0x7fc44c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 >> 172.21.15.5:0/1926530376 conn(0x7fc44c0fc480 msgr2=0x7fc44c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 shutdown_connections 2023-09-23T04:44:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.475+0000 7fc452bff700 1 -- 172.21.15.5:0/1926530376 wait complete. 2023-09-23T04:44:06.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:44:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 -- 172.21.15.5:0/603578342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340100eb0 msgr2=0x7f23401012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 --2- 172.21.15.5:0/603578342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340100eb0 0x7f23401012d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2330009990 tx=0x7f2330009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 -- 172.21.15.5:0/603578342 shutdown_connections 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 --2- 172.21.15.5:0/603578342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340102a50 0x7f2340109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 --2- 172.21.15.5:0/603578342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340100eb0 0x7f23401012d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 --2- 172.21.15.5:0/603578342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 0x7f2340102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 -- 172.21.15.5:0/603578342 >> 172.21.15.5:0/603578342 conn(0x7f23400fc470 msgr2=0x7f23400fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 -- 172.21.15.5:0/603578342 shutdown_connections 2023-09-23T04:44:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.700+0000 7f23457ff700 1 -- 172.21.15.5:0/603578342 wait complete. 2023-09-23T04:44:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.701+0000 7f23457ff700 1 Processor -- start 2023-09-23T04:44:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 -- start start 2023-09-23T04:44:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 0x7f2340199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401a09a0 con 0x7f2340102a50 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401a0ae0 con 0x7f2340100eb0 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f23457ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401a0c70 con 0x7f2340102090 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233f7fe700 1 -- 172.21.15.5:0/44669241 learned_addr learned my addr 172.21.15.5:0/44669241 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52928/0 (socket says 172.21.15.5:52928) 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.702+0000 7f233e7fc700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 0x7f2340199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 -- 172.21.15.5:0/44669241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 msgr2=0x7f2340199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 0x7f2340199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 -- 172.21.15.5:0/44669241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 msgr2=0x7f2340198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 -- 172.21.15.5:0/44669241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23300095f0 con 0x7f2340102a50 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233effd700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f233f7fe700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f233800ef30 tx=0x7f233800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338018070 con 0x7f2340102a50 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f233800f040 con 0x7f2340102a50 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23401a0ef0 con 0x7f2340102a50 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233800cd00 con 0x7f2340102a50 2023-09-23T04:44:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.703+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23401a1320 con 0x7f2340102a50 2023-09-23T04:44:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.704+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234010c770 con 0x7f2340102a50 2023-09-23T04:44:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.706+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f2338003a00 con 0x7f2340102a50 2023-09-23T04:44:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.708+0000 7f2327fff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 0x7f23280a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.708+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f23380ccc90 con 0x7f2340102a50 2023-09-23T04:44:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.708+0000 7f233effd700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 0x7f23280a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.708+0000 7f233effd700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 0x7f23280a3a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f233000b2e0 tx=0x7f2330005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.712+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f23380960c0 con 0x7f2340102a50 2023-09-23T04:44:06.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:06 smithi005 ceph-mon[97638]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:06.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:06 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1618357691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:06.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:06 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1926530376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.926+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23401a1510 con 0x7f2340102a50 2023-09-23T04:44:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.927+0000 7f2327fff700 1 -- 172.21.15.5:0/44669241 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f23380996e0 con 0x7f2340102a50 2023-09-23T04:44:06.928 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.929+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 msgr2=0x7f23280a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.929+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 0x7f23280a3a60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f233000b2e0 tx=0x7f2330005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.929+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 msgr2=0x7f23401a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.929+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f233800ef30 tx=0x7f233800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 shutdown_connections 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f23280a15a0 0x7f23280a3a60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102a50 0x7f23401a03d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2340100eb0 0x7f2340198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 --2- 172.21.15.5:0/44669241 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2340102090 0x7f2340199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 >> 172.21.15.5:0/44669241 conn(0x7f23400fc470 msgr2=0x7f23400fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 shutdown_connections 2023-09-23T04:44:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:06.930+0000 7f23457ff700 1 -- 172.21.15.5:0/44669241 wait complete. 2023-09-23T04:44:06.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T04:44:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:06 smithi069 ceph-mon[133931]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:06 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1618357691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:06 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1926530376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 -- 172.21.15.5:0/976503749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c100eb0 msgr2=0x7fb89c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/976503749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c100eb0 0x7fb89c1012d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009990 tx=0x7fb88c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 -- 172.21.15.5:0/976503749 shutdown_connections 2023-09-23T04:44:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/976503749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 0x7fb89c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/976503749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c100eb0 0x7fb89c1012d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/976503749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c102090 0x7fb89c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 -- 172.21.15.5:0/976503749 >> 172.21.15.5:0/976503749 conn(0x7fb89c0fc470 msgr2=0x7fb89c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 -- 172.21.15.5:0/976503749 shutdown_connections 2023-09-23T04:44:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.152+0000 7fb8a27ff700 1 -- 172.21.15.5:0/976503749 wait complete. 2023-09-23T04:44:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 Processor -- start 2023-09-23T04:44:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 -- start start 2023-09-23T04:44:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 0x7fb89c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1a09b0 con 0x7fb89c102090 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.153+0000 7fb8a27ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1a0af0 con 0x7fb89c100eb0 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb8a27ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1a0c80 con 0x7fb89c102a50 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38184/0 (socket says 172.21.15.5:38184) 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89b7fe700 1 -- 172.21.15.5:0/2383226351 learned_addr learned my addr 172.21.15.5:0/2383226351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52936/0 (socket says 172.21.15.5:52936) 2023-09-23T04:44:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb8a0d9c700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 0x7fb89c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 -- 172.21.15.5:0/2383226351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 msgr2=0x7fb89c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 0x7fb89c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 -- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 msgr2=0x7fb89c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89bfff700 1 -- 172.21.15.5:0/2383226351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0095f0 con 0x7fb89c100eb0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.154+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb89bfff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0098e0 tx=0x7fb88c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c01d070 con 0x7fb89c100eb0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb88c021410 con 0x7fb89c100eb0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c1a0f00 con 0x7fb89c100eb0 2023-09-23T04:44:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c005720 con 0x7fb89c100eb0 2023-09-23T04:44:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.155+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c1a12c0 con 0x7fb89c100eb0 2023-09-23T04:44:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.156+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c10c770 con 0x7fb89c100eb0 2023-09-23T04:44:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.158+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fb88c021580 con 0x7fb89c100eb0 2023-09-23T04:44:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.159+0000 7fb8997fa700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 0x7fb8840a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.159+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb88c0ce3f0 con 0x7fb89c100eb0 2023-09-23T04:44:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.160+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 0x7fb8840a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:06 smithi118 ceph-mon[134297]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:06 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1618357691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:44:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:06 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1926530376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:44:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.160+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 0x7fb8840a3ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb890005950 tx=0x7fb890009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.161+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb88c097820 con 0x7fb89c100eb0 2023-09-23T04:44:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.391+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb89c1a14b0 con 0x7fb89c100eb0 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.392+0000 7fb8997fa700 1 -- 172.21.15.5:0/2383226351 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+580 (secure 0 0 0) 0x7fb88c026080 con 0x7fb89c100eb0 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3461224331 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1525019998 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:07.393 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:07.394 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2022747688 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:07.394 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2627772518 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:07.394 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3431143820 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:07.394 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T04:44:07.394 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1934869339 2023-09-24T04:37:23.553088+0000 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.394+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 msgr2=0x7fb8840a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.394+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 0x7fb8840a3ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb890005950 tx=0x7fb890009260 comp rx=0 tx=0).stop 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 msgr2=0x7fb89c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0098e0 tx=0x7fb88c009580 comp rx=0 tx=0).stop 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 shutdown_connections 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb8840a1620 0x7fb8840a3ae0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb89c102a50 0x7fb89c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb89c100eb0 0x7fb89c198e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 --2- 172.21.15.5:0/2383226351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c102090 0x7fb89c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 >> 172.21.15.5:0/2383226351 conn(0x7fb89c0fc470 msgr2=0x7fb89c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 shutdown_connections 2023-09-23T04:44:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:07.395+0000 7fb8a27ff700 1 -- 172.21.15.5:0/2383226351 wait complete. 2023-09-23T04:44:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2023-09-23T04:44:07.413 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2023-09-23T04:44:07.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:07.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-23T04:44:07.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[152035]: fuse finished with error 0 and tester_r 0 2023-09-23T04:44:07.484 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T04:44:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:07 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/44669241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:08.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:07 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2383226351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:07 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/44669241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:07 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2383226351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:07 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/44669241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:44:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:07 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2383226351' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T04:44:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:08 smithi005 ceph-mon[97638]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:08 smithi118 ceph-mon[134297]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:08 smithi069 ceph-mon[133931]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-09-23T04:44:11.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:10 smithi005 ceph-mon[97638]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:10 smithi118 ceph-mon[134297]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:10 smithi069 ceph-mon[133931]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:12 smithi005 ceph-mon[97638]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:13.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:12 smithi118 ceph-mon[134297]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:12 smithi069 ceph-mon[133931]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:13.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:44:13.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:44:13.586 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-23T04:44:13.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:44:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 -- 172.21.15.5:0/3287923863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614101980 msgr2=0x7f9614103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 --2- 172.21.15.5:0/3287923863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614101980 0x7f9614103d70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9604009990 tx=0x7f9604009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 -- 172.21.15.5:0/3287923863 shutdown_connections 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 --2- 172.21.15.5:0/3287923863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614106be0 0x7f9614109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 --2- 172.21.15.5:0/3287923863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614101980 0x7f9614103d70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 --2- 172.21.15.5:0/3287923863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 0x7f96141066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 -- 172.21.15.5:0/3287923863 >> 172.21.15.5:0/3287923863 conn(0x7f96140fb350 msgr2=0x7f96140fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 -- 172.21.15.5:0/3287923863 shutdown_connections 2023-09-23T04:44:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.861+0000 7f96189ff700 1 -- 172.21.15.5:0/3287923863 wait complete. 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.862+0000 7f96189ff700 1 Processor -- start 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.862+0000 7f96189ff700 1 -- start start 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 0x7f9614196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 0x7f96141970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419e730 con 0x7f9614106be0 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419e870 con 0x7f9614101980 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f96189ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419ea00 con 0x7f96141042b0 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36490/0 (socket says 172.21.15.5:36490) 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 -- 172.21.15.5:0/3265836139 learned_addr learned my addr 172.21.15.5:0/3265836139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f961259c700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 0x7f9614196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9611d9b700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 0x7f96141970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 -- 172.21.15.5:0/3265836139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 msgr2=0x7f96141970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 0x7f96141970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 -- 172.21.15.5:0/3265836139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 msgr2=0x7f9614196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 0x7f9614196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f9612d9d700 1 -- 172.21.15.5:0/3265836139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040095f0 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.863+0000 7f961259c700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 0x7f9614196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f9612d9d700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f960c00ef30 tx=0x7f960c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c018070 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960c00f040 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00cd00 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961419ec80 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.864+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961419f140 con 0x7f9614106be0 2023-09-23T04:44:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.865+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96140fced0 con 0x7f9614106be0 2023-09-23T04:44:13.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.867+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f960c003a00 con 0x7f9614106be0 2023-09-23T04:44:13.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.868+0000 7f96037fe700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 0x7f95fc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:13.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.869+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f960c0ccc70 con 0x7f9614106be0 2023-09-23T04:44:13.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.869+0000 7f961259c700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 0x7f95fc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:13.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.869+0000 7f961259c700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 0x7f95fc0a3a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f960400b2e0 tx=0x7f9604005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:13.873+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f960c0960a0 con 0x7f9614106be0 2023-09-23T04:44:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.085+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f961410c730 con 0x7f9614106be0 2023-09-23T04:44:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.087+0000 7f96037fe700 1 -- 172.21.15.5:0/3265836139 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f960c0996c0 con 0x7f9614106be0 2023-09-23T04:44:14.088 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:14.088 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 msgr2=0x7f95fc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 0x7f95fc0a3a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f960400b2e0 tx=0x7f9604005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 msgr2=0x7f961419e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f960c00ef30 tx=0x7f960c00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 shutdown_connections 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f95fc0a15a0 0x7f95fc0a3a60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9614106be0 0x7f961419e160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9614101980 0x7f9614196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 --2- 172.21.15.5:0/3265836139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f96141042b0 0x7f96141970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 >> 172.21.15.5:0/3265836139 conn(0x7f96140fb350 msgr2=0x7f96141025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 shutdown_connections 2023-09-23T04:44:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.090+0000 7f96189ff700 1 -- 172.21.15.5:0/3265836139 wait complete. 2023-09-23T04:44:14.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:44:14.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 -- 172.21.15.5:0/3744347695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc106c80 msgr2=0x7f92dc109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 --2- 172.21.15.5:0/3744347695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc106c80 0x7f92dc109070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009f90 tx=0x7f92d400b330 comp rx=0 tx=0).stop 2023-09-23T04:44:14.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 -- 172.21.15.5:0/3744347695 shutdown_connections 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 --2- 172.21.15.5:0/3744347695 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc106c80 0x7f92dc109070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 --2- 172.21.15.5:0/3744347695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc101a20 0x7f92dc103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 --2- 172.21.15.5:0/3744347695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc104350 0x7f92dc106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 -- 172.21.15.5:0/3744347695 >> 172.21.15.5:0/3744347695 conn(0x7f92dc0fb350 msgr2=0x7f92dc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 -- 172.21.15.5:0/3744347695 shutdown_connections 2023-09-23T04:44:14.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.312+0000 7f92e31ff700 1 -- 172.21.15.5:0/3744347695 wait complete. 2023-09-23T04:44:14.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.313+0000 7f92e31ff700 1 Processor -- start 2023-09-23T04:44:14.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.313+0000 7f92e31ff700 1 -- start start 2023-09-23T04:44:14.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc104350 0x7f92dc0ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc106c80 0x7f92dc0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1006b0 con 0x7f92dc101a20 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc100820 con 0x7f92dc106c80 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e31ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc1a0d20 con 0x7f92dc104350 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36502/0 (socket says 172.21.15.5:36502) 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 -- 172.21.15.5:0/2483833644 learned_addr learned my addr 172.21.15.5:0/2483833644 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e179c700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc106c80 0x7f92dc0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92dbfff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc104350 0x7f92dc0ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 -- 172.21.15.5:0/2483833644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc104350 msgr2=0x7f92dc0ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc104350 0x7f92dc0ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 -- 172.21.15.5:0/2483833644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc106c80 msgr2=0x7f92dc0fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc106c80 0x7f92dc0fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.314+0000 7f92e0f9b700 1 -- 172.21.15.5:0/2483833644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009c40 con 0x7f92dc101a20 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92e0f9b700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92d000e9f0 tx=0x7f92d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0004a30 con 0x7f92dc101a20 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92d000cc80 con 0x7f92dc101a20 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc1a0fa0 con 0x7f92dc101a20 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0005600 con 0x7f92dc101a20 2023-09-23T04:44:14.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.315+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc1a13d0 con 0x7f92dc101a20 2023-09-23T04:44:14.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.316+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc10c750 con 0x7f92dc101a20 2023-09-23T04:44:14.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.318+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f92d0004c20 con 0x7f92dc101a20 2023-09-23T04:44:14.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.319+0000 7f92d9ffb700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 0x7f92cc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.320+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f92d0014070 con 0x7f92dc101a20 2023-09-23T04:44:14.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.320+0000 7f92dbfff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 0x7f92cc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.320+0000 7f92dbfff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 0x7f92cc0a3a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009bf0 tx=0x7f92c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:14.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.323+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f92d0096920 con 0x7f92dc101a20 2023-09-23T04:44:14.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.539+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92dc1a1650 con 0x7f92dc101a20 2023-09-23T04:44:14.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.541+0000 7f92d9ffb700 1 -- 172.21.15.5:0/2483833644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7f92d0099f40 con 0x7f92dc101a20 2023-09-23T04:44:14.542 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:14.542 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:14.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 msgr2=0x7f92cc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 0x7f92cc0a3a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009bf0 tx=0x7f92c8009260 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 msgr2=0x7f92dc101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92d000e9f0 tx=0x7f92d000ed00 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 shutdown_connections 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.543+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f92cc0a15a0 0x7f92cc0a3a60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92dc106c80 0x7f92dc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92dc101a20 0x7f92dc101390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 --2- 172.21.15.5:0/2483833644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f92dc104350 0x7f92dc0ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 >> 172.21.15.5:0/2483833644 conn(0x7f92dc0fb350 msgr2=0x7f92dc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:14.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 shutdown_connections 2023-09-23T04:44:14.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.544+0000 7f92e31ff700 1 -- 172.21.15.5:0/2483833644 wait complete. 2023-09-23T04:44:14.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T04:44:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 -- 172.21.15.5:0/4200377286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 msgr2=0x7ff0040732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 --2- 172.21.15.5:0/4200377286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff0040732b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feff4009990 tx=0x7feff4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 -- 172.21.15.5:0/4200377286 shutdown_connections 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 --2- 172.21.15.5:0/4200377286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004108dd0 0x7ff00410b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 --2- 172.21.15.5:0/4200377286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff0040732b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 --2- 172.21.15.5:0/4200377286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004073880 0x7ff004073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 -- 172.21.15.5:0/4200377286 >> 172.21.15.5:0/4200377286 conn(0x7ff0040fc480 msgr2=0x7ff0040fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 -- 172.21.15.5:0/4200377286 shutdown_connections 2023-09-23T04:44:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.772+0000 7ff0093ff700 1 -- 172.21.15.5:0/4200377286 wait complete. 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.773+0000 7ff0093ff700 1 Processor -- start 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.773+0000 7ff0093ff700 1 -- start start 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 0x7ff004198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004108dd0 0x7ff0041a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a0980 con 0x7ff004074e60 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a0ac0 con 0x7ff004073880 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0093ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a0c50 con 0x7ff004108dd0 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 0x7ff004198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36526/0 (socket says 172.21.15.5:36526) 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0027fc700 1 -- 172.21.15.5:0/562454455 learned_addr learned my addr 172.21.15.5:0/562454455 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0037fe700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004108dd0 0x7ff0041a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 -- 172.21.15.5:0/562454455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004108dd0 msgr2=0x7ff0041a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004108dd0 0x7ff0041a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 -- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 msgr2=0x7ff004199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff002ffd700 1 -- 172.21.15.5:0/562454455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40095f0 con 0x7ff004073880 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.774+0000 7ff0027fc700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7ff002ffd700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 0x7ff004198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feff400b680 tx=0x7feff4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff401d070 con 0x7ff004073880 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff4021410 con 0x7ff004073880 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041a0ed0 con 0x7ff004073880 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4017650 con 0x7ff004073880 2023-09-23T04:44:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.775+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a1290 con 0x7ff004073880 2023-09-23T04:44:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.776+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff004107480 con 0x7ff004073880 2023-09-23T04:44:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.778+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7feff4021810 con 0x7ff004073880 2023-09-23T04:44:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.779+0000 7fefebfff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 0x7fefec0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.779+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feff40ce530 con 0x7ff004073880 2023-09-23T04:44:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.780+0000 7ff0027fc700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 0x7fefec0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.780+0000 7ff0027fc700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 0x7fefec0a3ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feff8005950 tx=0x7feff8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:14.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:14.782+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7feff4097960 con 0x7ff004073880 2023-09-23T04:44:14.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:14 smithi005 ceph-mon[97638]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:14.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:14 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3265836139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:14.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:14 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2483833644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.037+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff00402d180 con 0x7ff004073880 2023-09-23T04:44:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:14 smithi118 ceph-mon[134297]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:14 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3265836139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:14 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2483833644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:14 smithi069 ceph-mon[133931]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:14 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3265836139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:14 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2483833644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.760+0000 7fefebfff700 1 -- 172.21.15.5:0/562454455 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7feff409af80 con 0x7ff004073880 2023-09-23T04:44:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 msgr2=0x7fefec0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 0x7fefec0a3ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feff8005950 tx=0x7feff8009260 comp rx=0 tx=0).stop 2023-09-23T04:44:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 msgr2=0x7ff004198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 0x7ff004198e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feff400b680 tx=0x7feff4009580 comp rx=0 tx=0).stop 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 shutdown_connections 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fefec0a1620 0x7fefec0a3ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff004108dd0 0x7ff0041a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff004073880 0x7ff004198e00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 --2- 172.21.15.5:0/562454455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff004074e60 0x7ff004199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 >> 172.21.15.5:0/562454455 conn(0x7ff0040fc480 msgr2=0x7ff0040fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 shutdown_connections 2023-09-23T04:44:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.763+0000 7ff0093ff700 1 -- 172.21.15.5:0/562454455 wait complete. 2023-09-23T04:44:15.786 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-09-23T04:44:15.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-09-23T04:44:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 -- 172.21.15.5:0/2833098629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350102a20 msgr2=0x7f4350109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 --2- 172.21.15.5:0/2833098629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350102a20 0x7f4350109340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f434800b0a0 tx=0x7f434800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 -- 172.21.15.5:0/2833098629 shutdown_connections 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 --2- 172.21.15.5:0/2833098629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350102a20 0x7f4350109340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 --2- 172.21.15.5:0/2833098629 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350100f10 0x7f4350101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 --2- 172.21.15.5:0/2833098629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 -- 172.21.15.5:0/2833098629 >> 172.21.15.5:0/2833098629 conn(0x7f43500fc480 msgr2=0x7f43500fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 -- 172.21.15.5:0/2833098629 shutdown_connections 2023-09-23T04:44:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.996+0000 7f43587ff700 1 -- 172.21.15.5:0/2833098629 wait complete. 2023-09-23T04:44:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.997+0000 7f43587ff700 1 Processor -- start 2023-09-23T04:44:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 -- start start 2023-09-23T04:44:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350100f10 0x7f4350198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350102a20 0x7f43501a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501a0a30 con 0x7f4350102060 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501a0b70 con 0x7f4350100f10 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f43587ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43501a0d00 con 0x7f4350102a20 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f4355d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f4355d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36532/0 (socket says 172.21.15.5:36532) 2023-09-23T04:44:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f4355d9a700 1 -- 172.21.15.5:0/89704859 learned_addr learned my addr 172.21.15.5:0/89704859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f435659b700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350100f10 0x7f4350198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f4356d9c700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350102a20 0x7f43501a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.998+0000 7f4355d9a700 1 -- 172.21.15.5:0/89704859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350102a20 msgr2=0x7f43501a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f4355d9a700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350102a20 0x7f43501a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f4355d9a700 1 -- 172.21.15.5:0/89704859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350100f10 msgr2=0x7f4350198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f4355d9a700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350100f10 0x7f4350198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f4355d9a700 1 -- 172.21.15.5:0/89704859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009520 con 0x7f4350102060 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f4355d9a700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f434000eb30 tx=0x7f434000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340009880 con 0x7f4350102060 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43400044e0 con 0x7f4350102060 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348009d00 con 0x7f4350102060 2023-09-23T04:44:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340003e90 con 0x7f4350102060 2023-09-23T04:44:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:15.999+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501a1320 con 0x7f4350102060 2023-09-23T04:44:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.000+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435010c710 con 0x7f4350102060 2023-09-23T04:44:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.002+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4340004000 con 0x7f4350102060 2023-09-23T04:44:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.003+0000 7f43477fe700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 0x7f433c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.004+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4340014070 con 0x7f4350102060 2023-09-23T04:44:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.004+0000 7f435659b700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 0x7f433c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.004+0000 7f435659b700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 0x7f433c0a3ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f434c009c60 tx=0x7f434c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.008+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f43400968c0 con 0x7f4350102060 2023-09-23T04:44:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/562454455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:15 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:15 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/562454455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:15 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:15 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/562454455' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:15 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-23T04:44:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.266+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f43501a0ec0 con 0x7f4350102060 2023-09-23T04:44:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.763+0000 7f43477fe700 1 -- 172.21.15.5:0/89704859 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4340099ee0 con 0x7f4350102060 2023-09-23T04:44:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.765+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 msgr2=0x7f433c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.765+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 0x7f433c0a3ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f434c009c60 tx=0x7f434c009260 comp rx=0 tx=0).stop 2023-09-23T04:44:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.765+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 msgr2=0x7f43501993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.765+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f434000eb30 tx=0x7f434000ee40 comp rx=0 tx=0).stop 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 shutdown_connections 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f433c0a1620 0x7f433c0a3ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4350102a20 0x7f43501a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4350100f10 0x7f4350198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 --2- 172.21.15.5:0/89704859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350102060 0x7f43501993f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 >> 172.21.15.5:0/89704859 conn(0x7f43500fc480 msgr2=0x7f4350107980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 shutdown_connections 2023-09-23T04:44:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:16.766+0000 7f43587ff700 1 -- 172.21.15.5:0/89704859 wait complete. 2023-09-23T04:44:16.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.000+0000 7fc95cfff700 1 -- 172.21.15.5:0/3789847619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 msgr2=0x7fc9581024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.000+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3789847619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc9581024e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009990 tx=0x7fc94c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 -- 172.21.15.5:0/3789847619 shutdown_connections 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3789847619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958102a20 0x7fc958109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3789847619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958100f10 0x7fc958101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3789847619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc9581024e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 -- 172.21.15.5:0/3789847619 >> 172.21.15.5:0/3789847619 conn(0x7fc9580fc480 msgr2=0x7fc9580fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 -- 172.21.15.5:0/3789847619 shutdown_connections 2023-09-23T04:44:17.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.001+0000 7fc95cfff700 1 -- 172.21.15.5:0/3789847619 wait complete. 2023-09-23T04:44:17.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 Processor -- start 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 -- start start 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958100f10 0x7fc958074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958102a20 0x7fc958073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958073ef0 con 0x7fc958102060 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958074060 con 0x7fc958102a20 2023-09-23T04:44:17.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc95cfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9581a0c20 con 0x7fc958100f10 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc955d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc955d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36544/0 (socket says 172.21.15.5:36544) 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.002+0000 7fc955d9b700 1 -- 172.21.15.5:0/3679420940 learned_addr learned my addr 172.21.15.5:0/3679420940 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc956d9d700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958102a20 0x7fc958073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc95659c700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958100f10 0x7fc958074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 -- 172.21.15.5:0/3679420940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958100f10 msgr2=0x7fc958074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958100f10 0x7fc958074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 -- 172.21.15.5:0/3679420940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958102a20 msgr2=0x7fc958073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958102a20 0x7fc958073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 -- 172.21.15.5:0/3679420940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c0095f0 con 0x7fc958102060 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc955d9b700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c010 tx=0x7fc94c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c01d070 con 0x7fc958102060 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc94c022470 con 0x7fc958102060 2023-09-23T04:44:17.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9581a0ea0 con 0x7fc958102060 2023-09-23T04:44:17.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.003+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c017650 con 0x7fc958102060 2023-09-23T04:44:17.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.004+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9581a1260 con 0x7fc958102060 2023-09-23T04:44:17.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.005+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc958107de0 con 0x7fc958102060 2023-09-23T04:44:17.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.007+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc94c0225e0 con 0x7fc958102060 2023-09-23T04:44:17.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.008+0000 7fc9477fe700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 0x7fc9400a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:17.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.008+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc94c0cecc0 con 0x7fc958102060 2023-09-23T04:44:17.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.008+0000 7fc95659c700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 0x7fc9400a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:17.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.009+0000 7fc95659c700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 0x7fc9400a3a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc948009bf0 tx=0x7fc948009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:17.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.010+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc94c0980f0 con 0x7fc958102060 2023-09-23T04:44:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:16 smithi005 ceph-mon[97638]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:16 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T04:44:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:16 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:16 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-23T04:44:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:16 smithi118 ceph-mon[134297]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:16 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T04:44:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:16 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:16 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-23T04:44:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:16 smithi069 ceph-mon[133931]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-09-23T04:44:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:16 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-23T04:44:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:16 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:16 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-23T04:44:17.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.270+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc95810c9b0 con 0x7fc958102060 2023-09-23T04:44:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.773+0000 7fc9477fe700 1 -- 172.21.15.5:0/3679420940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc94c09b710 con 0x7fc958102060 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 msgr2=0x7fc9400a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 0x7fc9400a3a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc948009bf0 tx=0x7fc948009260 comp rx=0 tx=0).stop 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 msgr2=0x7fc958073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c010 tx=0x7fc94c004730 comp rx=0 tx=0).stop 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 shutdown_connections 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc9400a15a0 0x7fc9400a3a60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc958102a20 0x7fc958073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc958100f10 0x7fc958074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 --2- 172.21.15.5:0/3679420940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc958102060 0x7fc958073210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 >> 172.21.15.5:0/3679420940 conn(0x7fc9580fc480 msgr2=0x7fc9580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 shutdown_connections 2023-09-23T04:44:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:17.778+0000 7fc95cfff700 1 -- 172.21.15.5:0/3679420940 wait complete. 2023-09-23T04:44:17.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.014+0000 7fed36dff700 1 -- 172.21.15.5:0/2358878215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30100eb0 msgr2=0x7fed301012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.014+0000 7fed36dff700 1 --2- 172.21.15.5:0/2358878215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30100eb0 0x7fed301012d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fed24009990 tx=0x7fed24009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 -- 172.21.15.5:0/2358878215 shutdown_connections 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 --2- 172.21.15.5:0/2358878215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30102a50 0x7fed30109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 --2- 172.21.15.5:0/2358878215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30100eb0 0x7fed301012d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 --2- 172.21.15.5:0/2358878215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102090 0x7fed30102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 -- 172.21.15.5:0/2358878215 >> 172.21.15.5:0/2358878215 conn(0x7fed300fc470 msgr2=0x7fed300fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 -- 172.21.15.5:0/2358878215 shutdown_connections 2023-09-23T04:44:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.015+0000 7fed36dff700 1 -- 172.21.15.5:0/2358878215 wait complete. 2023-09-23T04:44:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 Processor -- start 2023-09-23T04:44:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 -- start start 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30100eb0 0x7fed30198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102a50 0x7fed301a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301a0990 con 0x7fed30102090 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301a0ad0 con 0x7fed30102a50 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed36dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301a0c60 con 0x7fed30100eb0 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed2ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed3539c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102a50 0x7fed301a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed2ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36566/0 (socket says 172.21.15.5:36566) 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.016+0000 7fed2ffff700 1 -- 172.21.15.5:0/2330976141 learned_addr learned my addr 172.21.15.5:0/2330976141 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed34b9b700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30100eb0 0x7fed30198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 -- 172.21.15.5:0/2330976141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30100eb0 msgr2=0x7fed30198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30100eb0 0x7fed30198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 -- 172.21.15.5:0/2330976141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102a50 msgr2=0x7fed301a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102a50 0x7fed301a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 -- 172.21.15.5:0/2330976141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240095f0 con 0x7fed30102090 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2ffff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00e970 tx=0x7fed1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c009790 con 0x7fed30102090 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed1c00c7e0 con 0x7fed30102090 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.017+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed301a0f40 con 0x7fed30102090 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.018+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c003c70 con 0x7fed30102090 2023-09-23T04:44:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.018+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301a13c0 con 0x7fed30102090 2023-09-23T04:44:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.019+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed30193560 con 0x7fed30102090 2023-09-23T04:44:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.021+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fed1c00cc50 con 0x7fed30102090 2023-09-23T04:44:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.022+0000 7fed2dffb700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.022+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fed1c014070 con 0x7fed30102090 2023-09-23T04:44:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.022+0000 7fed34b9b700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.023+0000 7fed34b9b700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b2e0 tx=0x7fed24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.026+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fed1c096670 con 0x7fed30102090 2023-09-23T04:44:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-23T04:44:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:17 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-23T04:44:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:17 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/89704859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: fsmap cephfs:1 {0=g=up:active} 11 up:standby 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: fsmap cephfs:2 {0=g=up:active,1=i=up:creating} 10 up:standby 2023-09-23T04:44:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-09-23T04:44:18.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:17 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-23T04:44:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.288+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fed3010c770 con 0x7fed30102090 2023-09-23T04:44:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.782+0000 7fed2dffb700 1 -- 172.21.15.5:0/2330976141 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fed1c003dd0 con 0x7fed30102090 2023-09-23T04:44:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 msgr2=0x7fed200a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b2e0 tx=0x7fed24005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 msgr2=0x7fed30199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00e970 tx=0x7fed1c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:44:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 shutdown_connections 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fed200a15a0 0x7fed200a3a60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed30102a50 0x7fed301a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fed30100eb0 0x7fed30198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 --2- 172.21.15.5:0/2330976141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed30102090 0x7fed30199350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 >> 172.21.15.5:0/2330976141 conn(0x7fed300fc470 msgr2=0x7fed301079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.785+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 shutdown_connections 2023-09-23T04:44:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:18.786+0000 7fed36dff700 1 -- 172.21.15.5:0/2330976141 wait complete. 2023-09-23T04:44:18.812 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-09-23T04:44:18.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-23T04:44:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 -- 172.21.15.5:0/3351487104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 msgr2=0x7f32bc103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 --2- 172.21.15.5:0/3351487104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc103d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009990 tx=0x7f32ac009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 -- 172.21.15.5:0/3351487104 shutdown_connections 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 --2- 172.21.15.5:0/3351487104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 --2- 172.21.15.5:0/3351487104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc103d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 --2- 172.21.15.5:0/3351487104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 0x7f32bc106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 -- 172.21.15.5:0/3351487104 >> 172.21.15.5:0/3351487104 conn(0x7f32bc0fb3a0 msgr2=0x7f32bc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 -- 172.21.15.5:0/3351487104 shutdown_connections 2023-09-23T04:44:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.022+0000 7f32c1dff700 1 -- 172.21.15.5:0/3351487104 wait complete. 2023-09-23T04:44:19.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 Processor -- start 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 -- start start 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 0x7f32bc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19e750 con 0x7f32bc106bd0 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19e890 con 0x7f32bc1042a0 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.024+0000 7f32c1dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19ea20 con 0x7f32bc101970 2023-09-23T04:44:19.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37920/0 (socket says 172.21.15.5:37920) 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 -- 172.21.15.5:0/2330091465 learned_addr learned my addr 172.21.15.5:0/2330091465 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32baffd700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 0x7f32bc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 -- 172.21.15.5:0/2330091465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 msgr2=0x7f32bc197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 0x7f32bc197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 -- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 msgr2=0x7f32bc19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 -- 172.21.15.5:0/2330091465 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0009520 con 0x7f32bc101970 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bbfff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.025+0000 7f32bb7fe700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b680 tx=0x7f32ac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.026+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac01d070 con 0x7f32bc101970 2023-09-23T04:44:19.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.026+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32ac022470 con 0x7f32bc101970 2023-09-23T04:44:19.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.026+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac0095f0 con 0x7f32bc101970 2023-09-23T04:44:19.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.026+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac017650 con 0x7f32bc101970 2023-09-23T04:44:19.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.026+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc19ef30 con 0x7f32bc101970 2023-09-23T04:44:19.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.027+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc0fcf20 con 0x7f32bc101970 2023-09-23T04:44:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.029+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f32ac0225e0 con 0x7f32bc101970 2023-09-23T04:44:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.031+0000 7f32b8ff9700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 0x7f32a40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:19.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.031+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f32ac0cece0 con 0x7f32bc101970 2023-09-23T04:44:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.031+0000 7f32baffd700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 0x7f32a40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:19.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.031+0000 7f32baffd700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 0x7f32a40a3a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009df0 tx=0x7f32b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:19.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.033+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f32ac098110 con 0x7f32bc101970 2023-09-23T04:44:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:active 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-09-23T04:44:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:active 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-09-23T04:44:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3679420940' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] up:active 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: fsmap cephfs:2 {0=g=up:active,1=i=up:active} 10 up:standby 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:creating} 9 up:standby 2023-09-23T04:44:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: daemon mds.b is now active in filesystem cephfs as rank 2 2023-09-23T04:44:19.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-23T04:44:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.282+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f32bc02d1d0 con 0x7f32a40a15a0 2023-09-23T04:44:19.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:44:19.288+0000 7f8c97c43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:44:19.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:44:19.288+0000 7f8c97c43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:44:19.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:44:19.288+0000 7f8c97c43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:44:19.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:44:19.288+0000 7f8c97c43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:44:19.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:44:19.289+0000 7f8c97c43700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:44:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.838+0000 7f32b8ff9700 1 -- 172.21.15.5:0/2330091465 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32bc02d1d0 con 0x7f32a40a15a0 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 msgr2=0x7f32a40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 0x7f32a40a3a60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009df0 tx=0x7f32b0009260 comp rx=0 tx=0).stop 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 msgr2=0x7f32bc196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b680 tx=0x7f32ac005740 comp rx=0 tx=0).stop 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 shutdown_connections 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32a40a15a0 0x7f32a40a3a60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32bc106bd0 0x7f32bc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32bc101970 0x7f32bc196bd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 --2- 172.21.15.5:0/2330091465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1042a0 0x7f32bc197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 >> 172.21.15.5:0/2330091465 conn(0x7f32bc0fb3a0 msgr2=0x7f32bc1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 shutdown_connections 2023-09-23T04:44:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:19.841+0000 7f32c1dff700 1 -- 172.21.15.5:0/2330091465 wait complete. 2023-09-23T04:44:19.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-23T04:44:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 -- 172.21.15.5:0/2633521333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934102a50 msgr2=0x7f5934109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 --2- 172.21.15.5:0/2633521333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934102a50 0x7f5934109370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f592c009f90 tx=0x7f592c00b330 comp rx=0 tx=0).stop 2023-09-23T04:44:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 -- 172.21.15.5:0/2633521333 shutdown_connections 2023-09-23T04:44:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 --2- 172.21.15.5:0/2633521333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934102a50 0x7f5934109370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 --2- 172.21.15.5:0/2633521333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934100eb0 0x7f59341012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 --2- 172.21.15.5:0/2633521333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 0x7f5934102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 -- 172.21.15.5:0/2633521333 >> 172.21.15.5:0/2633521333 conn(0x7f59340fc470 msgr2=0x7f59340fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 -- 172.21.15.5:0/2633521333 shutdown_connections 2023-09-23T04:44:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.068+0000 7f593bdff700 1 -- 172.21.15.5:0/2633521333 wait complete. 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.069+0000 7f593bdff700 1 Processor -- start 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.069+0000 7f593bdff700 1 -- start start 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 0x7f5934199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934102a50 0x7f59341a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59341a0a80 con 0x7f5934100eb0 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59341a0bc0 con 0x7f5934102090 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593bdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59341a0d50 con 0x7f5934102a50 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36634/0 (socket says 172.21.15.5:36634) 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 -- 172.21.15.5:0/1659801221 learned_addr learned my addr 172.21.15.5:0/1659801221 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593a39c700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934102a50 0x7f59341a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f593939a700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 0x7f5934199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 -- 172.21.15.5:0/1659801221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934102a50 msgr2=0x7f59341a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934102a50 0x7f59341a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 -- 172.21.15.5:0/1659801221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 msgr2=0x7f5934199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 0x7f5934199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.070+0000 7f5939b9b700 1 -- 172.21.15.5:0/1659801221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300095f0 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f5939b9b700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f593000ebf0 tx=0x7f593000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930009a10 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593000cc90 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c009c40 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930018750 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.071+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59341a12c0 con 0x7f5934100eb0 2023-09-23T04:44:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.072+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593410c770 con 0x7f5934100eb0 2023-09-23T04:44:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.076+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f59300188b0 con 0x7f5934100eb0 2023-09-23T04:44:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.077+0000 7f592affd700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 0x7f59200a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.077+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5930014070 con 0x7f5934100eb0 2023-09-23T04:44:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.078+0000 7f593939a700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 0x7f59200a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.078+0000 7f593939a700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 0x7f59200a3a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5934100bd0 tx=0x7f5924009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.080+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f59300971c0 con 0x7f5934100eb0 2023-09-23T04:44:20.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T04:44:20.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:active 2023-09-23T04:44:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-23T04:44:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2023-09-23T04:44:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:creating} 8 up:standby 2023-09-23T04:44:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:19 smithi005 ceph-mon[97638]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:active 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:creating} 8 up:standby 2023-09-23T04:44:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:19 smithi118 ceph-mon[134297]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2330976141' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] up:active 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: fsmap cephfs:3 {0=g=up:active,1=i=up:active,2=b=up:active} 9 up:standby 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:creating} 8 up:standby 2023-09-23T04:44:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:19 smithi069 ceph-mon[133931]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-09-23T04:44:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.326+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f59341090e0 con 0x7f59200a15a0 2023-09-23T04:44:20.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.330+0000 7f592affd700 1 -- 172.21.15.5:0/1659801221 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f59341090e0 con 0x7f59200a15a0 2023-09-23T04:44:20.331 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/_nogroup/sv_0/fc6e15b6-efbc-4286-8452-71a1606f48cc 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 msgr2=0x7f59200a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 0x7f59200a3a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5934100bd0 tx=0x7f5924009260 comp rx=0 tx=0).stop 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 msgr2=0x7f5934198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f593000ebf0 tx=0x7f593000ef00 comp rx=0 tx=0).stop 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 shutdown_connections 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59200a15a0 0x7f59200a3a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5934102a50 0x7f59341a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5934100eb0 0x7f5934198f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 --2- 172.21.15.5:0/1659801221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5934102090 0x7f5934199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 >> 172.21.15.5:0/1659801221 conn(0x7f59340fc470 msgr2=0x7f59341079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 shutdown_connections 2023-09-23T04:44:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.333+0000 7f593bdff700 1 -- 172.21.15.5:0/1659801221 wait complete. 2023-09-23T04:44:20.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-23T04:44:20.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 -- 172.21.15.5:0/2127828909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8074e60 msgr2=0x7f90a80732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 --2- 172.21.15.5:0/2127828909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8074e60 0x7f90a80732b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f90980098f0 tx=0x7f9098009c00 comp rx=0 tx=0).stop 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 -- 172.21.15.5:0/2127828909 shutdown_connections 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 --2- 172.21.15.5:0/2127828909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 0x7f90a810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 --2- 172.21.15.5:0/2127828909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8074e60 0x7f90a80732b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 --2- 172.21.15.5:0/2127828909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8073880 0x7f90a8073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 -- 172.21.15.5:0/2127828909 >> 172.21.15.5:0/2127828909 conn(0x7f90a80fc470 msgr2=0x7f90a80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 -- 172.21.15.5:0/2127828909 shutdown_connections 2023-09-23T04:44:20.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.563+0000 7f90acdff700 1 -- 172.21.15.5:0/2127828909 wait complete. 2023-09-23T04:44:20.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.565+0000 7f90acdff700 1 Processor -- start 2023-09-23T04:44:20.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.565+0000 7f90acdff700 1 -- start start 2023-09-23T04:44:20.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.565+0000 7f90acdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8073880 0x7f90a8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.565+0000 7f90acdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90acdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 0x7f90a81a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90acdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81a0990 con 0x7f90a8074e60 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90acdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81a0ad0 con 0x7f90a8073880 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90acdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a81a0c60 con 0x7f90a8108dc0 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36660/0 (socket says 172.21.15.5:36660) 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 -- 172.21.15.5:0/153789439 learned_addr learned my addr 172.21.15.5:0/153789439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90a659c700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8073880 0x7f90a8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f90a6d9d700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 0x7f90a81a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 -- 172.21.15.5:0/153789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 msgr2=0x7f90a81a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 0x7f90a81a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 -- 172.21.15.5:0/153789439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8073880 msgr2=0x7f90a8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8073880 0x7f90a8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 -- 172.21.15.5:0/153789439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900095f0 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.566+0000 7f909dd9b700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f909000ebf0 tx=0x7f909000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.567+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090009a10 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.567+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f909000cc90 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.567+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090018750 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.567+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9098009520 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.567+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a81a1210 con 0x7f90a8074e60 2023-09-23T04:44:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.568+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8107470 con 0x7f90a8074e60 2023-09-23T04:44:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.570+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f90900188b0 con 0x7f90a8074e60 2023-09-23T04:44:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.571+0000 7f909ffff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 0x7f90940a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.572+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9090014070 con 0x7f90a8074e60 2023-09-23T04:44:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.572+0000 7f90a659c700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 0x7f90940a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.572+0000 7f90a659c700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 0x7f90940a3a60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f909800fa40 tx=0x7f9098019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.574+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f90900971e0 con 0x7f90a8074e60 2023-09-23T04:44:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.819+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f90a802d210 con 0x7f90940a15a0 2023-09-23T04:44:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.842+0000 7f909ffff700 1 -- 172.21.15.5:0/153789439 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f90a802d210 con 0x7f90940a15a0 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 msgr2=0x7f90940a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 0x7f90940a3a60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f909800fa40 tx=0x7f9098019040 comp rx=0 tx=0).stop 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 msgr2=0x7f90a8199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f909000ebf0 tx=0x7f909000ef00 comp rx=0 tx=0).stop 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 shutdown_connections 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90940a15a0 0x7f90940a3a60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90a8108dc0 0x7f90a81a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a8073880 0x7f90a8198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 --2- 172.21.15.5:0/153789439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a8074e60 0x7f90a8199350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:20.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 >> 172.21.15.5:0/153789439 conn(0x7f90a80fc470 msgr2=0x7f90a80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:20.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 shutdown_connections 2023-09-23T04:44:20.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:20.845+0000 7f90acdff700 1 -- 172.21.15.5:0/153789439 wait complete. 2023-09-23T04:44:20.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-23T04:44:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 -- 172.21.15.5:0/585412252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 msgr2=0x7f2da4109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/585412252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da4109340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00b0a0 tx=0x7f2d9c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 -- 172.21.15.5:0/585412252 shutdown_connections 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/585412252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da4109340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/585412252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 0x7f2da4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/585412252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 0x7f2da41024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 -- 172.21.15.5:0/585412252 >> 172.21.15.5:0/585412252 conn(0x7f2da40fc480 msgr2=0x7f2da40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 -- 172.21.15.5:0/585412252 shutdown_connections 2023-09-23T04:44:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.070+0000 7f2dab9ff700 1 -- 172.21.15.5:0/585412252 wait complete. 2023-09-23T04:44:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.071+0000 7f2dab9ff700 1 Processor -- start 2023-09-23T04:44:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.071+0000 7f2dab9ff700 1 -- start start 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 0x7f2da4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 0x7f2da4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41a0a70 con 0x7f2da4102a20 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41a0bb0 con 0x7f2da4102060 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2dab9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da41a0d40 con 0x7f2da4100f10 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da9f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da9f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43812/0 (socket says 172.21.15.5:43812) 2023-09-23T04:44:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 0x7f2da4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da9f9c700 1 -- 172.21.15.5:0/3255383292 learned_addr learned my addr 172.21.15.5:0/3255383292 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da8f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 0x7f2da4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da9f9c700 1 -- 172.21.15.5:0/3255383292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 msgr2=0x7f2da4198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.072+0000 7f2da9f9c700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 0x7f2da4198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2da9f9c700 1 -- 172.21.15.5:0/3255383292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 msgr2=0x7f2da4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2da9f9c700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 0x7f2da4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2da9f9c700 1 -- 172.21.15.5:0/3255383292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c009d00 con 0x7f2da4102a20 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2da9f9c700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00d010 tx=0x7f2d9c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00e040 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9c003e10 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da41a0fc0 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0052a0 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.073+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da41a13e0 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.074+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da410c760 con 0x7f2da4102a20 2023-09-23T04:44:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.076+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f2d9c01d480 con 0x7f2da4102a20 2023-09-23T04:44:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.079+0000 7f2d9a7fc700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 0x7f2d900a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.079+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2d9c0ce730 con 0x7f2da4102a20 2023-09-23T04:44:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.079+0000 7f2da979b700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 0x7f2d900a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.080+0000 7f2da979b700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 0x7f2d900a3ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009c60 tx=0x7f2da0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.081+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2d9c097b60 con 0x7f2da4102a20 2023-09-23T04:44:21.154 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T04:44:20.707+0000 7fd91f0a5700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T04:44:20.707+0000 7fd91f0a5700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T04:44:20.707+0000 7fd91f0a5700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: pgmap v285: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:active 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T04:44:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:active} 8 up:standby 2023-09-23T04:44:21.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-23T04:44:21.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-23T04:44:21.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:active 2023-09-23T04:44:21.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:20 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: pgmap v285: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:active 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:active} 8 up:standby 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-23T04:44:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:active 2023-09-23T04:44:21.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:20 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:44:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: pgmap v285: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2023-09-23T04:44:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] up:active 2023-09-23T04:44:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: fsmap cephfs:4 {0=g=up:active,1=i=up:active,2=b=up:active,3=c=up:active} 8 up:standby 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] up:active 2023-09-23T04:44:21.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:20 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:44:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.333+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2da41090b0 con 0x7f2d900a1620 2023-09-23T04:44:21.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.337+0000 7f2d9a7fc700 1 -- 172.21.15.5:0/3255383292 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2da41090b0 con 0x7f2d900a1620 2023-09-23T04:44:21.337 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/_nogroup/sv_1/089c9db7-dbed-4b4e-aa4f-fbdfdd050c26 2023-09-23T04:44:21.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 msgr2=0x7f2d900a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 0x7f2d900a3ae0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009c60 tx=0x7f2da0009260 comp rx=0 tx=0).stop 2023-09-23T04:44:21.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 msgr2=0x7f2da41a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00d010 tx=0x7f2d9c00bc80 comp rx=0 tx=0).stop 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 shutdown_connections 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d900a1620 0x7f2d900a3ae0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da4102a20 0x7f2da41a04a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2da4100f10 0x7f2da4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 --2- 172.21.15.5:0/3255383292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2da4102060 0x7f2da4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 >> 172.21.15.5:0/3255383292 conn(0x7f2da40fc480 msgr2=0x7f2da4107990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 shutdown_connections 2023-09-23T04:44:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.340+0000 7f2dab9ff700 1 -- 172.21.15.5:0/3255383292 wait complete. 2023-09-23T04:44:21.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:44:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 -- 172.21.15.5:0/105133385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c074e00 msgr2=0x7f678c073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 --2- 172.21.15.5:0/105133385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c074e00 0x7f678c073250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6780009990 tx=0x7f6780009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 -- 172.21.15.5:0/105133385 shutdown_connections 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 --2- 172.21.15.5:0/105133385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c108db0 0x7f678c10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 --2- 172.21.15.5:0/105133385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c074e00 0x7f678c073250 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 --2- 172.21.15.5:0/105133385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 -- 172.21.15.5:0/105133385 >> 172.21.15.5:0/105133385 conn(0x7f678c0fc470 msgr2=0x7f678c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 -- 172.21.15.5:0/105133385 shutdown_connections 2023-09-23T04:44:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.562+0000 7f6792fff700 1 -- 172.21.15.5:0/105133385 wait complete. 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.563+0000 7f6792fff700 1 Processor -- start 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.563+0000 7f6792fff700 1 -- start start 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c074e00 0x7f678c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a0980 con 0x7f678c108db0 2023-09-23T04:44:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a0ac0 con 0x7f678c074e00 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6792fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1a0c50 con 0x7f678c073820 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43832/0 (socket says 172.21.15.5:43832) 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 -- 172.21.15.5:0/2773120495 learned_addr learned my addr 172.21.15.5:0/2773120495 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f678bfff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c074e00 0x7f678c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6790d9b700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 -- 172.21.15.5:0/2773120495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 msgr2=0x7f678c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 -- 172.21.15.5:0/2773120495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c074e00 msgr2=0x7f678c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c074e00 0x7f678c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f679159c700 1 -- 172.21.15.5:0/2773120495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009c40 con 0x7f678c108db0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.564+0000 7f6790d9b700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f679159c700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f678400c370 tx=0x7f678400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784017070 con 0x7f678c108db0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67800095f0 con 0x7f678c108db0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678400f040 con 0x7f678c108db0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400e050 con 0x7f678c108db0 2023-09-23T04:44:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.565+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c1a11a0 con 0x7f678c108db0 2023-09-23T04:44:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.566+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c1074b0 con 0x7f678c108db0 2023-09-23T04:44:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.568+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f6784004520 con 0x7f678c108db0 2023-09-23T04:44:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.569+0000 7f6789ffb700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 0x7f677c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.570+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f67840ccb10 con 0x7f678c108db0 2023-09-23T04:44:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.570+0000 7f6790d9b700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 0x7f677c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.570+0000 7f6790d9b700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 0x7f677c0a3a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f678000b2e0 tx=0x7f6780005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.573+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6784095eb0 con 0x7f678c108db0 2023-09-23T04:44:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.804+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f6784096280 con 0x7f678c108db0 2023-09-23T04:44:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.837+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f678c10e920 con 0x7f678c108db0 2023-09-23T04:44:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.838+0000 7f6789ffb700 1 -- 172.21.15.5:0/2773120495 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f678405ef60 con 0x7f678c108db0 2023-09-23T04:44:21.838 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:21.838 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:44:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 msgr2=0x7f677c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 0x7f677c0a3a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f678000b2e0 tx=0x7f6780005fd0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 msgr2=0x7f678c1a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f678400c370 tx=0x7f678400c680 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 shutdown_connections 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f677c0a15a0 0x7f677c0a3a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f678c108db0 0x7f678c1a03b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f678c073820 0x7f678c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 --2- 172.21.15.5:0/2773120495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f678c074e00 0x7f678c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 >> 172.21.15.5:0/2773120495 conn(0x7f678c0fc470 msgr2=0x7f678c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 shutdown_connections 2023-09-23T04:44:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:21.841+0000 7f6792fff700 1 -- 172.21.15.5:0/2773120495 wait complete. 2023-09-23T04:44:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:44:21.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:44:22.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.063+0000 7fa32e3ff700 1 -- 172.21.15.5:0/3057969704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328102070 msgr2=0x7fa3281024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.063+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/3057969704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328102070 0x7fa3281024f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009990 tx=0x7fa31c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.063+0000 7fa32e3ff700 1 -- 172.21.15.5:0/3057969704 shutdown_connections 2023-09-23T04:44:22.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.063+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/3057969704 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102a30 0x7fa328109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.063+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/3057969704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328100e90 0x7fa3281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.064+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/3057969704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328102070 0x7fa3281024f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.064+0000 7fa32e3ff700 1 -- 172.21.15.5:0/3057969704 >> 172.21.15.5:0/3057969704 conn(0x7fa3280fc450 msgr2=0x7fa3280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:22.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.064+0000 7fa32e3ff700 1 -- 172.21.15.5:0/3057969704 shutdown_connections 2023-09-23T04:44:22.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.064+0000 7fa32e3ff700 1 -- 172.21.15.5:0/3057969704 wait complete. 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 Processor -- start 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 -- start start 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 0x7fa328198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102070 0x7fa328199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a09a0 con 0x7fa328102a30 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a0ae0 con 0x7fa328100e90 2023-09-23T04:44:22.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32e3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281a0c70 con 0x7fa328102070 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.065+0000 7fa32c99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa327fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 0x7fa328198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43856/0 (socket says 172.21.15.5:43856) 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 -- 172.21.15.5:0/2500386372 learned_addr learned my addr 172.21.15.5:0/2500386372 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102070 0x7fa328199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 -- 172.21.15.5:0/2500386372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102070 msgr2=0x7fa328199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102070 0x7fa328199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 -- 172.21.15.5:0/2500386372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 msgr2=0x7fa328198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 0x7fa328198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 -- 172.21.15.5:0/2500386372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318009520 con 0x7fa328102a30 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa327fff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 0x7fa328198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32c99c700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c370 tx=0x7fa32000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320017070 con 0x7fa328102a30 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa320009c40 con 0x7fa328102a30 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa32000f040 con 0x7fa328102a30 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.066+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32000e050 con 0x7fa328102a30 2023-09-23T04:44:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.067+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c0095f0 con 0x7fa328102a30 2023-09-23T04:44:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.068+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32810c700 con 0x7fa328102a30 2023-09-23T04:44:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.070+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fa32000cb60 con 0x7fa328102a30 2023-09-23T04:44:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.071+0000 7fa3257fa700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 0x7fa3100a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.071+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa3200ccc40 con 0x7fa328102a30 2023-09-23T04:44:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.071+0000 7fa327fff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 0x7fa3100a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.072+0000 7fa327fff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 0x7fa3100a3af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa318009fd0 tx=0x7fa318009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.075+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa3200d3080 con 0x7fa328102a30 2023-09-23T04:44:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:21 smithi005 ceph-mon[97638]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:21 smithi118 ceph-mon[134297]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:21 smithi069 ceph-mon[133931]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:22.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.290+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa32810c910 con 0x7fa328102a30 2023-09-23T04:44:22.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.292+0000 7fa3257fa700 1 -- 172.21.15.5:0/2500386372 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23866 (secure 0 0 0) 0x7fa320095ee0 con 0x7fa328102a30 2023-09-23T04:44:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:22.292 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:43:52.237066+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.294+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 msgr2=0x7fa3100a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.294+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 0x7fa3100a3af0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa318009fd0 tx=0x7fa318009260 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.294+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 msgr2=0x7fa3281a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.294+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c370 tx=0x7fa32000c680 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.294+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 shutdown_connections 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa3100a1630 0x7fa3100a3af0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328102a30 0x7fa3281a03d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa328100e90 0x7fa328198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 --2- 172.21.15.5:0/2500386372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa328102070 0x7fa328199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 >> 172.21.15.5:0/2500386372 conn(0x7fa3280fc450 msgr2=0x7fa3280fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 shutdown_connections 2023-09-23T04:44:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.295+0000 7fa32e3ff700 1 -- 172.21.15.5:0/2500386372 wait complete. 2023-09-23T04:44:22.313 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T04:44:22.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T04:44:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 -- 172.21.15.5:0/2566599188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0102080 msgr2=0x7f44b0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2566599188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0102080 0x7f44b0102500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44a00098f0 tx=0x7f44a0009c00 comp rx=0 tx=0).stop 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 -- 172.21.15.5:0/2566599188 shutdown_connections 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2566599188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 0x7f44b0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2566599188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0100ea0 0x7f44b01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2566599188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0102080 0x7f44b0102500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 -- 172.21.15.5:0/2566599188 >> 172.21.15.5:0/2566599188 conn(0x7f44b00fc480 msgr2=0x7f44b00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 -- 172.21.15.5:0/2566599188 shutdown_connections 2023-09-23T04:44:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.518+0000 7f44b7bff700 1 -- 172.21.15.5:0/2566599188 wait complete. 2023-09-23T04:44:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.519+0000 7f44b7bff700 1 Processor -- start 2023-09-23T04:44:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.519+0000 7f44b7bff700 1 -- start start 2023-09-23T04:44:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0100ea0 0x7f44b0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 0x7f44b0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0073ef0 con 0x7f44b0102080 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0074060 con 0x7f44b0102a40 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b7bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b01a0c80 con 0x7f44b0100ea0 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43866/0 (socket says 172.21.15.5:43866) 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 -- 172.21.15.5:0/2324848868 learned_addr learned my addr 172.21.15.5:0/2324848868 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b619c700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 0x7f44b0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b599b700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0100ea0 0x7f44b0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 -- 172.21.15.5:0/2324848868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0100ea0 msgr2=0x7f44b0074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0100ea0 0x7f44b0074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 -- 172.21.15.5:0/2324848868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 msgr2=0x7f44b0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.520+0000 7f44b519a700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 0x7f44b0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44b519a700 1 -- 172.21.15.5:0/2324848868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0009520 con 0x7f44b0102080 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44b519a700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009fd0 tx=0x7f44a000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a001c070 con 0x7f44b0102080 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a000f7d0 con 0x7f44b0102080 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b01a0f00 con 0x7f44b0102080 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0003a20 con 0x7f44b0102080 2023-09-23T04:44:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.521+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b01a1270 con 0x7f44b0102080 2023-09-23T04:44:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.522+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b0107ea0 con 0x7f44b0102080 2023-09-23T04:44:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.524+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f44a0020030 con 0x7f44b0102080 2023-09-23T04:44:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.526+0000 7f44a6ffd700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 0x7f449c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.526+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44a00ce580 con 0x7f44b0102080 2023-09-23T04:44:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.526+0000 7f44b599b700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 0x7f449c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.527+0000 7f44b599b700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 0x7f449c0a3b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44b0101bf0 tx=0x7f44ac009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.528+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f44a0097850 con 0x7f44b0102080 2023-09-23T04:44:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.734+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44b002d1d0 con 0x7f449c0a16b0 2023-09-23T04:44:22.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.750+0000 7f44a6ffd700 1 -- 172.21.15.5:0/2324848868 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423025 (secure 0 0 0) 0x7f44b002d1d0 con 0x7f449c0a16b0 2023-09-23T04:44:22.756 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:22.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 msgr2=0x7f449c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 0x7f449c0a3b70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44b0101bf0 tx=0x7f44ac009310 comp rx=0 tx=0).stop 2023-09-23T04:44:22.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 msgr2=0x7f44b0073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009fd0 tx=0x7f44a000b790 comp rx=0 tx=0).stop 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 shutdown_connections 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f449c0a16b0 0x7f449c0a3b70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44b0102a40 0x7f44b0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f44b0100ea0 0x7f44b0074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 --2- 172.21.15.5:0/2324848868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b0102080 0x7f44b0073210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 >> 172.21.15.5:0/2324848868 conn(0x7f44b00fc480 msgr2=0x7f44b00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 shutdown_connections 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:22.755+0000 7f44b7bff700 1 -- 172.21.15.5:0/2324848868 wait complete. 2023-09-23T04:44:22.760 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:44:22.774 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-09-23T04:44:21.638017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599473,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":327,"num_write_kb":3001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335572,"kb_used_data":11360,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606252,"statfs":{"total":1151940427776,"available":1151596802048,"internally_reserved":0,"allocated":11632640,"data_stored":7365091,"data_compressed":249699,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4465,"num_objects":47,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":97,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717797+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:43:51.717797+0000","last_peered":"2023-09-23T04:43:51.717797+0000","last_clean":"2023-09-23T04:43:51.717797+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:43:51.717797+0000","last_undegraded":"2023-09-23T04:43:51.717797+0000","last_fullsized":"2023-09-23T04:43:51.717797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719824+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:43:51.719824+0000","last_peered":"2023-09-23T04:43:51.719824+0000","last_clean":"2023-09-23T04:43:51.719824+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:43:51.719824+0000","last_undegraded":"2023-09-23T04:43:51.719824+0000","last_fullsized":"2023-09-23T04:43:51.719824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719079+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:43:51.719079+0000","last_peered":"2023-09-23T04:43:51.719079+0000","last_clean":"2023-09-23T04:43:51.719079+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:43:51.719079+0000","last_undegraded":"2023-09-23T04:43:51.719079+0000","last_fullsized":"2023-09-23T04:43:51.719079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717311+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:43:51.717311+0000","last_peered":"2023-09-23T04:43:51.717311+0000","last_clean":"2023-09-23T04:43:51.717311+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:43:51.717311+0000","last_undegraded":"2023-09-23T04:43:51.717311+0000","last_fullsized":"2023-09-23T04:43:51.717311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717830+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:43:51.717830+0000","last_peered":"2023-09-23T04:43:51.717830+0000","last_clean":"2023-09-23T04:43:51.717830+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:43:51.717830+0000","last_undegraded":"2023-09-23T04:43:51.717830+0000","last_fullsized":"2023-09-23T04:43:51.717830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711621+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:43:51.711621+0000","last_peered":"2023-09-23T04:43:51.711621+0000","last_clean":"2023-09-23T04:43:51.711621+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:43:51.711621+0000","last_undegraded":"2023-09-23T04:43:51.711621+0000","last_fullsized":"2023-09-23T04:43:51.711621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711566+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:43:51.711566+0000","last_peered":"2023-09-23T04:43:51.711566+0000","last_clean":"2023-09-23T04:43:51.711566+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:43:51.711566+0000","last_undegraded":"2023-09-23T04:43:51.711566+0000","last_fullsized":"2023-09-23T04:43:51.711566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716735+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:43:51.716735+0000","last_peered":"2023-09-23T04:43:51.716735+0000","last_clean":"2023-09-23T04:43:51.716735+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:43:51.716735+0000","last_undegraded":"2023-09-23T04:43:51.716735+0000","last_fullsized":"2023-09-23T04:43:51.716735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716786+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:43:51.716786+0000","last_peered":"2023-09-23T04:43:51.716786+0000","last_clean":"2023-09-23T04:43:51.716786+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:43:51.716786+0000","last_undegraded":"2023-09-23T04:43:51.716786+0000","last_fullsized":"2023-09-23T04:43:51.716786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712252+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:43:51.712252+0000","last_peered":"2023-09-23T04:43:51.712252+0000","last_clean":"2023-09-23T04:43:51.712252+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:43:51.712252+0000","last_undegraded":"2023-09-23T04:43:51.712252+0000","last_fullsized":"2023-09-23T04:43:51.712252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712172+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:43:51.712172+0000","last_peered":"2023-09-23T04:43:51.712172+0000","last_clean":"2023-09-23T04:43:51.712172+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:43:51.712172+0000","last_undegraded":"2023-09-23T04:43:51.712172+0000","last_fullsized":"2023-09-23T04:43:51.712172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717442+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:43:51.717442+0000","last_peered":"2023-09-23T04:43:51.717442+0000","last_clean":"2023-09-23T04:43:51.717442+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:43:51.717442+0000","last_undegraded":"2023-09-23T04:43:51.717442+0000","last_fullsized":"2023-09-23T04:43:51.717442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711174+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:43:51.711174+0000","last_peered":"2023-09-23T04:43:51.711174+0000","last_clean":"2023-09-23T04:43:51.711174+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:43:51.711174+0000","last_undegraded":"2023-09-23T04:43:51.711174+0000","last_fullsized":"2023-09-23T04:43:51.711174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718910+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:43:51.718910+0000","last_peered":"2023-09-23T04:43:51.718910+0000","last_clean":"2023-09-23T04:43:51.718910+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:43:51.718910+0000","last_undegraded":"2023-09-23T04:43:51.718910+0000","last_fullsized":"2023-09-23T04:43:51.718910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719366+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:43:51.719366+0000","last_peered":"2023-09-23T04:43:51.719366+0000","last_clean":"2023-09-23T04:43:51.719366+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:43:51.719366+0000","last_undegraded":"2023-09-23T04:43:51.719366+0000","last_fullsized":"2023-09-23T04:43:51.719366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716266+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:43:51.716266+0000","last_peered":"2023-09-23T04:43:51.716266+0000","last_clean":"2023-09-23T04:43:51.716266+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:43:51.716266+0000","last_undegraded":"2023-09-23T04:43:51.716266+0000","last_fullsized":"2023-09-23T04:43:51.716266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717700+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:43:51.717700+0000","last_peered":"2023-09-23T04:43:51.717700+0000","last_clean":"2023-09-23T04:43:51.717700+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:43:51.717700+0000","last_undegraded":"2023-09-23T04:43:51.717700+0000","last_fullsized":"2023-09-23T04:43:51.717700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717958+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:43:51.717958+0000","last_peered":"2023-09-23T04:43:51.717958+0000","last_clean":"2023-09-23T04:43:51.717958+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:43:51.717958+0000","last_undegraded":"2023-09-23T04:43:51.717958+0000","last_fullsized":"2023-09-23T04:43:51.717958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716415+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:43:51.716415+0000","last_peered":"2023-09-23T04:43:51.716415+0000","last_clean":"2023-09-23T04:43:51.716415+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:43:51.716415+0000","last_undegraded":"2023-09-23T04:43:51.716415+0000","last_fullsized":"2023-09-23T04:43:51.716415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747383+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:43:52.747383+0000","last_peered":"2023-09-23T04:43:52.747383+0000","last_clean":"2023-09-23T04:43:52.747383+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:43:52.747383+0000","last_undegraded":"2023-09-23T04:43:52.747383+0000","last_fullsized":"2023-09-23T04:43:52.747383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793602+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:44:17.793602+0000","last_peered":"2023-09-23T04:44:17.793602+0000","last_clean":"2023-09-23T04:44:17.793602+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:44:17.793602+0000","last_undegraded":"2023-09-23T04:44:17.793602+0000","last_fullsized":"2023-09-23T04:44:17.793602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717484+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:43:51.717484+0000","last_peered":"2023-09-23T04:43:51.717484+0000","last_clean":"2023-09-23T04:43:51.717484+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:43:51.717484+0000","last_undegraded":"2023-09-23T04:43:51.717484+0000","last_fullsized":"2023-09-23T04:43:51.717484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748086+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:43:52.748086+0000","last_peered":"2023-09-23T04:43:52.748086+0000","last_clean":"2023-09-23T04:43:52.748086+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:43:52.748086+0000","last_undegraded":"2023-09-23T04:43:52.748086+0000","last_fullsized":"2023-09-23T04:43:52.748086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748662+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:43:52.748662+0000","last_peered":"2023-09-23T04:43:52.748662+0000","last_clean":"2023-09-23T04:43:52.748662+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:43:52.748662+0000","last_undegraded":"2023-09-23T04:43:52.748662+0000","last_fullsized":"2023-09-23T04:43:52.748662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718258+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:43:51.718258+0000","last_peered":"2023-09-23T04:43:51.718258+0000","last_clean":"2023-09-23T04:43:51.718258+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:43:51.718258+0000","last_undegraded":"2023-09-23T04:43:51.718258+0000","last_fullsized":"2023-09-23T04:43:51.718258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748514+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:43:52.748514+0000","last_peered":"2023-09-23T04:43:52.748514+0000","last_clean":"2023-09-23T04:43:52.748514+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:43:52.748514+0000","last_undegraded":"2023-09-23T04:43:52.748514+0000","last_fullsized":"2023-09-23T04:43:52.748514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.791739+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:44:17.791739+0000","last_peered":"2023-09-23T04:44:17.791739+0000","last_clean":"2023-09-23T04:44:17.791739+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:44:17.791739+0000","last_undegraded":"2023-09-23T04:44:17.791739+0000","last_fullsized":"2023-09-23T04:44:17.791739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716893+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:43:51.716893+0000","last_peered":"2023-09-23T04:43:51.716893+0000","last_clean":"2023-09-23T04:43:51.716893+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:43:51.716893+0000","last_undegraded":"2023-09-23T04:43:51.716893+0000","last_fullsized":"2023-09-23T04:43:51.716893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747647+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:43:52.747647+0000","last_peered":"2023-09-23T04:43:52.747647+0000","last_clean":"2023-09-23T04:43:52.747647+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:43:52.747647+0000","last_undegraded":"2023-09-23T04:43:52.747647+0000","last_fullsized":"2023-09-23T04:43:52.747647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718945+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:43:51.718945+0000","last_peered":"2023-09-23T04:43:51.718945+0000","last_clean":"2023-09-23T04:43:51.718945+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:43:51.718945+0000","last_undegraded":"2023-09-23T04:43:51.718945+0000","last_fullsized":"2023-09-23T04:43:51.718945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717196+0000","last_change":"2023-09-23T04:43:47.591420+0000","last_active":"2023-09-23T04:43:51.717196+0000","last_peered":"2023-09-23T04:43:51.717196+0000","last_clean":"2023-09-23T04:43:51.717196+0000","last_became_active":"2023-09-23T04:43:47.591121+0000","last_became_peered":"2023-09-23T04:43:47.591121+0000","last_unstale":"2023-09-23T04:43:51.717196+0000","last_undegraded":"2023-09-23T04:43:51.717196+0000","last_fullsized":"2023-09-23T04:43:51.717196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:52:14.710768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747924+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:43:52.747924+0000","last_peered":"2023-09-23T04:43:52.747924+0000","last_clean":"2023-09-23T04:43:52.747924+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:43:52.747924+0000","last_undegraded":"2023-09-23T04:43:52.747924+0000","last_fullsized":"2023-09-23T04:43:52.747924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.783110+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:44:16.783110+0000","last_peered":"2023-09-23T04:44:16.783110+0000","last_clean":"2023-09-23T04:44:16.783110+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:44:16.783110+0000","last_undegraded":"2023-09-23T04:44:16.783110+0000","last_fullsized":"2023-09-23T04:44:16.783110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716947+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:43:51.716947+0000","last_peered":"2023-09-23T04:43:51.716947+0000","last_clean":"2023-09-23T04:43:51.716947+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:43:51.716947+0000","last_undegraded":"2023-09-23T04:43:51.716947+0000","last_fullsized":"2023-09-23T04:43:51.716947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747510+0000","last_change":"2023-09-23T04:43:51.735548+0000","last_active":"2023-09-23T04:43:52.747510+0000","last_peered":"2023-09-23T04:43:52.747510+0000","last_clean":"2023-09-23T04:43:52.747510+0000","last_became_active":"2023-09-23T04:43:51.735048+0000","last_became_peered":"2023-09-23T04:43:51.735048+0000","last_unstale":"2023-09-23T04:43:52.747510+0000","last_undegraded":"2023-09-23T04:43:52.747510+0000","last_fullsized":"2023-09-23T04:43:52.747510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:15:25.552028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793643+0000","last_change":"2023-09-23T04:43:46.586528+0000","last_active":"2023-09-23T04:44:17.793643+0000","last_peered":"2023-09-23T04:44:17.793643+0000","last_clean":"2023-09-23T04:44:17.793643+0000","last_became_active":"2023-09-23T04:43:46.586207+0000","last_became_peered":"2023-09-23T04:43:46.586207+0000","last_unstale":"2023-09-23T04:44:17.793643+0000","last_undegraded":"2023-09-23T04:44:17.793643+0000","last_fullsized":"2023-09-23T04:44:17.793643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:53:43.466578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718509+0000","last_change":"2023-09-23T04:43:47.602346+0000","last_active":"2023-09-23T04:43:51.718509+0000","last_peered":"2023-09-23T04:43:51.718509+0000","last_clean":"2023-09-23T04:43:51.718509+0000","last_became_active":"2023-09-23T04:43:47.601998+0000","last_became_peered":"2023-09-23T04:43:47.601998+0000","last_unstale":"2023-09-23T04:43:51.718509+0000","last_undegraded":"2023-09-23T04:43:51.718509+0000","last_fullsized":"2023-09-23T04:43:51.718509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:57:33.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749525+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:43:52.749525+0000","last_peered":"2023-09-23T04:43:52.749525+0000","last_clean":"2023-09-23T04:43:52.749525+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:43:52.749525+0000","last_undegraded":"2023-09-23T04:43:52.749525+0000","last_fullsized":"2023-09-23T04:43:52.749525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.794124+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:44:17.794124+0000","last_peered":"2023-09-23T04:44:17.794124+0000","last_clean":"2023-09-23T04:44:17.794124+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:44:17.794124+0000","last_undegraded":"2023-09-23T04:44:17.794124+0000","last_fullsized":"2023-09-23T04:44:17.794124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718354+0000","last_change":"2023-09-23T04:43:47.591680+0000","last_active":"2023-09-23T04:43:51.718354+0000","last_peered":"2023-09-23T04:43:51.718354+0000","last_clean":"2023-09-23T04:43:51.718354+0000","last_became_active":"2023-09-23T04:43:47.590872+0000","last_became_peered":"2023-09-23T04:43:47.590872+0000","last_unstale":"2023-09-23T04:43:51.718354+0000","last_undegraded":"2023-09-23T04:43:51.718354+0000","last_fullsized":"2023-09-23T04:43:51.718354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:49:20.600268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.716175+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:43:52.716175+0000","last_peered":"2023-09-23T04:43:52.716175+0000","last_clean":"2023-09-23T04:43:52.716175+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:43:52.716175+0000","last_undegraded":"2023-09-23T04:43:52.716175+0000","last_fullsized":"2023-09-23T04:43:52.716175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.799922+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:44:18.799922+0000","last_peered":"2023-09-23T04:44:18.799922+0000","last_clean":"2023-09-23T04:44:18.799922+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:44:18.799922+0000","last_undegraded":"2023-09-23T04:44:18.799922+0000","last_fullsized":"2023-09-23T04:44:18.799922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716620+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:43:51.716620+0000","last_peered":"2023-09-23T04:43:51.716620+0000","last_clean":"2023-09-23T04:43:51.716620+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:43:51.716620+0000","last_undegraded":"2023-09-23T04:43:51.716620+0000","last_fullsized":"2023-09-23T04:43:51.716620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747876+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:43:52.747876+0000","last_peered":"2023-09-23T04:43:52.747876+0000","last_clean":"2023-09-23T04:43:52.747876+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:43:52.747876+0000","last_undegraded":"2023-09-23T04:43:52.747876+0000","last_fullsized":"2023-09-23T04:43:52.747876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779500+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:44:16.779500+0000","last_peered":"2023-09-23T04:44:16.779500+0000","last_clean":"2023-09-23T04:44:16.779500+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:44:16.779500+0000","last_undegraded":"2023-09-23T04:44:16.779500+0000","last_fullsized":"2023-09-23T04:44:16.779500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716901+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:43:51.716901+0000","last_peered":"2023-09-23T04:43:51.716901+0000","last_clean":"2023-09-23T04:43:51.716901+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:43:51.716901+0000","last_undegraded":"2023-09-23T04:43:51.716901+0000","last_fullsized":"2023-09-23T04:43:51.716901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747995+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:43:52.747995+0000","last_peered":"2023-09-23T04:43:52.747995+0000","last_clean":"2023-09-23T04:43:52.747995+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:43:52.747995+0000","last_undegraded":"2023-09-23T04:43:52.747995+0000","last_fullsized":"2023-09-23T04:43:52.747995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711940+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:43:51.711940+0000","last_peered":"2023-09-23T04:43:51.711940+0000","last_clean":"2023-09-23T04:43:51.711940+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:43:51.711940+0000","last_undegraded":"2023-09-23T04:43:51.711940+0000","last_fullsized":"2023-09-23T04:43:51.711940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718343+0000","last_change":"2023-09-23T04:43:47.596424+0000","last_active":"2023-09-23T04:43:51.718343+0000","last_peered":"2023-09-23T04:43:51.718343+0000","last_clean":"2023-09-23T04:43:51.718343+0000","last_became_active":"2023-09-23T04:43:47.595568+0000","last_became_peered":"2023-09-23T04:43:47.595568+0000","last_unstale":"2023-09-23T04:43:51.718343+0000","last_undegraded":"2023-09-23T04:43:51.718343+0000","last_fullsized":"2023-09-23T04:43:51.718343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:46.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748728+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:43:52.748728+0000","last_peered":"2023-09-23T04:43:52.748728+0000","last_clean":"2023-09-23T04:43:52.748728+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:43:52.748728+0000","last_undegraded":"2023-09-23T04:43:52.748728+0000","last_fullsized":"2023-09-23T04:43:52.748728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.778760+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:44:16.778760+0000","last_peered":"2023-09-23T04:44:16.778760+0000","last_clean":"2023-09-23T04:44:16.778760+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:44:16.778760+0000","last_undegraded":"2023-09-23T04:44:16.778760+0000","last_fullsized":"2023-09-23T04:44:16.778760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716787+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:43:51.716787+0000","last_peered":"2023-09-23T04:43:51.716787+0000","last_clean":"2023-09-23T04:43:51.716787+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:43:51.716787+0000","last_undegraded":"2023-09-23T04:43:51.716787+0000","last_fullsized":"2023-09-23T04:43:51.716787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748735+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:43:52.748735+0000","last_peered":"2023-09-23T04:43:52.748735+0000","last_clean":"2023-09-23T04:43:52.748735+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:43:52.748735+0000","last_undegraded":"2023-09-23T04:43:52.748735+0000","last_fullsized":"2023-09-23T04:43:52.748735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779678+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:44:16.779678+0000","last_peered":"2023-09-23T04:44:16.779678+0000","last_clean":"2023-09-23T04:44:16.779678+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:44:16.779678+0000","last_undegraded":"2023-09-23T04:44:16.779678+0000","last_fullsized":"2023-09-23T04:44:16.779678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718345+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:43:51.718345+0000","last_peered":"2023-09-23T04:43:51.718345+0000","last_clean":"2023-09-23T04:43:51.718345+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:43:51.718345+0000","last_undegraded":"2023-09-23T04:43:51.718345+0000","last_fullsized":"2023-09-23T04:43:51.718345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748704+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:43:52.748704+0000","last_peered":"2023-09-23T04:43:52.748704+0000","last_clean":"2023-09-23T04:43:52.748704+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:43:52.748704+0000","last_undegraded":"2023-09-23T04:43:52.748704+0000","last_fullsized":"2023-09-23T04:43:52.748704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.794618+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:44:17.794618+0000","last_peered":"2023-09-23T04:44:17.794618+0000","last_clean":"2023-09-23T04:44:17.794618+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:44:17.794618+0000","last_undegraded":"2023-09-23T04:44:17.794618+0000","last_fullsized":"2023-09-23T04:44:17.794618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.710991+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:43:51.710991+0000","last_peered":"2023-09-23T04:43:51.710991+0000","last_clean":"2023-09-23T04:43:51.710991+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:43:51.710991+0000","last_undegraded":"2023-09-23T04:43:51.710991+0000","last_fullsized":"2023-09-23T04:43:51.710991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749482+0000","last_change":"2023-09-23T04:43:51.731564+0000","last_active":"2023-09-23T04:43:52.749482+0000","last_peered":"2023-09-23T04:43:52.749482+0000","last_clean":"2023-09-23T04:43:52.749482+0000","last_became_active":"2023-09-23T04:43:51.731151+0000","last_became_peered":"2023-09-23T04:43:51.731151+0000","last_unstale":"2023-09-23T04:43:52.749482+0000","last_undegraded":"2023-09-23T04:43:52.749482+0000","last_fullsized":"2023-09-23T04:43:52.749482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.954294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.801133+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:44:18.801133+0000","last_peered":"2023-09-23T04:44:18.801133+0000","last_clean":"2023-09-23T04:44:18.801133+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:44:18.801133+0000","last_undegraded":"2023-09-23T04:44:18.801133+0000","last_fullsized":"2023-09-23T04:44:18.801133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711691+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:43:51.711691+0000","last_peered":"2023-09-23T04:43:51.711691+0000","last_clean":"2023-09-23T04:43:51.711691+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:43:51.711691+0000","last_undegraded":"2023-09-23T04:43:51.711691+0000","last_fullsized":"2023-09-23T04:43:51.711691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748136+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:43:52.748136+0000","last_peered":"2023-09-23T04:43:52.748136+0000","last_clean":"2023-09-23T04:43:52.748136+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:43:52.748136+0000","last_undegraded":"2023-09-23T04:43:52.748136+0000","last_fullsized":"2023-09-23T04:43:52.748136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.455290+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:44:20.455290+0000","last_peered":"2023-09-23T04:44:20.455290+0000","last_clean":"2023-09-23T04:44:20.455290+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:44:20.455290+0000","last_undegraded":"2023-09-23T04:44:20.455290+0000","last_fullsized":"2023-09-23T04:44:20.455290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711560+0000","last_change":"2023-09-23T04:43:47.584835+0000","last_active":"2023-09-23T04:43:51.711560+0000","last_peered":"2023-09-23T04:43:51.711560+0000","last_clean":"2023-09-23T04:43:51.711560+0000","last_became_active":"2023-09-23T04:43:47.584538+0000","last_became_peered":"2023-09-23T04:43:47.584538+0000","last_unstale":"2023-09-23T04:43:51.711560+0000","last_undegraded":"2023-09-23T04:43:51.711560+0000","last_fullsized":"2023-09-23T04:43:51.711560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:16.222318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713651+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:43:52.713651+0000","last_peered":"2023-09-23T04:43:52.713651+0000","last_clean":"2023-09-23T04:43:52.713651+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:43:52.713651+0000","last_undegraded":"2023-09-23T04:43:52.713651+0000","last_fullsized":"2023-09-23T04:43:52.713651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793701+0000","last_change":"2023-09-23T04:43:46.593590+0000","last_active":"2023-09-23T04:44:17.793701+0000","last_peered":"2023-09-23T04:44:17.793701+0000","last_clean":"2023-09-23T04:44:17.793701+0000","last_became_active":"2023-09-23T04:43:46.593301+0000","last_became_peered":"2023-09-23T04:43:46.593301+0000","last_unstale":"2023-09-23T04:44:17.793701+0000","last_undegraded":"2023-09-23T04:44:17.793701+0000","last_fullsized":"2023-09-23T04:44:17.793701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:44.113054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.720396+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:43:51.720396+0000","last_peered":"2023-09-23T04:43:51.720396+0000","last_clean":"2023-09-23T04:43:51.720396+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:43:51.720396+0000","last_undegraded":"2023-09-23T04:43:51.720396+0000","last_fullsized":"2023-09-23T04:43:51.720396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747767+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:43:52.747767+0000","last_peered":"2023-09-23T04:43:52.747767+0000","last_clean":"2023-09-23T04:43:52.747767+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:43:52.747767+0000","last_undegraded":"2023-09-23T04:43:52.747767+0000","last_fullsized":"2023-09-23T04:43:52.747767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716138+0000","last_change":"2023-09-23T04:43:46.592918+0000","last_active":"2023-09-23T04:43:51.716138+0000","last_peered":"2023-09-23T04:43:51.716138+0000","last_clean":"2023-09-23T04:43:51.716138+0000","last_became_active":"2023-09-23T04:43:46.592612+0000","last_became_peered":"2023-09-23T04:43:46.592612+0000","last_unstale":"2023-09-23T04:43:51.716138+0000","last_undegraded":"2023-09-23T04:43:51.716138+0000","last_fullsized":"2023-09-23T04:43:51.716138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:59:48.213727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.720629+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:43:51.720629+0000","last_peered":"2023-09-23T04:43:51.720629+0000","last_clean":"2023-09-23T04:43:51.720629+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:43:51.720629+0000","last_undegraded":"2023-09-23T04:43:51.720629+0000","last_fullsized":"2023-09-23T04:43:51.720629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713696+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:43:52.713696+0000","last_peered":"2023-09-23T04:43:52.713696+0000","last_clean":"2023-09-23T04:43:52.713696+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:43:52.713696+0000","last_undegraded":"2023-09-23T04:43:52.713696+0000","last_fullsized":"2023-09-23T04:43:52.713696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716830+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:43:51.716830+0000","last_peered":"2023-09-23T04:43:51.716830+0000","last_clean":"2023-09-23T04:43:51.716830+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:43:51.716830+0000","last_undegraded":"2023-09-23T04:43:51.716830+0000","last_fullsized":"2023-09-23T04:43:51.716830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711937+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:43:51.711937+0000","last_peered":"2023-09-23T04:43:51.711937+0000","last_clean":"2023-09-23T04:43:51.711937+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:43:51.711937+0000","last_undegraded":"2023-09-23T04:43:51.711937+0000","last_fullsized":"2023-09-23T04:43:51.711937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713477+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:43:52.713477+0000","last_peered":"2023-09-23T04:43:52.713477+0000","last_clean":"2023-09-23T04:43:52.713477+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:43:52.713477+0000","last_undegraded":"2023-09-23T04:43:52.713477+0000","last_fullsized":"2023-09-23T04:43:52.713477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711884+0000","last_change":"2023-09-23T04:43:46.589899+0000","last_active":"2023-09-23T04:43:51.711884+0000","last_peered":"2023-09-23T04:43:51.711884+0000","last_clean":"2023-09-23T04:43:51.711884+0000","last_became_active":"2023-09-23T04:43:46.589550+0000","last_became_peered":"2023-09-23T04:43:46.589550+0000","last_unstale":"2023-09-23T04:43:51.711884+0000","last_undegraded":"2023-09-23T04:43:51.711884+0000","last_fullsized":"2023-09-23T04:43:51.711884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:55:05.083411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711818+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:43:51.711818+0000","last_peered":"2023-09-23T04:43:51.711818+0000","last_clean":"2023-09-23T04:43:51.711818+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:43:51.711818+0000","last_undegraded":"2023-09-23T04:43:51.711818+0000","last_fullsized":"2023-09-23T04:43:51.711818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713648+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:43:52.713648+0000","last_peered":"2023-09-23T04:43:52.713648+0000","last_clean":"2023-09-23T04:43:52.713648+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:43:52.713648+0000","last_undegraded":"2023-09-23T04:43:52.713648+0000","last_fullsized":"2023-09-23T04:43:52.713648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.721201+0000","last_change":"2023-09-23T04:43:47.587791+0000","last_active":"2023-09-23T04:43:51.721201+0000","last_peered":"2023-09-23T04:43:51.721201+0000","last_clean":"2023-09-23T04:43:51.721201+0000","last_became_active":"2023-09-23T04:43:47.587346+0000","last_became_peered":"2023-09-23T04:43:47.587346+0000","last_unstale":"2023-09-23T04:43:51.721201+0000","last_undegraded":"2023-09-23T04:43:51.721201+0000","last_fullsized":"2023-09-23T04:43:51.721201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:13:40.267593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.452275+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:44:20.452275+0000","last_peered":"2023-09-23T04:44:20.452275+0000","last_clean":"2023-09-23T04:44:20.452275+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:44:20.452275+0000","last_undegraded":"2023-09-23T04:44:20.452275+0000","last_fullsized":"2023-09-23T04:44:20.452275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.750010+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:43:52.750010+0000","last_peered":"2023-09-23T04:43:52.750010+0000","last_clean":"2023-09-23T04:43:52.750010+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:43:52.750010+0000","last_undegraded":"2023-09-23T04:43:52.750010+0000","last_fullsized":"2023-09-23T04:43:52.750010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718239+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:43:51.718239+0000","last_peered":"2023-09-23T04:43:51.718239+0000","last_clean":"2023-09-23T04:43:51.718239+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:43:51.718239+0000","last_undegraded":"2023-09-23T04:43:51.718239+0000","last_fullsized":"2023-09-23T04:43:51.718239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.456019+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:44:20.456019+0000","last_peered":"2023-09-23T04:44:20.456019+0000","last_clean":"2023-09-23T04:44:20.456019+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:44:20.456019+0000","last_undegraded":"2023-09-23T04:44:20.456019+0000","last_fullsized":"2023-09-23T04:44:20.456019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":321,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711304+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:43:51.711304+0000","last_peered":"2023-09-23T04:43:51.711304+0000","last_clean":"2023-09-23T04:43:51.711304+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:43:51.711304+0000","last_undegraded":"2023-09-23T04:43:51.711304+0000","last_fullsized":"2023-09-23T04:43:51.711304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748404+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:43:52.748404+0000","last_peered":"2023-09-23T04:43:52.748404+0000","last_clean":"2023-09-23T04:43:52.748404+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:43:52.748404+0000","last_undegraded":"2023-09-23T04:43:52.748404+0000","last_fullsized":"2023-09-23T04:43:52.748404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.710891+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:43:51.710891+0000","last_peered":"2023-09-23T04:43:51.710891+0000","last_clean":"2023-09-23T04:43:51.710891+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:43:51.710891+0000","last_undegraded":"2023-09-23T04:43:51.710891+0000","last_fullsized":"2023-09-23T04:43:51.710891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711468+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:43:51.711468+0000","last_peered":"2023-09-23T04:43:51.711468+0000","last_clean":"2023-09-23T04:43:51.711468+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:43:51.711468+0000","last_undegraded":"2023-09-23T04:43:51.711468+0000","last_fullsized":"2023-09-23T04:43:51.711468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749286+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:43:52.749286+0000","last_peered":"2023-09-23T04:43:52.749286+0000","last_clean":"2023-09-23T04:43:52.749286+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:43:52.749286+0000","last_undegraded":"2023-09-23T04:43:52.749286+0000","last_fullsized":"2023-09-23T04:43:52.749286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.714024+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:43:52.714024+0000","last_peered":"2023-09-23T04:43:52.714024+0000","last_clean":"2023-09-23T04:43:52.714024+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:43:52.714024+0000","last_undegraded":"2023-09-23T04:43:52.714024+0000","last_fullsized":"2023-09-23T04:43:52.714024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711188+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:43:51.711188+0000","last_peered":"2023-09-23T04:43:51.711188+0000","last_clean":"2023-09-23T04:43:51.711188+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:43:51.711188+0000","last_undegraded":"2023-09-23T04:43:51.711188+0000","last_fullsized":"2023-09-23T04:43:51.711188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747559+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:43:52.747559+0000","last_peered":"2023-09-23T04:43:52.747559+0000","last_clean":"2023-09-23T04:43:52.747559+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:43:52.747559+0000","last_undegraded":"2023-09-23T04:43:52.747559+0000","last_fullsized":"2023-09-23T04:43:52.747559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.795036+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:44:17.795036+0000","last_peered":"2023-09-23T04:44:17.795036+0000","last_clean":"2023-09-23T04:44:17.795036+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:44:17.795036+0000","last_undegraded":"2023-09-23T04:44:17.795036+0000","last_fullsized":"2023-09-23T04:44:17.795036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717512+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:43:51.717512+0000","last_peered":"2023-09-23T04:43:51.717512+0000","last_clean":"2023-09-23T04:43:51.717512+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:43:51.717512+0000","last_undegraded":"2023-09-23T04:43:51.717512+0000","last_fullsized":"2023-09-23T04:43:51.717512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747141+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:43:52.747141+0000","last_peered":"2023-09-23T04:43:52.747141+0000","last_clean":"2023-09-23T04:43:52.747141+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:43:52.747141+0000","last_undegraded":"2023-09-23T04:43:52.747141+0000","last_fullsized":"2023-09-23T04:43:52.747141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:19.816655+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:44:19.816655+0000","last_peered":"2023-09-23T04:44:19.816655+0000","last_clean":"2023-09-23T04:44:19.816655+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:44:19.816655+0000","last_undegraded":"2023-09-23T04:44:19.816655+0000","last_fullsized":"2023-09-23T04:44:19.816655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719714+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:43:51.719714+0000","last_peered":"2023-09-23T04:43:51.719714+0000","last_clean":"2023-09-23T04:43:51.719714+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:43:51.719714+0000","last_undegraded":"2023-09-23T04:43:51.719714+0000","last_fullsized":"2023-09-23T04:43:51.719714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749137+0000","last_change":"2023-09-23T04:43:51.737574+0000","last_active":"2023-09-23T04:43:52.749137+0000","last_peered":"2023-09-23T04:43:52.749137+0000","last_clean":"2023-09-23T04:43:52.749137+0000","last_became_active":"2023-09-23T04:43:51.736981+0000","last_became_peered":"2023-09-23T04:43:51.736981+0000","last_unstale":"2023-09-23T04:43:52.749137+0000","last_undegraded":"2023-09-23T04:43:52.749137+0000","last_fullsized":"2023-09-23T04:43:52.749137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:59:43.525900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779556+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:44:16.779556+0000","last_peered":"2023-09-23T04:44:16.779556+0000","last_clean":"2023-09-23T04:44:16.779556+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:44:16.779556+0000","last_undegraded":"2023-09-23T04:44:16.779556+0000","last_fullsized":"2023-09-23T04:44:16.779556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.715950+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:43:51.715950+0000","last_peered":"2023-09-23T04:43:51.715950+0000","last_clean":"2023-09-23T04:43:51.715950+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:43:51.715950+0000","last_undegraded":"2023-09-23T04:43:51.715950+0000","last_fullsized":"2023-09-23T04:43:51.715950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748382+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:43:52.748382+0000","last_peered":"2023-09-23T04:43:52.748382+0000","last_clean":"2023-09-23T04:43:52.748382+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:43:52.748382+0000","last_undegraded":"2023-09-23T04:43:52.748382+0000","last_fullsized":"2023-09-23T04:43:52.748382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.782955+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:44:16.782955+0000","last_peered":"2023-09-23T04:44:16.782955+0000","last_clean":"2023-09-23T04:44:16.782955+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:44:16.782955+0000","last_undegraded":"2023-09-23T04:44:16.782955+0000","last_fullsized":"2023-09-23T04:44:16.782955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716753+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:43:51.716753+0000","last_peered":"2023-09-23T04:43:51.716753+0000","last_clean":"2023-09-23T04:43:51.716753+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:43:51.716753+0000","last_undegraded":"2023-09-23T04:43:51.716753+0000","last_fullsized":"2023-09-23T04:43:51.716753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748956+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:43:52.748956+0000","last_peered":"2023-09-23T04:43:52.748956+0000","last_clean":"2023-09-23T04:43:52.748956+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:43:52.748956+0000","last_undegraded":"2023-09-23T04:43:52.748956+0000","last_fullsized":"2023-09-23T04:43:52.748956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716713+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:43:51.716713+0000","last_peered":"2023-09-23T04:43:51.716713+0000","last_clean":"2023-09-23T04:43:51.716713+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:43:51.716713+0000","last_undegraded":"2023-09-23T04:43:51.716713+0000","last_fullsized":"2023-09-23T04:43:51.716713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717716+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:43:51.717716+0000","last_peered":"2023-09-23T04:43:51.717716+0000","last_clean":"2023-09-23T04:43:51.717716+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:43:51.717716+0000","last_undegraded":"2023-09-23T04:43:51.717716+0000","last_fullsized":"2023-09-23T04:43:51.717716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748965+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:43:52.748965+0000","last_peered":"2023-09-23T04:43:52.748965+0000","last_clean":"2023-09-23T04:43:52.748965+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:43:52.748965+0000","last_undegraded":"2023-09-23T04:43:52.748965+0000","last_fullsized":"2023-09-23T04:43:52.748965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.781950+0000","last_change":"2023-09-23T04:43:46.590551+0000","last_active":"2023-09-23T04:44:16.781950+0000","last_peered":"2023-09-23T04:44:16.781950+0000","last_clean":"2023-09-23T04:44:16.781950+0000","last_became_active":"2023-09-23T04:43:46.590310+0000","last_became_peered":"2023-09-23T04:43:46.590310+0000","last_unstale":"2023-09-23T04:44:16.781950+0000","last_undegraded":"2023-09-23T04:44:16.781950+0000","last_fullsized":"2023-09-23T04:44:16.781950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:54.678727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711994+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:43:51.711994+0000","last_peered":"2023-09-23T04:43:51.711994+0000","last_clean":"2023-09-23T04:43:51.711994+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:43:51.711994+0000","last_undegraded":"2023-09-23T04:43:51.711994+0000","last_fullsized":"2023-09-23T04:43:51.711994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747503+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:43:52.747503+0000","last_peered":"2023-09-23T04:43:52.747503+0000","last_clean":"2023-09-23T04:43:52.747503+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:43:52.747503+0000","last_undegraded":"2023-09-23T04:43:52.747503+0000","last_fullsized":"2023-09-23T04:43:52.747503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793364+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:44:17.793364+0000","last_peered":"2023-09-23T04:44:17.793364+0000","last_clean":"2023-09-23T04:44:17.793364+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:44:17.793364+0000","last_undegraded":"2023-09-23T04:44:17.793364+0000","last_fullsized":"2023-09-23T04:44:17.793364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712075+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:43:51.712075+0000","last_peered":"2023-09-23T04:43:51.712075+0000","last_clean":"2023-09-23T04:43:51.712075+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:43:51.712075+0000","last_undegraded":"2023-09-23T04:43:51.712075+0000","last_fullsized":"2023-09-23T04:43:51.712075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.716174+0000","last_change":"2023-09-23T04:43:51.741159+0000","last_active":"2023-09-23T04:43:52.716174+0000","last_peered":"2023-09-23T04:43:52.716174+0000","last_clean":"2023-09-23T04:43:52.716174+0000","last_became_active":"2023-09-23T04:43:51.740696+0000","last_became_peered":"2023-09-23T04:43:51.740696+0000","last_unstale":"2023-09-23T04:43:52.716174+0000","last_undegraded":"2023-09-23T04:43:52.716174+0000","last_fullsized":"2023-09-23T04:43:52.716174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:34.832089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.801125+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:44:18.801125+0000","last_peered":"2023-09-23T04:44:18.801125+0000","last_clean":"2023-09-23T04:44:18.801125+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:44:18.801125+0000","last_undegraded":"2023-09-23T04:44:18.801125+0000","last_fullsized":"2023-09-23T04:44:18.801125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711797+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:43:51.711797+0000","last_peered":"2023-09-23T04:43:51.711797+0000","last_clean":"2023-09-23T04:43:51.711797+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:43:51.711797+0000","last_undegraded":"2023-09-23T04:43:51.711797+0000","last_fullsized":"2023-09-23T04:43:51.711797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747848+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:43:52.747848+0000","last_peered":"2023-09-23T04:43:52.747848+0000","last_clean":"2023-09-23T04:43:52.747848+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:43:52.747848+0000","last_undegraded":"2023-09-23T04:43:52.747848+0000","last_fullsized":"2023-09-23T04:43:52.747848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:19.817229+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:44:19.817229+0000","last_peered":"2023-09-23T04:44:19.817229+0000","last_clean":"2023-09-23T04:44:19.817229+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:44:19.817229+0000","last_undegraded":"2023-09-23T04:44:19.817229+0000","last_fullsized":"2023-09-23T04:44:19.817229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711841+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:43:51.711841+0000","last_peered":"2023-09-23T04:43:51.711841+0000","last_clean":"2023-09-23T04:43:51.711841+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:43:51.711841+0000","last_undegraded":"2023-09-23T04:43:51.711841+0000","last_fullsized":"2023-09-23T04:43:51.711841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716844+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:43:51.716844+0000","last_peered":"2023-09-23T04:43:51.716844+0000","last_clean":"2023-09-23T04:43:51.716844+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:43:51.716844+0000","last_undegraded":"2023-09-23T04:43:51.716844+0000","last_fullsized":"2023-09-23T04:43:51.716844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718946+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:43:51.718946+0000","last_peered":"2023-09-23T04:43:51.718946+0000","last_clean":"2023-09-23T04:43:51.718946+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:43:51.718946+0000","last_undegraded":"2023-09-23T04:43:51.718946+0000","last_fullsized":"2023-09-23T04:43:51.718946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716624+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:43:51.716624+0000","last_peered":"2023-09-23T04:43:51.716624+0000","last_clean":"2023-09-23T04:43:51.716624+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:43:51.716624+0000","last_undegraded":"2023-09-23T04:43:51.716624+0000","last_fullsized":"2023-09-23T04:43:51.716624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719350+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:43:51.719350+0000","last_peered":"2023-09-23T04:43:51.719350+0000","last_clean":"2023-09-23T04:43:51.719350+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:43:51.719350+0000","last_undegraded":"2023-09-23T04:43:51.719350+0000","last_fullsized":"2023-09-23T04:43:51.719350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711381+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:43:51.711381+0000","last_peered":"2023-09-23T04:43:51.711381+0000","last_clean":"2023-09-23T04:43:51.711381+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:43:51.711381+0000","last_undegraded":"2023-09-23T04:43:51.711381+0000","last_fullsized":"2023-09-23T04:43:51.711381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719943+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:43:51.719943+0000","last_peered":"2023-09-23T04:43:51.719943+0000","last_clean":"2023-09-23T04:43:51.719943+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:43:51.719943+0000","last_undegraded":"2023-09-23T04:43:51.719943+0000","last_fullsized":"2023-09-23T04:43:51.719943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717089+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:43:51.717089+0000","last_peered":"2023-09-23T04:43:51.717089+0000","last_clean":"2023-09-23T04:43:51.717089+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:43:51.717089+0000","last_undegraded":"2023-09-23T04:43:51.717089+0000","last_fullsized":"2023-09-23T04:43:51.717089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717606+0000","last_change":"2023-09-23T04:43:47.596319+0000","last_active":"2023-09-23T04:43:51.717606+0000","last_peered":"2023-09-23T04:43:51.717606+0000","last_clean":"2023-09-23T04:43:51.717606+0000","last_became_active":"2023-09-23T04:43:47.595532+0000","last_became_peered":"2023-09-23T04:43:47.595532+0000","last_unstale":"2023-09-23T04:43:51.717606+0000","last_undegraded":"2023-09-23T04:43:51.717606+0000","last_fullsized":"2023-09-23T04:43:51.717606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:04.830870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.721112+0000","last_change":"2023-09-23T04:43:47.592560+0000","last_active":"2023-09-23T04:43:51.721112+0000","last_peered":"2023-09-23T04:43:51.721112+0000","last_clean":"2023-09-23T04:43:51.721112+0000","last_became_active":"2023-09-23T04:43:47.592259+0000","last_became_peered":"2023-09-23T04:43:47.592259+0000","last_unstale":"2023-09-23T04:43:51.721112+0000","last_undegraded":"2023-09-23T04:43:51.721112+0000","last_fullsized":"2023-09-23T04:43:51.721112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:32:01.023978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719380+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:43:51.719380+0000","last_peered":"2023-09-23T04:43:51.719380+0000","last_clean":"2023-09-23T04:43:51.719380+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:43:51.719380+0000","last_undegraded":"2023-09-23T04:43:51.719380+0000","last_fullsized":"2023-09-23T04:43:51.719380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718855+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:43:51.718855+0000","last_peered":"2023-09-23T04:43:51.718855+0000","last_clean":"2023-09-23T04:43:51.718855+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:43:51.718855+0000","last_undegraded":"2023-09-23T04:43:51.718855+0000","last_fullsized":"2023-09-23T04:43:51.718855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711153+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:43:51.711153+0000","last_peered":"2023-09-23T04:43:51.711153+0000","last_clean":"2023-09-23T04:43:51.711153+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:43:51.711153+0000","last_undegraded":"2023-09-23T04:43:51.711153+0000","last_fullsized":"2023-09-23T04:43:51.711153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716397+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:43:51.716397+0000","last_peered":"2023-09-23T04:43:51.716397+0000","last_clean":"2023-09-23T04:43:51.716397+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:43:51.716397+0000","last_undegraded":"2023-09-23T04:43:51.716397+0000","last_fullsized":"2023-09-23T04:43:51.716397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8982,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":145,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":23180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":469360,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.433},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59199999999999997}]},{"osd":1,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63400000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622}]},{"osd":6,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.3540000000000001,"5min":4.3540000000000001,"15min":4.3540000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.91300000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.93500000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1059860,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.32900000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.8220000000000001,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.746}]},{"osd":9,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.64900000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76300000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":464890,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66800000000000004}]},{"osd":4,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.72099999999999997}]},{"osd":5,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.82599999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.628}]},{"osd":7,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.85899999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.79500000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.874},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":464810,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.747,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.73299999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.48699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.89200000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56599999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":1.0109999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.76200000000000001,"5min":1.008,"15min":1.008},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.52200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.94499999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.258},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":1.052}]},{"osd":7,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.497,"5min":0.435,"15min":0.435},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97299999999999998,"5min":1.284,"15min":1.284},"last":1.0700000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.23}]},{"osd":10,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.1839999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1054749,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.441,"15min":0.439},"min":{"1min":0.29599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.624}]},{"osd":3,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90900000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1,"5min":1,"15min":1},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57699999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92800000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73999999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":464794,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.59899999999999998,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.2750000000000004,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.57999999999999996,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.748,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.2480000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.012,"5min":0.79900000000000004,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.68899999999999995}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.83099999999999996,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81799999999999995,"15min":0.80600000000000005},"min":{"1min":0.62,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.86399999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.63100000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.109,"15min":0.98799999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":15.34,"5min":15.34,"15min":15.34},"last":0.873},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":467641,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.504}]},{"osd":2,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.373},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.47399999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40000000000000002}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.46899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":1.014,"15min":1.014},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76300000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.626},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49099999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.96999999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":0.997,"15min":0.997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0669999999999999,"5min":1.103,"15min":1.103},"last":1.004}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.71699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":464162,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.42299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.014,"5min":1.345,"15min":1.345},"last":0.59599999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.623,"15min":0.57799999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.091,"5min":1.306,"15min":1.306},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75600000000000001}]},{"osd":3,"last update":"Sat Sep 23 04:42:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.624}]},{"osd":4,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.046,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.58999999999999997,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81599999999999995}]},{"osd":5,"last update":"Sat Sep 23 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.94999999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.753,"15min":0.755},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Sat Sep 23 04:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88400000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.004,"5min":1.091,"15min":1.091},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.94399999999999995,"15min":0.97199999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.871,"15min":1.871},"last":0.97499999999999998}]},{"osd":9,"last update":"Sat Sep 23 04:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.77700000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.93799999999999994}]},{"osd":11,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.95999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":468570,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001}]},{"osd":1,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.372,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.745,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69699999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:43:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.56699999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.79800000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.442,"15min":0.434},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46999999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.624}]},{"osd":7,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.83999999999999997,"15min":0.87},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.73599999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.85899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55700000000000005}]},{"osd":9,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.86399999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":464025,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.434,"15min":0.39300000000000002},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.376}]},{"osd":1,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.52200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.627}]},{"osd":4,"last update":"Sat Sep 23 04:43:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.59099999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.752},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.504,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":0.999,"15min":0.999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.214,"15min":1.214},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.45300000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72499999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.57599999999999996}]},{"osd":10,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":465162,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.23400000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.30499999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72999999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.90200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.52200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.89900000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93400000000000005}]},{"osd":9,"last update":"Sat Sep 23 04:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1057068,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81899999999999995,"5min":1.246,"15min":1.246},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":1.115,"15min":1.115},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89900000000000002,"5min":1.071,"15min":1.071},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.314,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.629,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.439},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.41499999999999998,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40600000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57399999999999995,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":1.212,"15min":1.212},"last":0.627},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72899999999999998,"5min":1.042,"15min":1.042},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56000000000000005,"15min":0.53000000000000003},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":1.23,"15min":1.23},"last":0.67500000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.623,"15min":0.623},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.105,"15min":1.105},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.629,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54300000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.752}]},{"osd":10,"last update":"Sat Sep 23 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:44:22.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.047+0000 7f332cfff700 1 -- 172.21.15.5:0/3787865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 msgr2=0x7f3328109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.047+0000 7f332cfff700 1 --2- 172.21.15.5:0/3787865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f3328109350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3320009f90 tx=0x7f332000b330 comp rx=0 tx=0).stop 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 -- 172.21.15.5:0/3787865278 shutdown_connections 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 --2- 172.21.15.5:0/3787865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f3328109350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 --2- 172.21.15.5:0/3787865278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328100e90 0x7f33281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 --2- 172.21.15.5:0/3787865278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328102070 0x7f33281024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 -- 172.21.15.5:0/3787865278 >> 172.21.15.5:0/3787865278 conn(0x7f33280fc450 msgr2=0x7f33280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:23.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 -- 172.21.15.5:0/3787865278 shutdown_connections 2023-09-23T04:44:23.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.048+0000 7f332cfff700 1 -- 172.21.15.5:0/3787865278 wait complete. 2023-09-23T04:44:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 Processor -- start 2023-09-23T04:44:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 -- start start 2023-09-23T04:44:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328100e90 0x7f3328198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f33281a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a0a80 con 0x7f3328102070 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a0bc0 con 0x7f3328102a30 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.049+0000 7f332cfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a0d50 con 0x7f3328100e90 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f332659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328100e90 0x7f3328198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43874/0 (socket says 172.21.15.5:43874) 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 -- 172.21.15.5:0/3323774562 learned_addr learned my addr 172.21.15.5:0/3323774562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3326d9d700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f33281a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 -- 172.21.15.5:0/3323774562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328100e90 msgr2=0x7f3328198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328100e90 0x7f3328198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 -- 172.21.15.5:0/3323774562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 msgr2=0x7f33281a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f33281a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 -- 172.21.15.5:0/3323774562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180095f0 con 0x7f3328102070 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.050+0000 7f3325d9b700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e970 tx=0x7f331c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.051+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c009790 con 0x7f3328102070 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.051+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331c00c7e0 con 0x7f3328102070 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.051+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c003c70 con 0x7f3328102070 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.051+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3320009c40 con 0x7f3328102070 2023-09-23T04:44:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.051+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33281a12d0 con 0x7f3328102070 2023-09-23T04:44:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.052+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332810c750 con 0x7f3328102070 2023-09-23T04:44:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.054+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f331c00cc50 con 0x7f3328102070 2023-09-23T04:44:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.056+0000 7f33177fe700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 0x7f33100a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.056+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f331c014070 con 0x7f3328102070 2023-09-23T04:44:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.057+0000 7f332659c700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 0x7f33100a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.057+0000 7f332659c700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 0x7f33100a3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3328101be0 tx=0x7f3318009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:23.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.060+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f331c0966b0 con 0x7f3328102070 2023-09-23T04:44:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: pgmap v286: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2773120495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2500386372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:22 smithi005 ceph-mon[97638]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: pgmap v286: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-23T04:44:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T04:44:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2773120495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2500386372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:22 smithi118 ceph-mon[134297]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: pgmap v286: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2773120495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2500386372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:23.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:22 smithi069 ceph-mon[133931]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:23.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.268+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3328108a70 con 0x7f33100a1630 2023-09-23T04:44:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.281+0000 7f33177fe700 1 -- 172.21.15.5:0/3323774562 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423025 (secure 0 0 0) 0x7f3328108a70 con 0x7f33100a1630 2023-09-23T04:44:23.286 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 msgr2=0x7f33100a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 0x7f33100a3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3328101be0 tx=0x7f3318009310 comp rx=0 tx=0).stop 2023-09-23T04:44:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 msgr2=0x7f3328199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e970 tx=0x7f331c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 shutdown_connections 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f33100a1630 0x7f33100a3af0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3328102a30 0x7f33281a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3328100e90 0x7f3328198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 --2- 172.21.15.5:0/3323774562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3328102070 0x7f3328199440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 >> 172.21.15.5:0/3323774562 conn(0x7f33280fc450 msgr2=0x7f33280fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 shutdown_connections 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:23.286+0000 7f332cfff700 1 -- 172.21.15.5:0/3323774562 wait complete. 2023-09-23T04:44:23.290 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:44:23.305 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-09-23T04:44:21.638017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599473,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":327,"num_write_kb":3001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335572,"kb_used_data":11360,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606252,"statfs":{"total":1151940427776,"available":1151596802048,"internally_reserved":0,"allocated":11632640,"data_stored":7365091,"data_compressed":249699,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4465,"num_objects":47,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":97,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717797+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:43:51.717797+0000","last_peered":"2023-09-23T04:43:51.717797+0000","last_clean":"2023-09-23T04:43:51.717797+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:43:51.717797+0000","last_undegraded":"2023-09-23T04:43:51.717797+0000","last_fullsized":"2023-09-23T04:43:51.717797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719824+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:43:51.719824+0000","last_peered":"2023-09-23T04:43:51.719824+0000","last_clean":"2023-09-23T04:43:51.719824+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:43:51.719824+0000","last_undegraded":"2023-09-23T04:43:51.719824+0000","last_fullsized":"2023-09-23T04:43:51.719824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719079+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:43:51.719079+0000","last_peered":"2023-09-23T04:43:51.719079+0000","last_clean":"2023-09-23T04:43:51.719079+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:43:51.719079+0000","last_undegraded":"2023-09-23T04:43:51.719079+0000","last_fullsized":"2023-09-23T04:43:51.719079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717311+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:43:51.717311+0000","last_peered":"2023-09-23T04:43:51.717311+0000","last_clean":"2023-09-23T04:43:51.717311+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:43:51.717311+0000","last_undegraded":"2023-09-23T04:43:51.717311+0000","last_fullsized":"2023-09-23T04:43:51.717311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717830+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:43:51.717830+0000","last_peered":"2023-09-23T04:43:51.717830+0000","last_clean":"2023-09-23T04:43:51.717830+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:43:51.717830+0000","last_undegraded":"2023-09-23T04:43:51.717830+0000","last_fullsized":"2023-09-23T04:43:51.717830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711621+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:43:51.711621+0000","last_peered":"2023-09-23T04:43:51.711621+0000","last_clean":"2023-09-23T04:43:51.711621+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:43:51.711621+0000","last_undegraded":"2023-09-23T04:43:51.711621+0000","last_fullsized":"2023-09-23T04:43:51.711621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711566+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:43:51.711566+0000","last_peered":"2023-09-23T04:43:51.711566+0000","last_clean":"2023-09-23T04:43:51.711566+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:43:51.711566+0000","last_undegraded":"2023-09-23T04:43:51.711566+0000","last_fullsized":"2023-09-23T04:43:51.711566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716735+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:43:51.716735+0000","last_peered":"2023-09-23T04:43:51.716735+0000","last_clean":"2023-09-23T04:43:51.716735+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:43:51.716735+0000","last_undegraded":"2023-09-23T04:43:51.716735+0000","last_fullsized":"2023-09-23T04:43:51.716735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716786+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:43:51.716786+0000","last_peered":"2023-09-23T04:43:51.716786+0000","last_clean":"2023-09-23T04:43:51.716786+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:43:51.716786+0000","last_undegraded":"2023-09-23T04:43:51.716786+0000","last_fullsized":"2023-09-23T04:43:51.716786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712252+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:43:51.712252+0000","last_peered":"2023-09-23T04:43:51.712252+0000","last_clean":"2023-09-23T04:43:51.712252+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:43:51.712252+0000","last_undegraded":"2023-09-23T04:43:51.712252+0000","last_fullsized":"2023-09-23T04:43:51.712252+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712172+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:43:51.712172+0000","last_peered":"2023-09-23T04:43:51.712172+0000","last_clean":"2023-09-23T04:43:51.712172+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:43:51.712172+0000","last_undegraded":"2023-09-23T04:43:51.712172+0000","last_fullsized":"2023-09-23T04:43:51.712172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717442+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:43:51.717442+0000","last_peered":"2023-09-23T04:43:51.717442+0000","last_clean":"2023-09-23T04:43:51.717442+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:43:51.717442+0000","last_undegraded":"2023-09-23T04:43:51.717442+0000","last_fullsized":"2023-09-23T04:43:51.717442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711174+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:43:51.711174+0000","last_peered":"2023-09-23T04:43:51.711174+0000","last_clean":"2023-09-23T04:43:51.711174+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:43:51.711174+0000","last_undegraded":"2023-09-23T04:43:51.711174+0000","last_fullsized":"2023-09-23T04:43:51.711174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718910+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:43:51.718910+0000","last_peered":"2023-09-23T04:43:51.718910+0000","last_clean":"2023-09-23T04:43:51.718910+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:43:51.718910+0000","last_undegraded":"2023-09-23T04:43:51.718910+0000","last_fullsized":"2023-09-23T04:43:51.718910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719366+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:43:51.719366+0000","last_peered":"2023-09-23T04:43:51.719366+0000","last_clean":"2023-09-23T04:43:51.719366+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:43:51.719366+0000","last_undegraded":"2023-09-23T04:43:51.719366+0000","last_fullsized":"2023-09-23T04:43:51.719366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716266+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:43:51.716266+0000","last_peered":"2023-09-23T04:43:51.716266+0000","last_clean":"2023-09-23T04:43:51.716266+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:43:51.716266+0000","last_undegraded":"2023-09-23T04:43:51.716266+0000","last_fullsized":"2023-09-23T04:43:51.716266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717700+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:43:51.717700+0000","last_peered":"2023-09-23T04:43:51.717700+0000","last_clean":"2023-09-23T04:43:51.717700+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:43:51.717700+0000","last_undegraded":"2023-09-23T04:43:51.717700+0000","last_fullsized":"2023-09-23T04:43:51.717700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717958+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:43:51.717958+0000","last_peered":"2023-09-23T04:43:51.717958+0000","last_clean":"2023-09-23T04:43:51.717958+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:43:51.717958+0000","last_undegraded":"2023-09-23T04:43:51.717958+0000","last_fullsized":"2023-09-23T04:43:51.717958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716415+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:43:51.716415+0000","last_peered":"2023-09-23T04:43:51.716415+0000","last_clean":"2023-09-23T04:43:51.716415+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:43:51.716415+0000","last_undegraded":"2023-09-23T04:43:51.716415+0000","last_fullsized":"2023-09-23T04:43:51.716415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747383+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:43:52.747383+0000","last_peered":"2023-09-23T04:43:52.747383+0000","last_clean":"2023-09-23T04:43:52.747383+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:43:52.747383+0000","last_undegraded":"2023-09-23T04:43:52.747383+0000","last_fullsized":"2023-09-23T04:43:52.747383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793602+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:44:17.793602+0000","last_peered":"2023-09-23T04:44:17.793602+0000","last_clean":"2023-09-23T04:44:17.793602+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:44:17.793602+0000","last_undegraded":"2023-09-23T04:44:17.793602+0000","last_fullsized":"2023-09-23T04:44:17.793602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717484+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:43:51.717484+0000","last_peered":"2023-09-23T04:43:51.717484+0000","last_clean":"2023-09-23T04:43:51.717484+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:43:51.717484+0000","last_undegraded":"2023-09-23T04:43:51.717484+0000","last_fullsized":"2023-09-23T04:43:51.717484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748086+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:43:52.748086+0000","last_peered":"2023-09-23T04:43:52.748086+0000","last_clean":"2023-09-23T04:43:52.748086+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:43:52.748086+0000","last_undegraded":"2023-09-23T04:43:52.748086+0000","last_fullsized":"2023-09-23T04:43:52.748086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748662+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:43:52.748662+0000","last_peered":"2023-09-23T04:43:52.748662+0000","last_clean":"2023-09-23T04:43:52.748662+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:43:52.748662+0000","last_undegraded":"2023-09-23T04:43:52.748662+0000","last_fullsized":"2023-09-23T04:43:52.748662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718258+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:43:51.718258+0000","last_peered":"2023-09-23T04:43:51.718258+0000","last_clean":"2023-09-23T04:43:51.718258+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:43:51.718258+0000","last_undegraded":"2023-09-23T04:43:51.718258+0000","last_fullsized":"2023-09-23T04:43:51.718258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748514+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:43:52.748514+0000","last_peered":"2023-09-23T04:43:52.748514+0000","last_clean":"2023-09-23T04:43:52.748514+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:43:52.748514+0000","last_undegraded":"2023-09-23T04:43:52.748514+0000","last_fullsized":"2023-09-23T04:43:52.748514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.791739+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:44:17.791739+0000","last_peered":"2023-09-23T04:44:17.791739+0000","last_clean":"2023-09-23T04:44:17.791739+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:44:17.791739+0000","last_undegraded":"2023-09-23T04:44:17.791739+0000","last_fullsized":"2023-09-23T04:44:17.791739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716893+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:43:51.716893+0000","last_peered":"2023-09-23T04:43:51.716893+0000","last_clean":"2023-09-23T04:43:51.716893+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:43:51.716893+0000","last_undegraded":"2023-09-23T04:43:51.716893+0000","last_fullsized":"2023-09-23T04:43:51.716893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747647+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:43:52.747647+0000","last_peered":"2023-09-23T04:43:52.747647+0000","last_clean":"2023-09-23T04:43:52.747647+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:43:52.747647+0000","last_undegraded":"2023-09-23T04:43:52.747647+0000","last_fullsized":"2023-09-23T04:43:52.747647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718945+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:43:51.718945+0000","last_peered":"2023-09-23T04:43:51.718945+0000","last_clean":"2023-09-23T04:43:51.718945+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:43:51.718945+0000","last_undegraded":"2023-09-23T04:43:51.718945+0000","last_fullsized":"2023-09-23T04:43:51.718945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717196+0000","last_change":"2023-09-23T04:43:47.591420+0000","last_active":"2023-09-23T04:43:51.717196+0000","last_peered":"2023-09-23T04:43:51.717196+0000","last_clean":"2023-09-23T04:43:51.717196+0000","last_became_active":"2023-09-23T04:43:47.591121+0000","last_became_peered":"2023-09-23T04:43:47.591121+0000","last_unstale":"2023-09-23T04:43:51.717196+0000","last_undegraded":"2023-09-23T04:43:51.717196+0000","last_fullsized":"2023-09-23T04:43:51.717196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:52:14.710768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747924+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:43:52.747924+0000","last_peered":"2023-09-23T04:43:52.747924+0000","last_clean":"2023-09-23T04:43:52.747924+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:43:52.747924+0000","last_undegraded":"2023-09-23T04:43:52.747924+0000","last_fullsized":"2023-09-23T04:43:52.747924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.783110+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:44:16.783110+0000","last_peered":"2023-09-23T04:44:16.783110+0000","last_clean":"2023-09-23T04:44:16.783110+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:44:16.783110+0000","last_undegraded":"2023-09-23T04:44:16.783110+0000","last_fullsized":"2023-09-23T04:44:16.783110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716947+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:43:51.716947+0000","last_peered":"2023-09-23T04:43:51.716947+0000","last_clean":"2023-09-23T04:43:51.716947+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:43:51.716947+0000","last_undegraded":"2023-09-23T04:43:51.716947+0000","last_fullsized":"2023-09-23T04:43:51.716947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747510+0000","last_change":"2023-09-23T04:43:51.735548+0000","last_active":"2023-09-23T04:43:52.747510+0000","last_peered":"2023-09-23T04:43:52.747510+0000","last_clean":"2023-09-23T04:43:52.747510+0000","last_became_active":"2023-09-23T04:43:51.735048+0000","last_became_peered":"2023-09-23T04:43:51.735048+0000","last_unstale":"2023-09-23T04:43:52.747510+0000","last_undegraded":"2023-09-23T04:43:52.747510+0000","last_fullsized":"2023-09-23T04:43:52.747510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:15:25.552028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793643+0000","last_change":"2023-09-23T04:43:46.586528+0000","last_active":"2023-09-23T04:44:17.793643+0000","last_peered":"2023-09-23T04:44:17.793643+0000","last_clean":"2023-09-23T04:44:17.793643+0000","last_became_active":"2023-09-23T04:43:46.586207+0000","last_became_peered":"2023-09-23T04:43:46.586207+0000","last_unstale":"2023-09-23T04:44:17.793643+0000","last_undegraded":"2023-09-23T04:44:17.793643+0000","last_fullsized":"2023-09-23T04:44:17.793643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:53:43.466578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718509+0000","last_change":"2023-09-23T04:43:47.602346+0000","last_active":"2023-09-23T04:43:51.718509+0000","last_peered":"2023-09-23T04:43:51.718509+0000","last_clean":"2023-09-23T04:43:51.718509+0000","last_became_active":"2023-09-23T04:43:47.601998+0000","last_became_peered":"2023-09-23T04:43:47.601998+0000","last_unstale":"2023-09-23T04:43:51.718509+0000","last_undegraded":"2023-09-23T04:43:51.718509+0000","last_fullsized":"2023-09-23T04:43:51.718509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:57:33.953719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749525+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:43:52.749525+0000","last_peered":"2023-09-23T04:43:52.749525+0000","last_clean":"2023-09-23T04:43:52.749525+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:43:52.749525+0000","last_undegraded":"2023-09-23T04:43:52.749525+0000","last_fullsized":"2023-09-23T04:43:52.749525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.794124+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:44:17.794124+0000","last_peered":"2023-09-23T04:44:17.794124+0000","last_clean":"2023-09-23T04:44:17.794124+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:44:17.794124+0000","last_undegraded":"2023-09-23T04:44:17.794124+0000","last_fullsized":"2023-09-23T04:44:17.794124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718354+0000","last_change":"2023-09-23T04:43:47.591680+0000","last_active":"2023-09-23T04:43:51.718354+0000","last_peered":"2023-09-23T04:43:51.718354+0000","last_clean":"2023-09-23T04:43:51.718354+0000","last_became_active":"2023-09-23T04:43:47.590872+0000","last_became_peered":"2023-09-23T04:43:47.590872+0000","last_unstale":"2023-09-23T04:43:51.718354+0000","last_undegraded":"2023-09-23T04:43:51.718354+0000","last_fullsized":"2023-09-23T04:43:51.718354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:49:20.600268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.716175+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:43:52.716175+0000","last_peered":"2023-09-23T04:43:52.716175+0000","last_clean":"2023-09-23T04:43:52.716175+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:43:52.716175+0000","last_undegraded":"2023-09-23T04:43:52.716175+0000","last_fullsized":"2023-09-23T04:43:52.716175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.799922+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:44:18.799922+0000","last_peered":"2023-09-23T04:44:18.799922+0000","last_clean":"2023-09-23T04:44:18.799922+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:44:18.799922+0000","last_undegraded":"2023-09-23T04:44:18.799922+0000","last_fullsized":"2023-09-23T04:44:18.799922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716620+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:43:51.716620+0000","last_peered":"2023-09-23T04:43:51.716620+0000","last_clean":"2023-09-23T04:43:51.716620+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:43:51.716620+0000","last_undegraded":"2023-09-23T04:43:51.716620+0000","last_fullsized":"2023-09-23T04:43:51.716620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747876+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:43:52.747876+0000","last_peered":"2023-09-23T04:43:52.747876+0000","last_clean":"2023-09-23T04:43:52.747876+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:43:52.747876+0000","last_undegraded":"2023-09-23T04:43:52.747876+0000","last_fullsized":"2023-09-23T04:43:52.747876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779500+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:44:16.779500+0000","last_peered":"2023-09-23T04:44:16.779500+0000","last_clean":"2023-09-23T04:44:16.779500+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:44:16.779500+0000","last_undegraded":"2023-09-23T04:44:16.779500+0000","last_fullsized":"2023-09-23T04:44:16.779500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716901+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:43:51.716901+0000","last_peered":"2023-09-23T04:43:51.716901+0000","last_clean":"2023-09-23T04:43:51.716901+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:43:51.716901+0000","last_undegraded":"2023-09-23T04:43:51.716901+0000","last_fullsized":"2023-09-23T04:43:51.716901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747995+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:43:52.747995+0000","last_peered":"2023-09-23T04:43:52.747995+0000","last_clean":"2023-09-23T04:43:52.747995+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:43:52.747995+0000","last_undegraded":"2023-09-23T04:43:52.747995+0000","last_fullsized":"2023-09-23T04:43:52.747995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711940+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:43:51.711940+0000","last_peered":"2023-09-23T04:43:51.711940+0000","last_clean":"2023-09-23T04:43:51.711940+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:43:51.711940+0000","last_undegraded":"2023-09-23T04:43:51.711940+0000","last_fullsized":"2023-09-23T04:43:51.711940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718343+0000","last_change":"2023-09-23T04:43:47.596424+0000","last_active":"2023-09-23T04:43:51.718343+0000","last_peered":"2023-09-23T04:43:51.718343+0000","last_clean":"2023-09-23T04:43:51.718343+0000","last_became_active":"2023-09-23T04:43:47.595568+0000","last_became_peered":"2023-09-23T04:43:47.595568+0000","last_unstale":"2023-09-23T04:43:51.718343+0000","last_undegraded":"2023-09-23T04:43:51.718343+0000","last_fullsized":"2023-09-23T04:43:51.718343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:46.814902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748728+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:43:52.748728+0000","last_peered":"2023-09-23T04:43:52.748728+0000","last_clean":"2023-09-23T04:43:52.748728+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:43:52.748728+0000","last_undegraded":"2023-09-23T04:43:52.748728+0000","last_fullsized":"2023-09-23T04:43:52.748728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.778760+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:44:16.778760+0000","last_peered":"2023-09-23T04:44:16.778760+0000","last_clean":"2023-09-23T04:44:16.778760+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:44:16.778760+0000","last_undegraded":"2023-09-23T04:44:16.778760+0000","last_fullsized":"2023-09-23T04:44:16.778760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716787+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:43:51.716787+0000","last_peered":"2023-09-23T04:43:51.716787+0000","last_clean":"2023-09-23T04:43:51.716787+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:43:51.716787+0000","last_undegraded":"2023-09-23T04:43:51.716787+0000","last_fullsized":"2023-09-23T04:43:51.716787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748735+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:43:52.748735+0000","last_peered":"2023-09-23T04:43:52.748735+0000","last_clean":"2023-09-23T04:43:52.748735+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:43:52.748735+0000","last_undegraded":"2023-09-23T04:43:52.748735+0000","last_fullsized":"2023-09-23T04:43:52.748735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779678+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:44:16.779678+0000","last_peered":"2023-09-23T04:44:16.779678+0000","last_clean":"2023-09-23T04:44:16.779678+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:44:16.779678+0000","last_undegraded":"2023-09-23T04:44:16.779678+0000","last_fullsized":"2023-09-23T04:44:16.779678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718345+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:43:51.718345+0000","last_peered":"2023-09-23T04:43:51.718345+0000","last_clean":"2023-09-23T04:43:51.718345+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:43:51.718345+0000","last_undegraded":"2023-09-23T04:43:51.718345+0000","last_fullsized":"2023-09-23T04:43:51.718345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748704+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:43:52.748704+0000","last_peered":"2023-09-23T04:43:52.748704+0000","last_clean":"2023-09-23T04:43:52.748704+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:43:52.748704+0000","last_undegraded":"2023-09-23T04:43:52.748704+0000","last_fullsized":"2023-09-23T04:43:52.748704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.794618+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:44:17.794618+0000","last_peered":"2023-09-23T04:44:17.794618+0000","last_clean":"2023-09-23T04:44:17.794618+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:44:17.794618+0000","last_undegraded":"2023-09-23T04:44:17.794618+0000","last_fullsized":"2023-09-23T04:44:17.794618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.710991+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:43:51.710991+0000","last_peered":"2023-09-23T04:43:51.710991+0000","last_clean":"2023-09-23T04:43:51.710991+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:43:51.710991+0000","last_undegraded":"2023-09-23T04:43:51.710991+0000","last_fullsized":"2023-09-23T04:43:51.710991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749482+0000","last_change":"2023-09-23T04:43:51.731564+0000","last_active":"2023-09-23T04:43:52.749482+0000","last_peered":"2023-09-23T04:43:52.749482+0000","last_clean":"2023-09-23T04:43:52.749482+0000","last_became_active":"2023-09-23T04:43:51.731151+0000","last_became_peered":"2023-09-23T04:43:51.731151+0000","last_unstale":"2023-09-23T04:43:52.749482+0000","last_undegraded":"2023-09-23T04:43:52.749482+0000","last_fullsized":"2023-09-23T04:43:52.749482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.954294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.801133+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:44:18.801133+0000","last_peered":"2023-09-23T04:44:18.801133+0000","last_clean":"2023-09-23T04:44:18.801133+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:44:18.801133+0000","last_undegraded":"2023-09-23T04:44:18.801133+0000","last_fullsized":"2023-09-23T04:44:18.801133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711691+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:43:51.711691+0000","last_peered":"2023-09-23T04:43:51.711691+0000","last_clean":"2023-09-23T04:43:51.711691+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:43:51.711691+0000","last_undegraded":"2023-09-23T04:43:51.711691+0000","last_fullsized":"2023-09-23T04:43:51.711691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748136+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:43:52.748136+0000","last_peered":"2023-09-23T04:43:52.748136+0000","last_clean":"2023-09-23T04:43:52.748136+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:43:52.748136+0000","last_undegraded":"2023-09-23T04:43:52.748136+0000","last_fullsized":"2023-09-23T04:43:52.748136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.455290+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:44:20.455290+0000","last_peered":"2023-09-23T04:44:20.455290+0000","last_clean":"2023-09-23T04:44:20.455290+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:44:20.455290+0000","last_undegraded":"2023-09-23T04:44:20.455290+0000","last_fullsized":"2023-09-23T04:44:20.455290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711560+0000","last_change":"2023-09-23T04:43:47.584835+0000","last_active":"2023-09-23T04:43:51.711560+0000","last_peered":"2023-09-23T04:43:51.711560+0000","last_clean":"2023-09-23T04:43:51.711560+0000","last_became_active":"2023-09-23T04:43:47.584538+0000","last_became_peered":"2023-09-23T04:43:47.584538+0000","last_unstale":"2023-09-23T04:43:51.711560+0000","last_undegraded":"2023-09-23T04:43:51.711560+0000","last_fullsized":"2023-09-23T04:43:51.711560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:16.222318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713651+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:43:52.713651+0000","last_peered":"2023-09-23T04:43:52.713651+0000","last_clean":"2023-09-23T04:43:52.713651+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:43:52.713651+0000","last_undegraded":"2023-09-23T04:43:52.713651+0000","last_fullsized":"2023-09-23T04:43:52.713651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793701+0000","last_change":"2023-09-23T04:43:46.593590+0000","last_active":"2023-09-23T04:44:17.793701+0000","last_peered":"2023-09-23T04:44:17.793701+0000","last_clean":"2023-09-23T04:44:17.793701+0000","last_became_active":"2023-09-23T04:43:46.593301+0000","last_became_peered":"2023-09-23T04:43:46.593301+0000","last_unstale":"2023-09-23T04:44:17.793701+0000","last_undegraded":"2023-09-23T04:44:17.793701+0000","last_fullsized":"2023-09-23T04:44:17.793701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:44.113054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.720396+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:43:51.720396+0000","last_peered":"2023-09-23T04:43:51.720396+0000","last_clean":"2023-09-23T04:43:51.720396+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:43:51.720396+0000","last_undegraded":"2023-09-23T04:43:51.720396+0000","last_fullsized":"2023-09-23T04:43:51.720396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747767+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:43:52.747767+0000","last_peered":"2023-09-23T04:43:52.747767+0000","last_clean":"2023-09-23T04:43:52.747767+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:43:52.747767+0000","last_undegraded":"2023-09-23T04:43:52.747767+0000","last_fullsized":"2023-09-23T04:43:52.747767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716138+0000","last_change":"2023-09-23T04:43:46.592918+0000","last_active":"2023-09-23T04:43:51.716138+0000","last_peered":"2023-09-23T04:43:51.716138+0000","last_clean":"2023-09-23T04:43:51.716138+0000","last_became_active":"2023-09-23T04:43:46.592612+0000","last_became_peered":"2023-09-23T04:43:46.592612+0000","last_unstale":"2023-09-23T04:43:51.716138+0000","last_undegraded":"2023-09-23T04:43:51.716138+0000","last_fullsized":"2023-09-23T04:43:51.716138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:59:48.213727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.720629+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:43:51.720629+0000","last_peered":"2023-09-23T04:43:51.720629+0000","last_clean":"2023-09-23T04:43:51.720629+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:43:51.720629+0000","last_undegraded":"2023-09-23T04:43:51.720629+0000","last_fullsized":"2023-09-23T04:43:51.720629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713696+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:43:52.713696+0000","last_peered":"2023-09-23T04:43:52.713696+0000","last_clean":"2023-09-23T04:43:52.713696+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:43:52.713696+0000","last_undegraded":"2023-09-23T04:43:52.713696+0000","last_fullsized":"2023-09-23T04:43:52.713696+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716830+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:43:51.716830+0000","last_peered":"2023-09-23T04:43:51.716830+0000","last_clean":"2023-09-23T04:43:51.716830+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:43:51.716830+0000","last_undegraded":"2023-09-23T04:43:51.716830+0000","last_fullsized":"2023-09-23T04:43:51.716830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711937+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:43:51.711937+0000","last_peered":"2023-09-23T04:43:51.711937+0000","last_clean":"2023-09-23T04:43:51.711937+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:43:51.711937+0000","last_undegraded":"2023-09-23T04:43:51.711937+0000","last_fullsized":"2023-09-23T04:43:51.711937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713477+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:43:52.713477+0000","last_peered":"2023-09-23T04:43:52.713477+0000","last_clean":"2023-09-23T04:43:52.713477+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:43:52.713477+0000","last_undegraded":"2023-09-23T04:43:52.713477+0000","last_fullsized":"2023-09-23T04:43:52.713477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711884+0000","last_change":"2023-09-23T04:43:46.589899+0000","last_active":"2023-09-23T04:43:51.711884+0000","last_peered":"2023-09-23T04:43:51.711884+0000","last_clean":"2023-09-23T04:43:51.711884+0000","last_became_active":"2023-09-23T04:43:46.589550+0000","last_became_peered":"2023-09-23T04:43:46.589550+0000","last_unstale":"2023-09-23T04:43:51.711884+0000","last_undegraded":"2023-09-23T04:43:51.711884+0000","last_fullsized":"2023-09-23T04:43:51.711884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:55:05.083411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711818+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:43:51.711818+0000","last_peered":"2023-09-23T04:43:51.711818+0000","last_clean":"2023-09-23T04:43:51.711818+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:43:51.711818+0000","last_undegraded":"2023-09-23T04:43:51.711818+0000","last_fullsized":"2023-09-23T04:43:51.711818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.713648+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:43:52.713648+0000","last_peered":"2023-09-23T04:43:52.713648+0000","last_clean":"2023-09-23T04:43:52.713648+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:43:52.713648+0000","last_undegraded":"2023-09-23T04:43:52.713648+0000","last_fullsized":"2023-09-23T04:43:52.713648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.721201+0000","last_change":"2023-09-23T04:43:47.587791+0000","last_active":"2023-09-23T04:43:51.721201+0000","last_peered":"2023-09-23T04:43:51.721201+0000","last_clean":"2023-09-23T04:43:51.721201+0000","last_became_active":"2023-09-23T04:43:47.587346+0000","last_became_peered":"2023-09-23T04:43:47.587346+0000","last_unstale":"2023-09-23T04:43:51.721201+0000","last_undegraded":"2023-09-23T04:43:51.721201+0000","last_fullsized":"2023-09-23T04:43:51.721201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:13:40.267593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.452275+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:44:20.452275+0000","last_peered":"2023-09-23T04:44:20.452275+0000","last_clean":"2023-09-23T04:44:20.452275+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:44:20.452275+0000","last_undegraded":"2023-09-23T04:44:20.452275+0000","last_fullsized":"2023-09-23T04:44:20.452275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.750010+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:43:52.750010+0000","last_peered":"2023-09-23T04:43:52.750010+0000","last_clean":"2023-09-23T04:43:52.750010+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:43:52.750010+0000","last_undegraded":"2023-09-23T04:43:52.750010+0000","last_fullsized":"2023-09-23T04:43:52.750010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718239+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:43:51.718239+0000","last_peered":"2023-09-23T04:43:51.718239+0000","last_clean":"2023-09-23T04:43:51.718239+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:43:51.718239+0000","last_undegraded":"2023-09-23T04:43:51.718239+0000","last_fullsized":"2023-09-23T04:43:51.718239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:20.456019+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:44:20.456019+0000","last_peered":"2023-09-23T04:44:20.456019+0000","last_clean":"2023-09-23T04:44:20.456019+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:44:20.456019+0000","last_undegraded":"2023-09-23T04:44:20.456019+0000","last_fullsized":"2023-09-23T04:44:20.456019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":321,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711304+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:43:51.711304+0000","last_peered":"2023-09-23T04:43:51.711304+0000","last_clean":"2023-09-23T04:43:51.711304+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:43:51.711304+0000","last_undegraded":"2023-09-23T04:43:51.711304+0000","last_fullsized":"2023-09-23T04:43:51.711304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748404+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:43:52.748404+0000","last_peered":"2023-09-23T04:43:52.748404+0000","last_clean":"2023-09-23T04:43:52.748404+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:43:52.748404+0000","last_undegraded":"2023-09-23T04:43:52.748404+0000","last_fullsized":"2023-09-23T04:43:52.748404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.710891+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:43:51.710891+0000","last_peered":"2023-09-23T04:43:51.710891+0000","last_clean":"2023-09-23T04:43:51.710891+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:43:51.710891+0000","last_undegraded":"2023-09-23T04:43:51.710891+0000","last_fullsized":"2023-09-23T04:43:51.710891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711468+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:43:51.711468+0000","last_peered":"2023-09-23T04:43:51.711468+0000","last_clean":"2023-09-23T04:43:51.711468+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:43:51.711468+0000","last_undegraded":"2023-09-23T04:43:51.711468+0000","last_fullsized":"2023-09-23T04:43:51.711468+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749286+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:43:52.749286+0000","last_peered":"2023-09-23T04:43:52.749286+0000","last_clean":"2023-09-23T04:43:52.749286+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:43:52.749286+0000","last_undegraded":"2023-09-23T04:43:52.749286+0000","last_fullsized":"2023-09-23T04:43:52.749286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.714024+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:43:52.714024+0000","last_peered":"2023-09-23T04:43:52.714024+0000","last_clean":"2023-09-23T04:43:52.714024+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:43:52.714024+0000","last_undegraded":"2023-09-23T04:43:52.714024+0000","last_fullsized":"2023-09-23T04:43:52.714024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711188+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:43:51.711188+0000","last_peered":"2023-09-23T04:43:51.711188+0000","last_clean":"2023-09-23T04:43:51.711188+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:43:51.711188+0000","last_undegraded":"2023-09-23T04:43:51.711188+0000","last_fullsized":"2023-09-23T04:43:51.711188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747559+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:43:52.747559+0000","last_peered":"2023-09-23T04:43:52.747559+0000","last_clean":"2023-09-23T04:43:52.747559+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:43:52.747559+0000","last_undegraded":"2023-09-23T04:43:52.747559+0000","last_fullsized":"2023-09-23T04:43:52.747559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.795036+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:44:17.795036+0000","last_peered":"2023-09-23T04:44:17.795036+0000","last_clean":"2023-09-23T04:44:17.795036+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:44:17.795036+0000","last_undegraded":"2023-09-23T04:44:17.795036+0000","last_fullsized":"2023-09-23T04:44:17.795036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717512+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:43:51.717512+0000","last_peered":"2023-09-23T04:43:51.717512+0000","last_clean":"2023-09-23T04:43:51.717512+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:43:51.717512+0000","last_undegraded":"2023-09-23T04:43:51.717512+0000","last_fullsized":"2023-09-23T04:43:51.717512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747141+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:43:52.747141+0000","last_peered":"2023-09-23T04:43:52.747141+0000","last_clean":"2023-09-23T04:43:52.747141+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:43:52.747141+0000","last_undegraded":"2023-09-23T04:43:52.747141+0000","last_fullsized":"2023-09-23T04:43:52.747141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:19.816655+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:44:19.816655+0000","last_peered":"2023-09-23T04:44:19.816655+0000","last_clean":"2023-09-23T04:44:19.816655+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:44:19.816655+0000","last_undegraded":"2023-09-23T04:44:19.816655+0000","last_fullsized":"2023-09-23T04:44:19.816655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719714+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:43:51.719714+0000","last_peered":"2023-09-23T04:43:51.719714+0000","last_clean":"2023-09-23T04:43:51.719714+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:43:51.719714+0000","last_undegraded":"2023-09-23T04:43:51.719714+0000","last_fullsized":"2023-09-23T04:43:51.719714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.749137+0000","last_change":"2023-09-23T04:43:51.737574+0000","last_active":"2023-09-23T04:43:52.749137+0000","last_peered":"2023-09-23T04:43:52.749137+0000","last_clean":"2023-09-23T04:43:52.749137+0000","last_became_active":"2023-09-23T04:43:51.736981+0000","last_became_peered":"2023-09-23T04:43:51.736981+0000","last_unstale":"2023-09-23T04:43:52.749137+0000","last_undegraded":"2023-09-23T04:43:52.749137+0000","last_fullsized":"2023-09-23T04:43:52.749137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:59:43.525900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.779556+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:44:16.779556+0000","last_peered":"2023-09-23T04:44:16.779556+0000","last_clean":"2023-09-23T04:44:16.779556+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:44:16.779556+0000","last_undegraded":"2023-09-23T04:44:16.779556+0000","last_fullsized":"2023-09-23T04:44:16.779556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.715950+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:43:51.715950+0000","last_peered":"2023-09-23T04:43:51.715950+0000","last_clean":"2023-09-23T04:43:51.715950+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:43:51.715950+0000","last_undegraded":"2023-09-23T04:43:51.715950+0000","last_fullsized":"2023-09-23T04:43:51.715950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748382+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:43:52.748382+0000","last_peered":"2023-09-23T04:43:52.748382+0000","last_clean":"2023-09-23T04:43:52.748382+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:43:52.748382+0000","last_undegraded":"2023-09-23T04:43:52.748382+0000","last_fullsized":"2023-09-23T04:43:52.748382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.782955+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:44:16.782955+0000","last_peered":"2023-09-23T04:44:16.782955+0000","last_clean":"2023-09-23T04:44:16.782955+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:44:16.782955+0000","last_undegraded":"2023-09-23T04:44:16.782955+0000","last_fullsized":"2023-09-23T04:44:16.782955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716753+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:43:51.716753+0000","last_peered":"2023-09-23T04:43:51.716753+0000","last_clean":"2023-09-23T04:43:51.716753+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:43:51.716753+0000","last_undegraded":"2023-09-23T04:43:51.716753+0000","last_fullsized":"2023-09-23T04:43:51.716753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748956+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:43:52.748956+0000","last_peered":"2023-09-23T04:43:52.748956+0000","last_clean":"2023-09-23T04:43:52.748956+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:43:52.748956+0000","last_undegraded":"2023-09-23T04:43:52.748956+0000","last_fullsized":"2023-09-23T04:43:52.748956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716713+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:43:51.716713+0000","last_peered":"2023-09-23T04:43:51.716713+0000","last_clean":"2023-09-23T04:43:51.716713+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:43:51.716713+0000","last_undegraded":"2023-09-23T04:43:51.716713+0000","last_fullsized":"2023-09-23T04:43:51.716713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717716+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:43:51.717716+0000","last_peered":"2023-09-23T04:43:51.717716+0000","last_clean":"2023-09-23T04:43:51.717716+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:43:51.717716+0000","last_undegraded":"2023-09-23T04:43:51.717716+0000","last_fullsized":"2023-09-23T04:43:51.717716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.748965+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:43:52.748965+0000","last_peered":"2023-09-23T04:43:52.748965+0000","last_clean":"2023-09-23T04:43:52.748965+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:43:52.748965+0000","last_undegraded":"2023-09-23T04:43:52.748965+0000","last_fullsized":"2023-09-23T04:43:52.748965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:16.781950+0000","last_change":"2023-09-23T04:43:46.590551+0000","last_active":"2023-09-23T04:44:16.781950+0000","last_peered":"2023-09-23T04:44:16.781950+0000","last_clean":"2023-09-23T04:44:16.781950+0000","last_became_active":"2023-09-23T04:43:46.590310+0000","last_became_peered":"2023-09-23T04:43:46.590310+0000","last_unstale":"2023-09-23T04:44:16.781950+0000","last_undegraded":"2023-09-23T04:44:16.781950+0000","last_fullsized":"2023-09-23T04:44:16.781950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:54.678727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711994+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:43:51.711994+0000","last_peered":"2023-09-23T04:43:51.711994+0000","last_clean":"2023-09-23T04:43:51.711994+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:43:51.711994+0000","last_undegraded":"2023-09-23T04:43:51.711994+0000","last_fullsized":"2023-09-23T04:43:51.711994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747503+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:43:52.747503+0000","last_peered":"2023-09-23T04:43:52.747503+0000","last_clean":"2023-09-23T04:43:52.747503+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:43:52.747503+0000","last_undegraded":"2023-09-23T04:43:52.747503+0000","last_fullsized":"2023-09-23T04:43:52.747503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:17.793364+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:44:17.793364+0000","last_peered":"2023-09-23T04:44:17.793364+0000","last_clean":"2023-09-23T04:44:17.793364+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:44:17.793364+0000","last_undegraded":"2023-09-23T04:44:17.793364+0000","last_fullsized":"2023-09-23T04:44:17.793364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.712075+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:43:51.712075+0000","last_peered":"2023-09-23T04:43:51.712075+0000","last_clean":"2023-09-23T04:43:51.712075+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:43:51.712075+0000","last_undegraded":"2023-09-23T04:43:51.712075+0000","last_fullsized":"2023-09-23T04:43:51.712075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.716174+0000","last_change":"2023-09-23T04:43:51.741159+0000","last_active":"2023-09-23T04:43:52.716174+0000","last_peered":"2023-09-23T04:43:52.716174+0000","last_clean":"2023-09-23T04:43:52.716174+0000","last_became_active":"2023-09-23T04:43:51.740696+0000","last_became_peered":"2023-09-23T04:43:51.740696+0000","last_unstale":"2023-09-23T04:43:52.716174+0000","last_undegraded":"2023-09-23T04:43:52.716174+0000","last_fullsized":"2023-09-23T04:43:52.716174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:34.832089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:18.801125+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:44:18.801125+0000","last_peered":"2023-09-23T04:44:18.801125+0000","last_clean":"2023-09-23T04:44:18.801125+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:44:18.801125+0000","last_undegraded":"2023-09-23T04:44:18.801125+0000","last_fullsized":"2023-09-23T04:44:18.801125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711797+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:43:51.711797+0000","last_peered":"2023-09-23T04:43:51.711797+0000","last_clean":"2023-09-23T04:43:51.711797+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:43:51.711797+0000","last_undegraded":"2023-09-23T04:43:51.711797+0000","last_fullsized":"2023-09-23T04:43:51.711797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:43:52.747848+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:43:52.747848+0000","last_peered":"2023-09-23T04:43:52.747848+0000","last_clean":"2023-09-23T04:43:52.747848+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:43:52.747848+0000","last_undegraded":"2023-09-23T04:43:52.747848+0000","last_fullsized":"2023-09-23T04:43:52.747848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-09-23T04:44:19.817229+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:44:19.817229+0000","last_peered":"2023-09-23T04:44:19.817229+0000","last_clean":"2023-09-23T04:44:19.817229+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:44:19.817229+0000","last_undegraded":"2023-09-23T04:44:19.817229+0000","last_fullsized":"2023-09-23T04:44:19.817229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711841+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:43:51.711841+0000","last_peered":"2023-09-23T04:43:51.711841+0000","last_clean":"2023-09-23T04:43:51.711841+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:43:51.711841+0000","last_undegraded":"2023-09-23T04:43:51.711841+0000","last_fullsized":"2023-09-23T04:43:51.711841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716844+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:43:51.716844+0000","last_peered":"2023-09-23T04:43:51.716844+0000","last_clean":"2023-09-23T04:43:51.716844+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:43:51.716844+0000","last_undegraded":"2023-09-23T04:43:51.716844+0000","last_fullsized":"2023-09-23T04:43:51.716844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718946+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:43:51.718946+0000","last_peered":"2023-09-23T04:43:51.718946+0000","last_clean":"2023-09-23T04:43:51.718946+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:43:51.718946+0000","last_undegraded":"2023-09-23T04:43:51.718946+0000","last_fullsized":"2023-09-23T04:43:51.718946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716624+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:43:51.716624+0000","last_peered":"2023-09-23T04:43:51.716624+0000","last_clean":"2023-09-23T04:43:51.716624+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:43:51.716624+0000","last_undegraded":"2023-09-23T04:43:51.716624+0000","last_fullsized":"2023-09-23T04:43:51.716624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719350+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:43:51.719350+0000","last_peered":"2023-09-23T04:43:51.719350+0000","last_clean":"2023-09-23T04:43:51.719350+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:43:51.719350+0000","last_undegraded":"2023-09-23T04:43:51.719350+0000","last_fullsized":"2023-09-23T04:43:51.719350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711381+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:43:51.711381+0000","last_peered":"2023-09-23T04:43:51.711381+0000","last_clean":"2023-09-23T04:43:51.711381+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:43:51.711381+0000","last_undegraded":"2023-09-23T04:43:51.711381+0000","last_fullsized":"2023-09-23T04:43:51.711381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719943+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:43:51.719943+0000","last_peered":"2023-09-23T04:43:51.719943+0000","last_clean":"2023-09-23T04:43:51.719943+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:43:51.719943+0000","last_undegraded":"2023-09-23T04:43:51.719943+0000","last_fullsized":"2023-09-23T04:43:51.719943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717089+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:43:51.717089+0000","last_peered":"2023-09-23T04:43:51.717089+0000","last_clean":"2023-09-23T04:43:51.717089+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:43:51.717089+0000","last_undegraded":"2023-09-23T04:43:51.717089+0000","last_fullsized":"2023-09-23T04:43:51.717089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.717606+0000","last_change":"2023-09-23T04:43:47.596319+0000","last_active":"2023-09-23T04:43:51.717606+0000","last_peered":"2023-09-23T04:43:51.717606+0000","last_clean":"2023-09-23T04:43:51.717606+0000","last_became_active":"2023-09-23T04:43:47.595532+0000","last_became_peered":"2023-09-23T04:43:47.595532+0000","last_unstale":"2023-09-23T04:43:51.717606+0000","last_undegraded":"2023-09-23T04:43:51.717606+0000","last_fullsized":"2023-09-23T04:43:51.717606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:04.830870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.721112+0000","last_change":"2023-09-23T04:43:47.592560+0000","last_active":"2023-09-23T04:43:51.721112+0000","last_peered":"2023-09-23T04:43:51.721112+0000","last_clean":"2023-09-23T04:43:51.721112+0000","last_became_active":"2023-09-23T04:43:47.592259+0000","last_became_peered":"2023-09-23T04:43:47.592259+0000","last_unstale":"2023-09-23T04:43:51.721112+0000","last_undegraded":"2023-09-23T04:43:51.721112+0000","last_fullsized":"2023-09-23T04:43:51.721112+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:32:01.023978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.719380+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:43:51.719380+0000","last_peered":"2023-09-23T04:43:51.719380+0000","last_clean":"2023-09-23T04:43:51.719380+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:43:51.719380+0000","last_undegraded":"2023-09-23T04:43:51.719380+0000","last_fullsized":"2023-09-23T04:43:51.719380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.718855+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:43:51.718855+0000","last_peered":"2023-09-23T04:43:51.718855+0000","last_clean":"2023-09-23T04:43:51.718855+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:43:51.718855+0000","last_undegraded":"2023-09-23T04:43:51.718855+0000","last_fullsized":"2023-09-23T04:43:51.718855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.711153+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:43:51.711153+0000","last_peered":"2023-09-23T04:43:51.711153+0000","last_clean":"2023-09-23T04:43:51.711153+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:43:51.711153+0000","last_undegraded":"2023-09-23T04:43:51.711153+0000","last_fullsized":"2023-09-23T04:43:51.711153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-09-23T04:43:51.716397+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:43:51.716397+0000","last_peered":"2023-09-23T04:43:51.716397+0000","last_clean":"2023-09-23T04:43:51.716397+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:43:51.716397+0000","last_undegraded":"2023-09-23T04:43:51.716397+0000","last_fullsized":"2023-09-23T04:43:51.716397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8982,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":145,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":23180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":469360,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.433},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59199999999999997}]},{"osd":1,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63400000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622}]},{"osd":6,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.3540000000000001,"5min":4.3540000000000001,"15min":4.3540000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.91300000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.93500000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1059860,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.32900000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.8220000000000001,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.746}]},{"osd":9,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.64900000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76300000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":464890,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66800000000000004}]},{"osd":4,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.72099999999999997}]},{"osd":5,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.82599999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.628}]},{"osd":7,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.85899999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.79500000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.874},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103198,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":464810,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.747,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.73299999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.48699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.89200000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42599999999999999,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56599999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.77000000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81100000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":1.0109999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.76200000000000001,"5min":1.008,"15min":1.008},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.52200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.94499999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.258},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85499999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":1.052}]},{"osd":7,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66700000000000004,"15min":0.66100000000000003},"min":{"1min":0.497,"5min":0.435,"15min":0.435},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97299999999999998,"5min":1.284,"15min":1.284},"last":1.0700000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.23}]},{"osd":10,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":1.1839999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1054749,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.441,"15min":0.439},"min":{"1min":0.29599999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.624}]},{"osd":3,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.49099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.40400000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45200000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.90900000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1,"5min":1,"15min":1},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57699999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92800000000000005,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.73999999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":464794,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.59899999999999998,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.2750000000000004,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.57999999999999996,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.748,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.2480000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.012,"5min":0.79900000000000004,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.68899999999999995}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.83099999999999996,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77600000000000002,"15min":0.76300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.81799999999999995,"15min":0.80600000000000005},"min":{"1min":0.62,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.86399999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.998,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.63100000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.109,"15min":0.98799999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":15.34,"5min":15.34,"15min":15.34},"last":0.873},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":467641,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54300000000000004,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.504}]},{"osd":2,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.373},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.44700000000000001,"15min":0.437},"min":{"1min":0.33800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.47399999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40000000000000002}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.46899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":1.014,"15min":1.014},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76300000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.626},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49099999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.96999999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":0.997,"15min":0.997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0669999999999999,"5min":1.103,"15min":1.103},"last":1.004}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.71699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":464162,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.42299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.014,"5min":1.345,"15min":1.345},"last":0.59599999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.623,"15min":0.57799999999999996},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.091,"5min":1.306,"15min":1.306},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.75600000000000001}]},{"osd":3,"last update":"Sat Sep 23 04:42:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.53400000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.624}]},{"osd":4,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.65300000000000002,"15min":0.61199999999999999},"min":{"1min":0.46500000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.046,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.58999999999999997,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81599999999999995}]},{"osd":5,"last update":"Sat Sep 23 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98499999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.94999999999999996}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.753,"15min":0.755},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Sat Sep 23 04:43:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88400000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.82699999999999996},"min":{"1min":0.53500000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.004,"5min":1.091,"15min":1.091},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.94399999999999995,"15min":0.97199999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0389999999999999,"5min":1.871,"15min":1.871},"last":0.97499999999999998}]},{"osd":9,"last update":"Sat Sep 23 04:43:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.77700000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:43:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.93799999999999994}]},{"osd":11,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.95999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":468570,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001}]},{"osd":1,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.372,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.745,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69699999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:43:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.56699999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":4.2290000000000001,"15min":4.2290000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.79800000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.442,"15min":0.434},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46999999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.624}]},{"osd":7,"last update":"Sat Sep 23 04:43:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.83999999999999997,"15min":0.87},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.73599999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.85899999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55700000000000005}]},{"osd":9,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.86399999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":464025,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.434,"15min":0.39300000000000002},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.376}]},{"osd":1,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.52200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.627}]},{"osd":4,"last update":"Sat Sep 23 04:43:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.63100000000000001,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.59099999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96299999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:43:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.83199999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.752},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.504,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.99299999999999999,"5min":0.999,"15min":0.999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.214,"15min":1.214},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.45300000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79300000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72499999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.57599999999999996}]},{"osd":10,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":465162,"data_compressed":11744,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.23400000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.30499999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72999999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65400000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56499999999999995,"15min":0.53100000000000003},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.90200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.52200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.89900000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76000000000000001,"15min":0.76500000000000001},"min":{"1min":0.56799999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93400000000000005}]},{"osd":9,"last update":"Sat Sep 23 04:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.91200000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:43:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1057068,"data_compressed":48001,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83699999999999997,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81899999999999995,"5min":1.246,"15min":1.246},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52300000000000002,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76700000000000002,"5min":1.115,"15min":1.115},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55100000000000005,"15min":0.52700000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89900000000000002,"5min":1.071,"15min":1.071},"last":0.60799999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38200000000000001,"15min":0.36599999999999999},"min":{"1min":0.314,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.629,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.439},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.41499999999999998,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40600000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57399999999999995,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.92400000000000004,"5min":1.212,"15min":1.212},"last":0.627},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72899999999999998,"5min":1.042,"15min":1.042},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56000000000000005,"15min":0.53000000000000003},"min":{"1min":0.441,"5min":0.374,"15min":0.374},"max":{"1min":0.81200000000000006,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":1.23,"15min":1.23},"last":0.67500000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:43:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66300000000000003,"15min":0.65500000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.623,"15min":0.623},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84499999999999997,"5min":1.105,"15min":1.105},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.629,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79000000000000004,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54300000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:43:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.752}]},{"osd":10,"last update":"Sat Sep 23 04:43:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:44:23.314 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T04:44:23.314 INFO:tasks.cephadm:Setting up client nodes... 2023-09-23T04:44:23.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-23T04:44:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:24.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-23T04:44:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:23 smithi005 ceph-mon[97638]: pgmap v287: 129 pgs: 129 active+clean; 587 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2023-09-23T04:44:24.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-23T04:44:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:23 smithi118 ceph-mon[134297]: pgmap v287: 129 pgs: 129 active+clean; 587 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]: dispatch 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-09-23T04:44:24.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-09-23T04:44:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:23 smithi069 ceph-mon[133931]: pgmap v287: 129 pgs: 129 active+clean; 587 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-09-23T04:44:25.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]': finished 2023-09-23T04:44:25.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-23T04:44:25.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2023-09-23T04:44:25.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-23T04:44:25.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:24 smithi005 ceph-mon[97638]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2023-09-23T04:44:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-23T04:44:25.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:24 smithi118 ceph-mon[134297]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-09-23T04:44:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7]}]': finished 2023-09-23T04:44:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2023-09-23T04:44:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2023-09-23T04:44:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-09-23T04:44:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:24 smithi069 ceph-mon[133931]: osdmap e88: 12 total, 12 up, 12 in 2023-09-23T04:44:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 -- 172.21.15.5:0/4188760969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102a50 msgr2=0x7f3040109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 --2- 172.21.15.5:0/4188760969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102a50 0x7f3040109370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f303800b0a0 tx=0x7f303800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 -- 172.21.15.5:0/4188760969 shutdown_connections 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 --2- 172.21.15.5:0/4188760969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102a50 0x7f3040109370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 --2- 172.21.15.5:0/4188760969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f30401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 --2- 172.21.15.5:0/4188760969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102090 0x7f3040102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 -- 172.21.15.5:0/4188760969 >> 172.21.15.5:0/4188760969 conn(0x7f30400fc470 msgr2=0x7f30400fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.732+0000 7f30457da700 1 -- 172.21.15.5:0/4188760969 shutdown_connections 2023-09-23T04:44:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.733+0000 7f30457da700 1 -- 172.21.15.5:0/4188760969 wait complete. 2023-09-23T04:44:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 Processor -- start 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 -- start start 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102a50 0x7f3040073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3040073ed0 con 0x7f3040102090 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3040074040 con 0x7f3040100ea0 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f30457da700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a0d00 con 0x7f3040102a50 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f303e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f303effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f303effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45830/0 (socket says 172.21.15.5:45830) 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f303e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43900/0 (socket says 172.21.15.5:43900) 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.734+0000 7f303effd700 1 -- 172.21.15.5:0/1871557133 learned_addr learned my addr 172.21.15.5:0/1871557133 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303f7fe700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102a50 0x7f3040073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 -- 172.21.15.5:0/1871557133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102a50 msgr2=0x7f3040073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102a50 0x7f3040073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 -- 172.21.15.5:0/1871557133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 msgr2=0x7f3040074bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 -- 172.21.15.5:0/1871557133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009d00 con 0x7f3040102090 2023-09-23T04:44:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303effd700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f303e7fc700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f303400e9c0 tx=0x7f303400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034004220 con 0x7f3040102090 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3034004380 con 0x7f3040102090 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.735+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30401a0fe0 con 0x7f3040102090 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.736+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034003be0 con 0x7f3040102090 2023-09-23T04:44:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.736+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401a1450 con 0x7f3040102090 2023-09-23T04:44:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.737+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3040107dc0 con 0x7f3040102090 2023-09-23T04:44:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.739+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f3034003d40 con 0x7f3040102090 2023-09-23T04:44:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.741+0000 7f3027fff700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 0x7f30280a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.741+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3034014070 con 0x7f3040102090 2023-09-23T04:44:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.741+0000 7f303effd700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 0x7f30280a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.742+0000 7f303effd700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 0x7f30280a3af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3030005950 tx=0x7f3030009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:25.745+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f30340967d0 con 0x7f3040102090 2023-09-23T04:44:26.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.035+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f304010c770 con 0x7f3040102090 2023-09-23T04:44:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.037+0000 7f3027fff700 1 -- 172.21.15.5:0/1871557133 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3034099df0 con 0x7f3040102090 2023-09-23T04:44:26.037 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2023-09-23T04:44:26.038 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAqbQ5lqcElAhAATKcwDkUZL34XnFWD7s/lSA== 2023-09-23T04:44:26.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 msgr2=0x7f30280a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:26.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 0x7f30280a3af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3030005950 tx=0x7f3030009260 comp rx=0 tx=0).stop 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 msgr2=0x7f3040073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f303400e9c0 tx=0x7f303400ecd0 comp rx=0 tx=0).stop 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 shutdown_connections 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f30280a1630 0x7f30280a3af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3040102a50 0x7f3040073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:26.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3040100ea0 0x7f3040074bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 --2- 172.21.15.5:0/1871557133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3040102090 0x7f3040073210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 >> 172.21.15.5:0/1871557133 conn(0x7f30400fc470 msgr2=0x7f30400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 shutdown_connections 2023-09-23T04:44:26.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:26.040+0000 7f30457da700 1 -- 172.21.15.5:0/1871557133 wait complete. 2023-09-23T04:44:26.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:25 smithi005 ceph-mon[97638]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T04:44:26.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:25 smithi005 ceph-mon[97638]: pgmap v290: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 29 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:25 smithi118 ceph-mon[134297]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T04:44:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:25 smithi118 ceph-mon[134297]: pgmap v290: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 29 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:25 smithi069 ceph-mon[133931]: osdmap e89: 12 total, 12 up, 12 in 2023-09-23T04:44:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:25 smithi069 ceph-mon[133931]: pgmap v290: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 KiB/s wr, 29 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:26.592 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:44:26.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-23T04:44:26.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-23T04:44:26.695 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-23T04:44:26.695 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-23T04:44:26.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph mgr dump --format=json 2023-09-23T04:44:26.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:26 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T04:44:26.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:26 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T04:44:27.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:26 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T04:44:27.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:26 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T04:44:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:26 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-23T04:44:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:26 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1871557133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-23T04:44:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:27 smithi005 ceph-mon[97638]: pgmap v291: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 24 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:27 smithi118 ceph-mon[134297]: pgmap v291: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 24 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:28.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:27 smithi069 ceph-mon[133931]: pgmap v291: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 24 op/s; 76 B/s, 0 objects/s recovering 2023-09-23T04:44:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 -- 172.21.15.5:0/1282137146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 msgr2=0x7f6d9c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 --2- 172.21.15.5:0/1282137146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c101480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 -- 172.21.15.5:0/1282137146 shutdown_connections 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 --2- 172.21.15.5:0/1282137146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 0x7f6d9c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 --2- 172.21.15.5:0/1282137146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c101480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 --2- 172.21.15.5:0/1282137146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.980+0000 7f6da2130700 1 -- 172.21.15.5:0/1282137146 >> 172.21.15.5:0/1282137146 conn(0x7f6d9c0fc910 msgr2=0x7f6d9c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.981+0000 7f6da2130700 1 -- 172.21.15.5:0/1282137146 shutdown_connections 2023-09-23T04:44:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.981+0000 7f6da2130700 1 -- 172.21.15.5:0/1282137146 wait complete. 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.981+0000 7f6da2130700 1 Processor -- start 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 -- start start 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 0x7f6d9c073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1a5410 con 0x7f6d9c1010a0 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1a5580 con 0x7f6d9c101a50 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6da2130700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1a5740 con 0x7f6d9c106180 2023-09-23T04:44:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6d9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6d9b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43922/0 (socket says 172.21.15.5:43922) 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6d9affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.982+0000 7f6d9b7fe700 1 -- 172.21.15.5:0/1552793558 learned_addr learned my addr 172.21.15.5:0/1552793558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9bfff700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 0x7f6d9c073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 -- 172.21.15.5:0/1552793558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 msgr2=0x7f6d9c073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 0x7f6d9c073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 -- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 msgr2=0x7f6d9c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 -- 172.21.15.5:0/1552793558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0095f0 con 0x7f6d9c101a50 2023-09-23T04:44:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9b7fe700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d9affd700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c073660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000e9c0 tx=0x7f6d9000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.983+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000c9c0 con 0x7f6d9c101a50 2023-09-23T04:44:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.984+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c1a5a20 con 0x7f6d9c101a50 2023-09-23T04:44:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.984+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d90010430 con 0x7f6d9c101a50 2023-09-23T04:44:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.984+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90018680 con 0x7f6d9c101a50 2023-09-23T04:44:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.984+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c1a5f70 con 0x7f6d9c101a50 2023-09-23T04:44:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.985+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9c04eb20 con 0x7f6d9c101a50 2023-09-23T04:44:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.987+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f6d9000cb20 con 0x7f6d9c101a50 2023-09-23T04:44:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.988+0000 7f6d98ff9700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 0x7f6d840a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.988+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6d90014070 con 0x7f6d9c101a50 2023-09-23T04:44:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.989+0000 7f6d9b7fe700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 0x7f6d840a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.989+0000 7f6d9b7fe700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 0x7f6d840a3af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b6f0 tx=0x7f6d8c005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:28.990+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6d900968e0 con 0x7f6d9c101a50 2023-09-23T04:44:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.260+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6d9c110e90 con 0x7f6d9c101a50 2023-09-23T04:44:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.266+0000 7f6d98ff9700 1 -- 172.21.15.5:0/1552793558 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260587 (secure 0 0 0) 0x7f6d90099f00 con 0x7f6d9c101a50 2023-09-23T04:44:29.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.270+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 msgr2=0x7f6d840a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.270+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 0x7f6d840a3af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b6f0 tx=0x7f6d8c005bc0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.270+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 msgr2=0x7f6d9c073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.270+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c073660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000e9c0 tx=0x7f6d9000ecd0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.270+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 shutdown_connections 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6d840a1630 0x7f6d840a3af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6d9c106180 0x7f6d9c073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d9c1010a0 0x7f6d9c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 --2- 172.21.15.5:0/1552793558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c101a50 0x7f6d9c073660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 >> 172.21.15.5:0/1552793558 conn(0x7f6d9c0fc910 msgr2=0x7f6d9c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 shutdown_connections 2023-09-23T04:44:29.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:29.271+0000 7f6da2130700 1 -- 172.21.15.5:0/1552793558 wait complete. 2023-09-23T04:44:29.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:29 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1552793558' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T04:44:29.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:29 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1552793558' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T04:44:29.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:29 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1552793558' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-23T04:44:29.763 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":4010853589},{"type":"v1","addr":"172.21.15.5:6801","nonce":4010853589}]},"active_addr":"172.21.15.5:6801/4010853589","active_change":"2023-09-23T04:37:23.553199+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1767282859}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":734408633}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3587586220}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2853321891}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3116664994}]}]} 2023-09-23T04:44:29.769 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-23T04:44:29.769 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-23T04:44:29.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T04:44:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:30.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:30 smithi005 ceph-mon[97638]: pgmap v292: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 21 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:30.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:30 smithi118 ceph-mon[134297]: pgmap v292: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 21 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:30.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:30 smithi069 ceph-mon[133931]: pgmap v292: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 21 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 -- 172.21.15.5:0/1087138560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba001010a0 msgr2=0x7fba00101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 --2- 172.21.15.5:0/1087138560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba001010a0 0x7fba00101480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009990 tx=0x7fb9f0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 -- 172.21.15.5:0/1087138560 shutdown_connections 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 --2- 172.21.15.5:0/1087138560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00106180 0x7fba0010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 --2- 172.21.15.5:0/1087138560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba001010a0 0x7fba00101480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 --2- 172.21.15.5:0/1087138560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba00101a50 0x7fba00105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 -- 172.21.15.5:0/1087138560 >> 172.21.15.5:0/1087138560 conn(0x7fba000fc910 msgr2=0x7fba000fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 -- 172.21.15.5:0/1087138560 shutdown_connections 2023-09-23T04:44:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.021+0000 7fba04b37700 1 -- 172.21.15.5:0/1087138560 wait complete. 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.022+0000 7fba04b37700 1 Processor -- start 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.022+0000 7fba04b37700 1 -- start start 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 0x7fba001974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 0x7fba0019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0019bf50 con 0x7fba00106180 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0019c0c0 con 0x7fba00101a50 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fba04b37700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0019c280 con 0x7fba001010a0 2023-09-23T04:44:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fb9fed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 0x7fba0019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fb9fdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 0x7fba001974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34902/0 (socket says 172.21.15.5:34902) 2023-09-23T04:44:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.023+0000 7fb9fe59c700 1 -- 172.21.15.5:0/260742862 learned_addr learned my addr 172.21.15.5:0/260742862 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 -- 172.21.15.5:0/260742862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 msgr2=0x7fba001974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 0x7fba001974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 -- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 msgr2=0x7fba0019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 0x7fba0019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 -- 172.21.15.5:0/260742862 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00095f0 con 0x7fba001010a0 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fdd9b700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 0x7fba001974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fed9d700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 0x7fba0019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9fe59c700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005910 tx=0x7fb9f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f001d070 con 0x7fba001010a0 2023-09-23T04:44:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0019c500 con 0x7fba001010a0 2023-09-23T04:44:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9f0022470 con 0x7fba001010a0 2023-09-23T04:44:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0017650 con 0x7fba001010a0 2023-09-23T04:44:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.024+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba001a3dc0 con 0x7fba001010a0 2023-09-23T04:44:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.025+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0004eb20 con 0x7fba001010a0 2023-09-23T04:44:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.028+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fb9f000fad0 con 0x7fba001010a0 2023-09-23T04:44:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.029+0000 7fb9ef7fe700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 0x7fb9e80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.029+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9f00cec50 con 0x7fba001010a0 2023-09-23T04:44:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.029+0000 7fb9fdd9b700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 0x7fb9e80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.030+0000 7fb9fdd9b700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 0x7fb9e80a3af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba001985b0 tx=0x7fb9f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:32.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.031+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb9f0097e00 con 0x7fba001010a0 2023-09-23T04:44:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.239+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba00110e00 con 0x7fba001010a0 2023-09-23T04:44:32.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.241+0000 7fb9ef7fe700 1 -- 172.21.15.5:0/260742862 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24393 (secure 0 0 0) 0x7fb9f009b420 con 0x7fba001010a0 2023-09-23T04:44:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:32.242 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:44:24.800265+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 msgr2=0x7fb9e80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 0x7fb9e80a3af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba001985b0 tx=0x7fb9f4009260 comp rx=0 tx=0).stop 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 msgr2=0x7fba00196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005910 tx=0x7fb9f0004730 comp rx=0 tx=0).stop 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 shutdown_connections 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9e80a1630 0x7fb9e80a3af0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba00106180 0x7fba0019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fba001010a0 0x7fba00196f90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 --2- 172.21.15.5:0/260742862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba00101a50 0x7fba001974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 >> 172.21.15.5:0/260742862 conn(0x7fba000fc910 msgr2=0x7fba000fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 shutdown_connections 2023-09-23T04:44:32.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:32.244+0000 7fba04b37700 1 -- 172.21.15.5:0/260742862 wait complete. 2023-09-23T04:44:32.707 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-23T04:44:32.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd dump --format=json 2023-09-23T04:44:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:32 smithi005 ceph-mon[97638]: pgmap v293: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 20 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:32 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/260742862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:32 smithi069 ceph-mon[133931]: pgmap v293: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 20 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:32 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/260742862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:32 smithi118 ceph-mon[134297]: pgmap v293: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 20 op/s; 90 B/s, 2 objects/s recovering 2023-09-23T04:44:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:32 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/260742862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:33.446 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.979+0000 7f540e0c6700 1 -- 172.21.15.5:0/1672791737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408101a50 msgr2=0x7f5408105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.979+0000 7f540e0c6700 1 --2- 172.21.15.5:0/1672791737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408101a50 0x7f5408105aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009990 tx=0x7f53fc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 -- 172.21.15.5:0/1672791737 shutdown_connections 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 --2- 172.21.15.5:0/1672791737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408106180 0x7f540810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 --2- 172.21.15.5:0/1672791737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 --2- 172.21.15.5:0/1672791737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408101a50 0x7f5408105aa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 -- 172.21.15.5:0/1672791737 >> 172.21.15.5:0/1672791737 conn(0x7f54080fc910 msgr2=0x7f54080fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 -- 172.21.15.5:0/1672791737 shutdown_connections 2023-09-23T04:44:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.980+0000 7f540e0c6700 1 -- 172.21.15.5:0/1672791737 wait complete. 2023-09-23T04:44:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 Processor -- start 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 -- start start 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408101a50 0x7f5408197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408106180 0x7f540819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819bf80 con 0x7f5408106180 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819c0f0 con 0x7f5408101a50 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.981+0000 7f540e0c6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819c2b0 con 0x7f54081010a0 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34926/0 (socket says 172.21.15.5:34926) 2023-09-23T04:44:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 -- 172.21.15.5:0/192018855 learned_addr learned my addr 172.21.15.5:0/192018855 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f5407fff700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408106180 0x7f540819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f5406ffd700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408101a50 0x7f5408197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 -- 172.21.15.5:0/192018855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408101a50 msgr2=0x7f5408197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408101a50 0x7f5408197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 -- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408106180 msgr2=0x7f540819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408106180 0x7f540819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 -- 172.21.15.5:0/192018855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0095f0 con 0x7f54081010a0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.982+0000 7f54077fe700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9a0 tx=0x7f53f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.983+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8009790 con 0x7f54081010a0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.983+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540819c590 con 0x7f54081010a0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.983+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f8010430 con 0x7f54081010a0 2023-09-23T04:44:34.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.983+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8018680 con 0x7f54081010a0 2023-09-23T04:44:34.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.983+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54081a3df0 con 0x7f54081010a0 2023-09-23T04:44:34.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.984+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540804eb20 con 0x7f54081010a0 2023-09-23T04:44:34.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.986+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f53f80105a0 con 0x7f54081010a0 2023-09-23T04:44:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.987+0000 7f5404ff9700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 0x7f53f00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.987+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53f8014070 con 0x7f54081010a0 2023-09-23T04:44:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.987+0000 7f5406ffd700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 0x7f53f00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.988+0000 7f5406ffd700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 0x7f53f00a3af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54081985e0 tx=0x7f53fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:34.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:34.990+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f53f80966f0 con 0x7f54081010a0 2023-09-23T04:44:34.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:34 smithi069 ceph-mon[133931]: pgmap v294: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 73 B/s, 1 objects/s recovering 2023-09-23T04:44:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:34 smithi005 ceph-mon[97638]: pgmap v294: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 73 B/s, 1 objects/s recovering 2023-09-23T04:44:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:34 smithi118 ceph-mon[134297]: pgmap v294: 129 pgs: 129 active+clean; 630 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 73 B/s, 1 objects/s recovering 2023-09-23T04:44:35.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.201+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5408110df0 con 0x7f54081010a0 2023-09-23T04:44:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.203+0000 7f5404ff9700 1 -- 172.21.15.5:0/192018855 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24393 (secure 0 0 0) 0x7f53f8099d10 con 0x7f54081010a0 2023-09-23T04:44:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:44:35.203 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:44:24.800265+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 msgr2=0x7f53f00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 0x7f53f00a3af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54081985e0 tx=0x7f53fc005fb0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 msgr2=0x7f5408196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9a0 tx=0x7f53f800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 shutdown_connections 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53f00a1630 0x7f53f00a3af0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5408106180 0x7f540819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f54081010a0 0x7f5408196fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 --2- 172.21.15.5:0/192018855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5408101a50 0x7f5408197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:35.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 >> 172.21.15.5:0/192018855 conn(0x7f54080fc910 msgr2=0x7f54080fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:35.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 shutdown_connections 2023-09-23T04:44:35.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:35.205+0000 7f540e0c6700 1 -- 172.21.15.5:0/192018855 wait complete. 2023-09-23T04:44:35.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-09-23T04:44:35.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-09-23T04:44:35.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-09-23T04:44:35.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-09-23T04:44:35.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-09-23T04:44:35.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-09-23T04:44:35.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-09-23T04:44:35.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-09-23T04:44:35.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-09-23T04:44:35.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-09-23T04:44:35.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-09-23T04:44:35.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-09-23T04:44:35.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:35 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/192018855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:35 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/192018855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:36.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:35 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/192018855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:44:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:36 smithi069 ceph-mon[133931]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 66 B/s, 1 objects/s recovering 2023-09-23T04:44:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:36 smithi005 ceph-mon[97638]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 66 B/s, 1 objects/s recovering 2023-09-23T04:44:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:36 smithi118 ceph-mon[134297]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s; 66 B/s, 1 objects/s recovering 2023-09-23T04:44:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:37.812 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.189 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.393 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.514 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.839 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:38 smithi069 ceph-mon[133931]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:39.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:38 smithi005 ceph-mon[97638]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:38 smithi118 ceph-mon[134297]: pgmap v296: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:39.359 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:40 smithi005 ceph-mon[97638]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:40 smithi069 ceph-mon[133931]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:40 smithi118 ceph-mon[134297]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2023-09-23T04:44:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:42 smithi069 ceph-mon[133931]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-23T04:44:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:42 smithi005 ceph-mon[97638]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-23T04:44:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:42 smithi118 ceph-mon[134297]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-23T04:44:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 -- 172.21.15.5:0/3953083994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c106180 msgr2=0x7f1c7c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 --2- 172.21.15.5:0/3953083994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c106180 0x7f1c7c10dd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400b0a0 tx=0x7f1c7400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 -- 172.21.15.5:0/3953083994 shutdown_connections 2023-09-23T04:44:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 --2- 172.21.15.5:0/3953083994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c106180 0x7f1c7c10dd50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 --2- 172.21.15.5:0/3953083994 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c7c1010a0 0x7f1c7c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 --2- 172.21.15.5:0/3953083994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c7c101a50 0x7f1c7c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 -- 172.21.15.5:0/3953083994 >> 172.21.15.5:0/3953083994 conn(0x7f1c7c0fc910 msgr2=0x7f1c7c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 -- 172.21.15.5:0/3953083994 shutdown_connections 2023-09-23T04:44:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.232+0000 7f1c81268700 1 -- 172.21.15.5:0/3953083994 wait complete. 2023-09-23T04:44:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.233+0000 7f1c81268700 1 Processor -- start 2023-09-23T04:44:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.233+0000 7f1c81268700 1 -- start start 2023-09-23T04:44:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c7c1010a0 0x7f1c7c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 0x7f1c7c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c7c106180 0x7f1c7c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c19e220 con 0x7f1c7c101a50 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c19e390 con 0x7f1c7c106180 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c81268700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c19e550 con 0x7f1c7c1010a0 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 0x7f1c7c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 0x7f1c7c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54960/0 (socket says 172.21.15.5:54960) 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7a7fc700 1 -- 172.21.15.5:0/1992816538 learned_addr learned my addr 172.21.15.5:0/1992816538 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7affd700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c7c1010a0 0x7f1c7c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7b7fe700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c7c106180 0x7f1c7c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7a7fc700 1 -- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c7c1010a0 msgr2=0x7f1c7c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.234+0000 7f1c7a7fc700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1c7c1010a0 0x7f1c7c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c7a7fc700 1 -- 172.21.15.5:0/1992816538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c7c106180 msgr2=0x7f1c7c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c7a7fc700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c7c106180 0x7f1c7c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c7a7fc700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009d00 con 0x7f1c7c101a50 2023-09-23T04:44:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c7a7fc700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 0x7f1c7c1997a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000e970 tx=0x7f1c7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70009790 con 0x7f1c7c101a50 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c19e830 con 0x7f1c7c101a50 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7000c7e0 con 0x7f1c7c101a50 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700054e0 con 0x7f1c7c101a50 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.235+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c1a6090 con 0x7f1c7c101a50 2023-09-23T04:44:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.236+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c7c111080 con 0x7f1c7c101a50 2023-09-23T04:44:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.238+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f1c7000c950 con 0x7f1c7c101a50 2023-09-23T04:44:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c63fff700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1c640a1630 0x7f1c640a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c70014070 con 0x7f1c7c101a50 2023-09-23T04:44:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c7affd700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1c640a1630 0x7f1c640a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c63fff700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] conn(0x7f1c640ac650 0x7f1c640aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c640af1c0 con 0x7f1c640ac650 2023-09-23T04:44:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.240+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c700cdbb0 con 0x7f1c7c101a50 2023-09-23T04:44:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.241+0000 7f1c7b7fe700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] conn(0x7f1c640ac650 0x7f1c640aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.241+0000 7f1c7affd700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1c640a1630 0x7f1c640a3af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009c60 tx=0x7f1c6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.241+0000 7f1c7b7fe700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] conn(0x7f1c640ac650 0x7f1c640aeaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.242+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== osd.8 v2:172.21.15.118:6816/1078105960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1c640af1c0 con 0x7f1c640ac650 2023-09-23T04:44:43.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.267+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 --> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c7c04eb20 con 0x7f1c640ac650 2023-09-23T04:44:43.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.268+0000 7f1c63fff700 1 -- 172.21.15.5:0/1992816538 <== osd.8 v2:172.21.15.118:6816/1078105960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c7c04eb20 con 0x7f1c640ac650 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] conn(0x7f1c640ac650 msgr2=0x7f1c640aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] conn(0x7f1c640ac650 0x7f1c640aeaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1c640a1630 msgr2=0x7f1c640a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1c640a1630 0x7f1c640a3af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009c60 tx=0x7f1c6c009260 comp rx=0 tx=0).stop 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 msgr2=0x7f1c7c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 --2- 172.21.15.5:0/1992816538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c7c101a50 0x7f1c7c1997a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000e970 tx=0x7f1c7000ec80 comp rx=0 tx=0).stop 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c7b7fe700 1 -- 172.21.15.5:0/1992816538 reap_dead start 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 shutdown_connections 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 >> 172.21.15.5:0/1992816538 conn(0x7f1c7c0fc910 msgr2=0x7f1c7c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 shutdown_connections 2023-09-23T04:44:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:43.269+0000 7f1c81268700 1 -- 172.21.15.5:0/1992816538 wait complete. 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 -- 172.21.15.5:0/3028967336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c1010a0 msgr2=0x7f3e8c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/3028967336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c101480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009990 tx=0x7f3e88009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 -- 172.21.15.5:0/3028967336 shutdown_connections 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/3028967336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c106180 0x7f3e8c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/3028967336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c101480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/3028967336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8c101a50 0x7f3e8c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.707+0000 7f3e93bc8700 1 -- 172.21.15.5:0/3028967336 >> 172.21.15.5:0/3028967336 conn(0x7f3e8c0fc910 msgr2=0x7f3e8c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.708+0000 7f3e93bc8700 1 -- 172.21.15.5:0/3028967336 shutdown_connections 2023-09-23T04:44:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.708+0000 7f3e93bc8700 1 -- 172.21.15.5:0/3028967336 wait complete. 2023-09-23T04:44:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.709+0000 7f3e93bc8700 1 Processor -- start 2023-09-23T04:44:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.709+0000 7f3e93bc8700 1 -- start start 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8c101a50 0x7f3e8c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 0x7f3e8c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19bef0 con 0x7f3e8c106180 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19c060 con 0x7f3e8c1010a0 2023-09-23T04:44:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e93bc8700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c19c220 con 0x7f3e8c101a50 2023-09-23T04:44:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e92165700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 0x7f3e8c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e92165700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 0x7f3e8c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54986/0 (socket says 172.21.15.5:54986) 2023-09-23T04:44:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e92165700 1 -- 172.21.15.5:0/1223411772 learned_addr learned my addr 172.21.15.5:0/1223411772 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e91964700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.710+0000 7f3e91163700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8c101a50 0x7f3e8c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 -- 172.21.15.5:0/1223411772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8c101a50 msgr2=0x7f3e8c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3e8c101a50 0x7f3e8c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 -- 172.21.15.5:0/1223411772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1010a0 msgr2=0x7f3e8c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880095f0 con 0x7f3e8c106180 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e91964700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e8c1010a0 0x7f3e8c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e92165700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 0x7f3e8c19b8e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400ef30 tx=0x7f3e8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84018070 con 0x7f3e8c106180 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c19c4a0 con 0x7f3e8c106180 2023-09-23T04:44:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e8400f040 con 0x7f3e8c106180 2023-09-23T04:44:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.711+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400cd00 con 0x7f3e8c106180 2023-09-23T04:44:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.712+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c1a3d60 con 0x7f3e8c106180 2023-09-23T04:44:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.713+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e8c04eb20 con 0x7f3e8c106180 2023-09-23T04:44:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.715+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f3e84003a00 con 0x7f3e8c106180 2023-09-23T04:44:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.716+0000 7f3e82ffd700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3e780a1630 0x7f3e780a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.716+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3e840cd860 con 0x7f3e8c106180 2023-09-23T04:44:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e91964700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3e780a1630 0x7f3e780a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e82ffd700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] conn(0x7f3e780ac650 0x7f3e780aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e780af1c0 con 0x7f3e780ac650 2023-09-23T04:44:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e840cdbd0 con 0x7f3e8c106180 2023-09-23T04:44:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e91163700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] conn(0x7f3e780ac650 0x7f3e780aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.717+0000 7f3e91964700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3e780a1630 0x7f3e780a3af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800b2e0 tx=0x7f3e88005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.718+0000 7f3e91163700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] conn(0x7f3e780ac650 0x7f3e780aeaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.719+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== osd.3 v2:172.21.15.5:6810/4155254124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3e780af1c0 con 0x7f3e780ac650 2023-09-23T04:44:44.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.737+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 --> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e8c111090 con 0x7f3e780ac650 2023-09-23T04:44:44.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e82ffd700 1 -- 172.21.15.5:0/1223411772 <== osd.3 v2:172.21.15.5:6810/4155254124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3e7c007c70 con 0x7f3e780ac650 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] conn(0x7f3e780ac650 msgr2=0x7f3e780aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] conn(0x7f3e780ac650 0x7f3e780aeaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3e780a1630 msgr2=0x7f3e780a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3e780a1630 0x7f3e780a3af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800b2e0 tx=0x7f3e88005960 comp rx=0 tx=0).stop 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 msgr2=0x7f3e8c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e93bc8700 1 --2- 172.21.15.5:0/1223411772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e8c106180 0x7f3e8c19b8e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400ef30 tx=0x7f3e8400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.738+0000 7f3e92165700 1 -- 172.21.15.5:0/1223411772 reap_dead start 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.739+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 shutdown_connections 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.739+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 >> 172.21.15.5:0/1223411772 conn(0x7f3e8c0fc910 msgr2=0x7f3e8c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:44.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.739+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 shutdown_connections 2023-09-23T04:44:44.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:44.739+0000 7f3e93bc8700 1 -- 172.21.15.5:0/1223411772 wait complete. 2023-09-23T04:44:44.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:44 smithi005 ceph-mon[97638]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-23T04:44:44.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:44 smithi069 ceph-mon[133931]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-23T04:44:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:44 smithi118 ceph-mon[134297]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-23T04:44:45.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 -- 172.21.15.5:0/519446879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c101a50 msgr2=0x7fa94c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 --2- 172.21.15.5:0/519446879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c101a50 0x7fa94c105aa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009990 tx=0x7fa93c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 -- 172.21.15.5:0/519446879 shutdown_connections 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 --2- 172.21.15.5:0/519446879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c106180 0x7fa94c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 --2- 172.21.15.5:0/519446879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c1010a0 0x7fa94c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 --2- 172.21.15.5:0/519446879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c101a50 0x7fa94c105aa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 -- 172.21.15.5:0/519446879 >> 172.21.15.5:0/519446879 conn(0x7fa94c0fc910 msgr2=0x7fa94c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 -- 172.21.15.5:0/519446879 shutdown_connections 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.135+0000 7fa953c67700 1 -- 172.21.15.5:0/519446879 wait complete. 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.136+0000 7fa953c67700 1 Processor -- start 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.136+0000 7fa953c67700 1 -- start start 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 0x7fa94c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c101a50 0x7fa94c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c106180 0x7fa94c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19e0c0 con 0x7fa94c1010a0 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19e230 con 0x7fa94c101a50 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa953c67700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c19e3f0 con 0x7fa94c106180 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa951a03700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 0x7fa94c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa951a03700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 0x7fa94c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55008/0 (socket says 172.21.15.5:55008) 2023-09-23T04:44:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa951a03700 1 -- 172.21.15.5:0/1536942841 learned_addr learned my addr 172.21.15.5:0/1536942841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa951202700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c101a50 0x7fa94c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.137+0000 7fa952204700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c106180 0x7fa94c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 -- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c106180 msgr2=0x7fa94c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa94c106180 0x7fa94c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 -- 172.21.15.5:0/1536942841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c101a50 msgr2=0x7fa94c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c101a50 0x7fa94c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0095f0 con 0x7fa94c1010a0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951202700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa94c101a50 0x7fa94c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa951a03700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 0x7fa94c199100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e9a0 tx=0x7fa94800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948009790 con 0x7fa94c1010a0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa948010430 con 0x7fa94c1010a0 2023-09-23T04:44:45.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c19e6d0 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948018680 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.138+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c1a5f30 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.139+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa94c111030 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.142+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fa9480105a0 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.143+0000 7fa942ffd700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9380a16b0 0x7fa9380a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.143+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa948014070 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.143+0000 7fa951202700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9380a16b0 0x7fa9380a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.144+0000 7fa942ffd700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] conn(0x7fa9380ac6d0 0x7fa9380aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.144+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9380af240 con 0x7fa9380ac6d0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.144+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9480cdb10 con 0x7fa94c1010a0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.144+0000 7fa951202700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9380a16b0 0x7fa9380a3b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006010 tx=0x7fa93c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.144+0000 7fa952204700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] conn(0x7fa9380ac6d0 0x7fa9380aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.145+0000 7fa952204700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] conn(0x7fa9380ac6d0 0x7fa9380aeb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.146+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== osd.5 v2:172.21.15.118:6808/2643723046 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa9380af240 con 0x7fa9380ac6d0 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.163+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 --> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa94c04eb20 con 0x7fa9380ac6d0 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa942ffd700 1 -- 172.21.15.5:0/1536942841 <== osd.5 v2:172.21.15.118:6808/2643723046 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa94c04eb20 con 0x7fa9380ac6d0 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] conn(0x7fa9380ac6d0 msgr2=0x7fa9380aeb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] conn(0x7fa9380ac6d0 0x7fa9380aeb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9380a16b0 msgr2=0x7fa9380a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa9380a16b0 0x7fa9380a3b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa93c006010 tx=0x7fa93c005960 comp rx=0 tx=0).stop 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 msgr2=0x7fa94c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 --2- 172.21.15.5:0/1536942841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1010a0 0x7fa94c199100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e9a0 tx=0x7fa94800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa952204700 1 -- 172.21.15.5:0/1536942841 reap_dead start 2023-09-23T04:44:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 shutdown_connections 2023-09-23T04:44:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.164+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 >> 172.21.15.5:0/1536942841 conn(0x7fa94c0fc910 msgr2=0x7fa94c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.165+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 shutdown_connections 2023-09-23T04:44:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.165+0000 7fa953c67700 1 -- 172.21.15.5:0/1536942841 wait complete. 2023-09-23T04:44:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.176+0000 7f36091e6700 1 -- 172.21.15.5:0/3916966187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604073ae0 msgr2=0x7f3604111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.176+0000 7f36091e6700 1 --2- 172.21.15.5:0/3916966187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604073ae0 0x7f3604111910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009990 tx=0x7f35f8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.176+0000 7f36091e6700 1 -- 172.21.15.5:0/3916966187 shutdown_connections 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.176+0000 7f36091e6700 1 --2- 172.21.15.5:0/3916966187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604111e50 0x7f3604114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.177+0000 7f36091e6700 1 --2- 172.21.15.5:0/3916966187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 0x7f36040735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.177+0000 7f36091e6700 1 --2- 172.21.15.5:0/3916966187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604073ae0 0x7f3604111910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.177+0000 7f36091e6700 1 -- 172.21.15.5:0/3916966187 >> 172.21.15.5:0/3916966187 conn(0x7f36040fc910 msgr2=0x7f36040fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.177+0000 7f36091e6700 1 -- 172.21.15.5:0/3916966187 shutdown_connections 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.177+0000 7f36091e6700 1 -- 172.21.15.5:0/3916966187 wait complete. 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 Processor -- start 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 -- start start 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 0x7f360419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 0x7f360419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604111e50 0x7f36041a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a2560 con 0x7f3604073ae0 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a26d0 con 0x7f36040731c0 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f36091e6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36041a2890 con 0x7f3604111e50 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.178+0000 7f360259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 0x7f360419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 0x7f360419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55026/0 (socket says 172.21.15.5:55026) 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 -- 172.21.15.5:0/3672934041 learned_addr learned my addr 172.21.15.5:0/3672934041 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f3602d9d700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 0x7f360419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360359e700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604111e50 0x7f36041a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 -- 172.21.15.5:0/3672934041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604111e50 msgr2=0x7f36041a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3604111e50 0x7f36041a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 -- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 msgr2=0x7f360419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 0x7f360419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80095f0 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f3602d9d700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f36040731c0 0x7f360419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.179+0000 7f360259c700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 0x7f360419dae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005910 tx=0x7f35f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.180+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f801d070 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.180+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36041a2b10 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.180+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35f8022470 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.180+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8017650 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.180+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041aa3d0 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.181+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35f0000ff0 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.183+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f35f800fad0 con 0x7f3604073ae0 2023-09-23T04:44:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.184+0000 7f35ebfff700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f35ec0a16b0 0x7f35ec0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.184+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35f80cf3f0 con 0x7f3604073ae0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f3602d9d700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f35ec0a16b0 0x7f35ec0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f35ebfff700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] conn(0x7f35ec0ac6d0 0x7f35ec0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35ec0af240 con 0x7f35ec0ac6d0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f80cf800 con 0x7f3604073ae0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f360359e700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] conn(0x7f35ec0ac6d0 0x7f35ec0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.185+0000 7f3602d9d700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f35ec0a16b0 0x7f35ec0a3b70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35f4007fd0 tx=0x7f35f40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.186+0000 7f360359e700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] conn(0x7f35ec0ac6d0 0x7f35ec0aeb70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.187+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== osd.7 v2:172.21.15.69:6816/1999230066 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f35ec0af240 con 0x7f35ec0ac6d0 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.204+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 --> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35f0002d70 con 0x7f35ec0ac6d0 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f35ebfff700 1 -- 172.21.15.5:0/3672934041 <== osd.7 v2:172.21.15.69:6816/1999230066 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f35f0002d70 con 0x7f35ec0ac6d0 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] conn(0x7f35ec0ac6d0 msgr2=0x7f35ec0aeb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] conn(0x7f35ec0ac6d0 0x7f35ec0aeb70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f35ec0a16b0 msgr2=0x7f35ec0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f35ec0a16b0 0x7f35ec0a3b70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35f4007fd0 tx=0x7f35f40073d0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 msgr2=0x7f360419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 --2- 172.21.15.5:0/3672934041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3604073ae0 0x7f360419dae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005910 tx=0x7f35f8005740 comp rx=0 tx=0).stop 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f360359e700 1 -- 172.21.15.5:0/3672934041 reap_dead start 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 shutdown_connections 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.205+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 >> 172.21.15.5:0/3672934041 conn(0x7f36040fc910 msgr2=0x7f36040fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.206+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 shutdown_connections 2023-09-23T04:44:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.206+0000 7f36091e6700 1 -- 172.21.15.5:0/3672934041 wait complete. 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 -- 172.21.15.5:0/1100362991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51501010a0 msgr2=0x7f5150101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 --2- 172.21.15.5:0/1100362991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51501010a0 0x7f5150101480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5144009990 tx=0x7f5144009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 -- 172.21.15.5:0/1100362991 shutdown_connections 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 --2- 172.21.15.5:0/1100362991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5150106180 0x7f515010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 --2- 172.21.15.5:0/1100362991 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f51501010a0 0x7f5150101480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 --2- 172.21.15.5:0/1100362991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5150101a50 0x7f5150105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 -- 172.21.15.5:0/1100362991 >> 172.21.15.5:0/1100362991 conn(0x7f51500fc910 msgr2=0x7f51500fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 -- 172.21.15.5:0/1100362991 shutdown_connections 2023-09-23T04:44:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.443+0000 7f5156c9c700 1 -- 172.21.15.5:0/1100362991 wait complete. 2023-09-23T04:44:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 Processor -- start 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 -- start start 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51501010a0 0x7f5150196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 0x7f51501974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5150106180 0x7f515019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019bf40 con 0x7f5150106180 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019c0b0 con 0x7f51501010a0 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.444+0000 7f5156c9c700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515019c270 con 0x7f5150101a50 2023-09-23T04:44:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 0x7f51501974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 0x7f51501974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40624/0 (socket says 172.21.15.5:40624) 2023-09-23T04:44:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f5154a38700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51501010a0 0x7f5150196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 -- 172.21.15.5:0/170144024 learned_addr learned my addr 172.21.15.5:0/170144024 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f5155239700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5150106180 0x7f515019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 -- 172.21.15.5:0/170144024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51501010a0 msgr2=0x7f5150196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51501010a0 0x7f5150196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 -- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5150106180 msgr2=0x7f515019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5150106180 0x7f515019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.445+0000 7f514ffff700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440095f0 con 0x7f5150101a50 2023-09-23T04:44:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f514ffff700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 0x7f51501974c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f513c00e970 tx=0x7f513c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c009790 con 0x7f5150101a50 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515019c550 con 0x7f5150101a50 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f513c00c7e0 con 0x7f5150101a50 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0054e0 con 0x7f5150101a50 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.446+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51501a3db0 con 0x7f5150101a50 2023-09-23T04:44:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.447+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51501110a0 con 0x7f5150101a50 2023-09-23T04:44:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.450+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f513c00c950 con 0x7f5150101a50 2023-09-23T04:44:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f514dffb700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f51400a16b0 0x7f51400a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f513c014070 con 0x7f5150101a50 2023-09-23T04:44:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f5154a38700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f51400a16b0 0x7f51400a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f514dffb700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] conn(0x7f51400ac6d0 0x7f51400aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51400af240 con 0x7f51400ac6d0 2023-09-23T04:44:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.451+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f513c0cdbc0 con 0x7f5150101a50 2023-09-23T04:44:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.452+0000 7f5155239700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] conn(0x7f51400ac6d0 0x7f51400aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.452+0000 7f5154a38700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f51400a16b0 0x7f51400a3b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f514400b2e0 tx=0x7f5144005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.452+0000 7f5155239700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] conn(0x7f51400ac6d0 0x7f51400aeb70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.453+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== osd.0 v2:172.21.15.5:6802/2285726296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f51400af240 con 0x7f51400ac6d0 2023-09-23T04:44:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.471+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 --> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f515004eb20 con 0x7f51400ac6d0 2023-09-23T04:44:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.471+0000 7f514dffb700 1 -- 172.21.15.5:0/170144024 <== osd.0 v2:172.21.15.5:6802/2285726296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f515004eb20 con 0x7f51400ac6d0 2023-09-23T04:44:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] conn(0x7f51400ac6d0 msgr2=0x7f51400aeb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] conn(0x7f51400ac6d0 0x7f51400aeb70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f51400a16b0 msgr2=0x7f51400a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f51400a16b0 0x7f51400a3b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f514400b2e0 tx=0x7f5144005fd0 comp rx=0 tx=0).stop 2023-09-23T04:44:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 msgr2=0x7f51501974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 --2- 172.21.15.5:0/170144024 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5150101a50 0x7f51501974c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f513c00e970 tx=0x7f513c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5155239700 1 -- 172.21.15.5:0/170144024 reap_dead start 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 shutdown_connections 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 >> 172.21.15.5:0/170144024 conn(0x7f51500fc910 msgr2=0x7f5150104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 shutdown_connections 2023-09-23T04:44:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:45.472+0000 7f5156c9c700 1 -- 172.21.15.5:0/170144024 wait complete. 2023-09-23T04:44:46.329 INFO:teuthology.orchestra.run.smithi005.stdout:107374182459 2023-09-23T04:44:46.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-09-23T04:44:46.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:46 smithi005 ceph-mon[97638]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-23T04:44:46.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:46 smithi069 ceph-mon[133931]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-23T04:44:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:34359738441 2023-09-23T04:44:46.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-09-23T04:44:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 -- 172.21.15.5:0/2350494532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc068940 msgr2=0x7f90fc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 --2- 172.21.15.5:0/2350494532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc068940 0x7f90fc068dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009990 tx=0x7f90f0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 -- 172.21.15.5:0/2350494532 shutdown_connections 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 --2- 172.21.15.5:0/2350494532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90fc069300 0x7f90fc10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 --2- 172.21.15.5:0/2350494532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc106fb0 0x7f90fc107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 --2- 172.21.15.5:0/2350494532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc068940 0x7f90fc068dc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 -- 172.21.15.5:0/2350494532 >> 172.21.15.5:0/2350494532 conn(0x7f90fc076e20 msgr2=0x7f90fc077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 -- 172.21.15.5:0/2350494532 shutdown_connections 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.782+0000 7f91019b6700 1 -- 172.21.15.5:0/2350494532 wait complete. 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.784+0000 7f91019b6700 1 Processor -- start 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.784+0000 7f91019b6700 1 -- start start 2023-09-23T04:44:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc068940 0x7f90fc10ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 0x7f90fc10cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90fc106fb0 0x7f90fc1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1a2490 con 0x7f90fc069300 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1a2600 con 0x7f90fc106fb0 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.785+0000 7f91019b6700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1a27c0 con 0x7f90fc068940 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 0x7f90fc10cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90fc106fb0 0x7f90fc1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90fa7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 0x7f90fc10cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55044/0 (socket says 172.21.15.5:55044) 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90fa7fc700 1 -- 172.21.15.5:0/3989735353 learned_addr learned my addr 172.21.15.5:0/3989735353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc068940 0x7f90fc10ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 -- 172.21.15.5:0/3989735353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90fc106fb0 msgr2=0x7f90fc1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90fc106fb0 0x7f90fc1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 -- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 msgr2=0x7f90fc10cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 0x7f90fc10cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90faffd700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00095f0 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.786+0000 7f90fa7fc700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90fc069300 0x7f90fc10cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f90faffd700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc068940 0x7f90fc10ca80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00e9f0 tx=0x7f90ec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00c9c0 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90ec010430 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec018680 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc1a2aa0 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.787+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1aa300 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.788+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90fc111090 con 0x7f90fc068940 2023-09-23T04:44:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.791+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f90ec010860 con 0x7f90fc068940 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f91009b4700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90e40a1780 0x7f90e40a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90ec014070 con 0x7f90fc068940 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f91009b4700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] conn(0x7f90e40ac7a0 0x7f90e40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90e40af310 con 0x7f90e40ac7a0 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90ec0cda50 con 0x7f90fc068940 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.793+0000 7f90fa7fc700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90e40a1780 0x7f90e40a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.794+0000 7f90fb7fe700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] conn(0x7f90e40ac7a0 0x7f90e40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.794+0000 7f90fa7fc700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90e40a1780 0x7f90e40a3c40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90fc10e0a0 tx=0x7f90f000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.794+0000 7f90fb7fe700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] conn(0x7f90e40ac7a0 0x7f90e40aec40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.795+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== osd.11 v2:172.21.15.118:6824/1838374502 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f90e40af310 con 0x7f90e40ac7a0 2023-09-23T04:44:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.817+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 --> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f90fc04eb20 con 0x7f90e40ac7a0 2023-09-23T04:44:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91009b4700 1 -- 172.21.15.5:0/3989735353 <== osd.11 v2:172.21.15.118:6824/1838374502 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f90fc04eb20 con 0x7f90e40ac7a0 2023-09-23T04:44:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] conn(0x7f90e40ac7a0 msgr2=0x7f90e40aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] conn(0x7f90e40ac7a0 0x7f90e40aec40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90e40a1780 msgr2=0x7f90e40a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f90e40a1780 0x7f90e40a3c40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90fc10e0a0 tx=0x7f90f000b260 comp rx=0 tx=0).stop 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc068940 msgr2=0x7f90fc10ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f91019b6700 1 --2- 172.21.15.5:0/3989735353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f90fc068940 0x7f90fc10ca80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00e9f0 tx=0x7f90ec00ed00 comp rx=0 tx=0).stop 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.818+0000 7f90fb7fe700 1 -- 172.21.15.5:0/3989735353 reap_dead start 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.819+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 shutdown_connections 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.819+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 >> 172.21.15.5:0/3989735353 conn(0x7f90fc076e20 msgr2=0x7f90fc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.819+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 shutdown_connections 2023-09-23T04:44:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:46.819+0000 7f91019b6700 1 -- 172.21.15.5:0/3989735353 wait complete. 2023-09-23T04:44:46.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:46 smithi118 ceph-mon[134297]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 -- 172.21.15.5:0/2990870803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c101a50 msgr2=0x7fcb4c105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 --2- 172.21.15.5:0/2990870803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c101a50 0x7fcb4c105aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009990 tx=0x7fcb40009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 -- 172.21.15.5:0/2990870803 shutdown_connections 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 --2- 172.21.15.5:0/2990870803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 0x7fcb4c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 --2- 172.21.15.5:0/2990870803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c1010a0 0x7fcb4c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 --2- 172.21.15.5:0/2990870803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c101a50 0x7fcb4c105aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 -- 172.21.15.5:0/2990870803 >> 172.21.15.5:0/2990870803 conn(0x7fcb4c0fc910 msgr2=0x7fcb4c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 -- 172.21.15.5:0/2990870803 shutdown_connections 2023-09-23T04:44:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.433+0000 7fcb50a52700 1 -- 172.21.15.5:0/2990870803 wait complete. 2023-09-23T04:44:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.434+0000 7fcb50a52700 1 Processor -- start 2023-09-23T04:44:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.434+0000 7fcb50a52700 1 -- start start 2023-09-23T04:44:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c1010a0 0x7fcb4c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 0x7fcb4c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 0x7fcb4c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19e180 con 0x7fcb4c1010a0 2023-09-23T04:44:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19e2f0 con 0x7fcb4c101a50 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb50a52700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19e4b0 con 0x7fcb4c106180 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb49d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 0x7fcb4c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb4a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c1010a0 0x7fcb4c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb49d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 0x7fcb4c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55620/0 (socket says 172.21.15.5:55620) 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.435+0000 7fcb49d9b700 1 -- 172.21.15.5:0/977739632 learned_addr learned my addr 172.21.15.5:0/977739632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb4ad9d700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 0x7fcb4c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 -- 172.21.15.5:0/977739632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 msgr2=0x7fcb4c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 0x7fcb4c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 -- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c1010a0 msgr2=0x7fcb4c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c1010a0 0x7fcb4c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c009520 con 0x7fcb4c101a50 2023-09-23T04:44:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb4ad9d700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb4c106180 0x7fcb4c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb4a59c700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4c1010a0 0x7fcb4c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb49d9b700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 0x7fcb4c199700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb40005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001d070 con 0x7fcb4c101a50 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb400095f0 con 0x7fcb4c101a50 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40021410 con 0x7fcb4c101a50 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40017650 con 0x7fcb4c101a50 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.436+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c1a5e20 con 0x7fcb4c101a50 2023-09-23T04:44:47.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.437+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb4c04eb20 con 0x7fcb4c101a50 2023-09-23T04:44:47.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.440+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fcb40021580 con 0x7fcb4c101a50 2023-09-23T04:44:47.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.441+0000 7fcb3b7fe700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcb340a1630 0x7fcb340a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:47.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.441+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcb400cf130 con 0x7fcb4c101a50 2023-09-23T04:44:47.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.441+0000 7fcb4a59c700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcb340a1630 0x7fcb340a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.442+0000 7fcb3b7fe700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] conn(0x7fcb340ac650 0x7fcb340aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.442+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb340af1c0 con 0x7fcb340ac650 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.442+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb400cf4a0 con 0x7fcb4c101a50 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.442+0000 7fcb4a59c700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcb340a1630 0x7fcb340a3af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009fd0 tx=0x7fcb3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.442+0000 7fcb4ad9d700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] conn(0x7fcb340ac650 0x7fcb340aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:47.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.443+0000 7fcb4ad9d700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] conn(0x7fcb340ac650 0x7fcb340aeaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:47.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.444+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== osd.1 v2:172.21.15.69:6800/286933964 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcb340af1c0 con 0x7fcb340ac650 2023-09-23T04:44:47.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.460+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 --> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb4c1110c0 con 0x7fcb340ac650 2023-09-23T04:44:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.461+0000 7fcb3b7fe700 1 -- 172.21.15.5:0/977739632 <== osd.1 v2:172.21.15.69:6800/286933964 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb44011040 con 0x7fcb340ac650 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] conn(0x7fcb340ac650 msgr2=0x7fcb340aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] conn(0x7fcb340ac650 0x7fcb340aeaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcb340a1630 msgr2=0x7fcb340a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcb340a1630 0x7fcb340a3af0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009fd0 tx=0x7fcb3c009260 comp rx=0 tx=0).stop 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 msgr2=0x7fcb4c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 --2- 172.21.15.5:0/977739632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb4c101a50 0x7fcb4c199700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb40000c00 tx=0x7fcb40005740 comp rx=0 tx=0).stop 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb4ad9d700 1 -- 172.21.15.5:0/977739632 reap_dead start 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 shutdown_connections 2023-09-23T04:44:47.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 >> 172.21.15.5:0/977739632 conn(0x7fcb4c0fc910 msgr2=0x7fcb4c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:47.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 shutdown_connections 2023-09-23T04:44:47.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:47.462+0000 7fcb50a52700 1 -- 172.21.15.5:0/977739632 wait complete. 2023-09-23T04:44:47.603 INFO:teuthology.orchestra.run.smithi005.stdout:163208757297 2023-09-23T04:44:47.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-09-23T04:44:47.650 INFO:teuthology.orchestra.run.smithi005.stdout:249108103203 2023-09-23T04:44:47.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-09-23T04:44:48.368 INFO:teuthology.orchestra.run.smithi005.stdout:335007449113 2023-09-23T04:44:48.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-09-23T04:44:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.569+0000 7f5402ba7700 1 -- 172.21.15.5:0/2656524491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc069300 msgr2=0x7f53fc10dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.569+0000 7f5402ba7700 1 --2- 172.21.15.5:0/2656524491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc069300 0x7f53fc10dc60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f53f400b0a0 tx=0x7f53f400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 -- 172.21.15.5:0/2656524491 shutdown_connections 2023-09-23T04:44:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 --2- 172.21.15.5:0/2656524491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc069300 0x7f53fc10dc60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 --2- 172.21.15.5:0/2656524491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53fc106fb0 0x7f53fc107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 --2- 172.21.15.5:0/2656524491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53fc068940 0x7f53fc068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 -- 172.21.15.5:0/2656524491 >> 172.21.15.5:0/2656524491 conn(0x7f53fc076e20 msgr2=0x7f53fc077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 -- 172.21.15.5:0/2656524491 shutdown_connections 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.570+0000 7f5402ba7700 1 -- 172.21.15.5:0/2656524491 wait complete. 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 Processor -- start 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 -- start start 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 0x7f53fc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53fc069300 0x7f53fc1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53fc106fb0 0x7f53fc19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19c050 con 0x7f53fc068940 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19c1c0 con 0x7f53fc069300 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.571+0000 7f5402ba7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc19c380 con 0x7f53fc106fb0 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 0x7f53fc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 0x7f53fc197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55066/0 (socket says 172.21.15.5:55066) 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 -- 172.21.15.5:0/3070560181 learned_addr learned my addr 172.21.15.5:0/3070560181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5401144700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53fc106fb0 0x7f53fc19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 -- 172.21.15.5:0/3070560181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53fc106fb0 msgr2=0x7f53fc19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f53fc106fb0 0x7f53fc19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 -- 172.21.15.5:0/3070560181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53fc069300 msgr2=0x7f53fc1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53fc069300 0x7f53fc1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.572+0000 7f5400943700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4009d00 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f5400943700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 0x7f53fc197090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9a0 tx=0x7f53f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0009790 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f000cbd0 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0018680 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc19c660 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.573+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc1a3ec0 con 0x7f53fc068940 2023-09-23T04:44:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.574+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53fc111020 con 0x7f53fc068940 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.577+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f53f000cd40 con 0x7f53fc068940 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.578+0000 7f53f9ffb700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53ec0a1630 0x7f53ec0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53f0014070 con 0x7f53fc068940 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f53fbfff700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53ec0a1630 0x7f53ec0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f53f9ffb700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] conn(0x7f53ec0ac650 0x7f53ec0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53ec0af1c0 con 0x7f53ec0ac650 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53f00cdd20 con 0x7f53fc068940 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.579+0000 7f5401144700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] conn(0x7f53ec0ac650 0x7f53ec0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.580+0000 7f53fbfff700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53ec0a1630 0x7f53ec0a3af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.580+0000 7f5401144700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] conn(0x7f53ec0ac650 0x7f53ec0aeaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.581+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== osd.6 v2:172.21.15.5:6818/1080291272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f53ec0af1c0 con 0x7f53ec0ac650 2023-09-23T04:44:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.599+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 --> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f53fc04eb20 con 0x7f53ec0ac650 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.599+0000 7f53f9ffb700 1 -- 172.21.15.5:0/3070560181 <== osd.6 v2:172.21.15.5:6818/1080291272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f53fc04eb20 con 0x7f53ec0ac650 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] conn(0x7f53ec0ac650 msgr2=0x7f53ec0aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] conn(0x7f53ec0ac650 0x7f53ec0aeaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53ec0a1630 msgr2=0x7f53ec0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f53ec0a1630 0x7f53ec0a3af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 msgr2=0x7f53fc197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 --2- 172.21.15.5:0/3070560181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc068940 0x7f53fc197090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e9a0 tx=0x7f53f000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5401144700 1 -- 172.21.15.5:0/3070560181 reap_dead start 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 shutdown_connections 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 >> 172.21.15.5:0/3070560181 conn(0x7f53fc076e20 msgr2=0x7f53fc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 shutdown_connections 2023-09-23T04:44:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.600+0000 7f5402ba7700 1 -- 172.21.15.5:0/3070560181 wait complete. 2023-09-23T04:44:48.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:48 smithi005 ceph-mon[97638]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-09-23T04:44:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:55834574916 2023-09-23T04:44:48.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 -- 172.21.15.5:0/4164739100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1308073ae0 msgr2=0x7f1308111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 --2- 172.21.15.5:0/4164739100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1308073ae0 0x7f1308111910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098f0 tx=0x7f12fc009c00 comp rx=0 tx=0).stop 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 -- 172.21.15.5:0/4164739100 shutdown_connections 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 --2- 172.21.15.5:0/4164739100 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1308111e50 0x7f1308114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 --2- 172.21.15.5:0/4164739100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13080731c0 0x7f13080735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 --2- 172.21.15.5:0/4164739100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1308073ae0 0x7f1308111910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 -- 172.21.15.5:0/4164739100 >> 172.21.15.5:0/4164739100 conn(0x7f13080fc910 msgr2=0x7f13080fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 -- 172.21.15.5:0/4164739100 shutdown_connections 2023-09-23T04:44:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.926+0000 7f130c8a7700 1 -- 172.21.15.5:0/4164739100 wait complete. 2023-09-23T04:44:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.927+0000 7f130c8a7700 1 Processor -- start 2023-09-23T04:44:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.929+0000 7f130c8a7700 1 -- start start 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13080731c0 0x7f130819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1308073ae0 0x7f130819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308111e50 0x7f13081a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a25a0 con 0x7f1308111e50 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a2710 con 0x7f13080731c0 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130c8a7700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a28d0 con 0x7f1308073ae0 2023-09-23T04:44:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308111e50 0x7f13081a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f130659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13080731c0 0x7f130819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f1305d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1308073ae0 0x7f130819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f1306d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308111e50 0x7f13081a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55084/0 (socket says 172.21.15.5:55084) 2023-09-23T04:44:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.930+0000 7f1306d9d700 1 -- 172.21.15.5:0/2845776990 learned_addr learned my addr 172.21.15.5:0/2845776990 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 -- 172.21.15.5:0/2845776990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1308073ae0 msgr2=0x7f130819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1308073ae0 0x7f130819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 -- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308111e50 msgr2=0x7f13081a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308111e50 0x7f13081a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80095f0 con 0x7f13080731c0 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130659c700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13080731c0 0x7f130819d5e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12f800eb90 tx=0x7f12f800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8009970 con 0x7f13080731c0 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f80044e0 con 0x7f13080731c0 2023-09-23T04:44:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8003e50 con 0x7f13080731c0 2023-09-23T04:44:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc009520 con 0x7f13080731c0 2023-09-23T04:44:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.931+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13081aa240 con 0x7f13080731c0 2023-09-23T04:44:48.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.933+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f130804eb20 con 0x7f13080731c0 2023-09-23T04:44:48.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.935+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f12f8004000 con 0x7f13080731c0 2023-09-23T04:44:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.936+0000 7f12f77fe700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12f00a1630 0x7f12f00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.937+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12f8014070 con 0x7f13080731c0 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.937+0000 7f12f77fe700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] conn(0x7f12f00ac650 0x7f12f00aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.937+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12f00af1c0 con 0x7f12f00ac650 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.937+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f80cdaa0 con 0x7f13080731c0 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.937+0000 7f1305d9b700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12f00a1630 0x7f12f00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.938+0000 7f1306d9d700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] conn(0x7f12f00ac650 0x7f12f00aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.938+0000 7f1305d9b700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12f00a1630 0x7f12f00a3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005e90 tx=0x7f12fc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.938+0000 7f1306d9d700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] conn(0x7f12f00ac650 0x7f12f00aeaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.940+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== osd.9 v2:172.21.15.5:6826/2457675866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f12f00af1c0 con 0x7f12f00ac650 2023-09-23T04:44:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.962+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 --> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1308117710 con 0x7f12f00ac650 2023-09-23T04:44:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.963+0000 7f12f77fe700 1 -- 172.21.15.5:0/2845776990 <== osd.9 v2:172.21.15.5:6826/2457675866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f130000f040 con 0x7f12f00ac650 2023-09-23T04:44:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.963+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] conn(0x7f12f00ac650 msgr2=0x7f12f00aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.963+0000 7f130c8a7700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] conn(0x7f12f00ac650 0x7f12f00aeaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12f00a1630 msgr2=0x7f12f00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12f00a1630 0x7f12f00a3af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005e90 tx=0x7f12fc019040 comp rx=0 tx=0).stop 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13080731c0 msgr2=0x7f130819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 --2- 172.21.15.5:0/2845776990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13080731c0 0x7f130819d5e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12f800eb90 tx=0x7f12f800eea0 comp rx=0 tx=0).stop 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f1306d9d700 1 -- 172.21.15.5:0/2845776990 reap_dead start 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 shutdown_connections 2023-09-23T04:44:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 >> 172.21.15.5:0/2845776990 conn(0x7f13080fc910 msgr2=0x7f13080fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 shutdown_connections 2023-09-23T04:44:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:48.964+0000 7f130c8a7700 1 -- 172.21.15.5:0/2845776990 wait complete. 2023-09-23T04:44:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:219043332137 2023-09-23T04:44:48.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-09-23T04:44:48.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:48 smithi069 ceph-mon[133931]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-09-23T04:44:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:48 smithi118 ceph-mon[134297]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 -- 172.21.15.5:0/3065807274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c068940 msgr2=0x7f7b9c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 --2- 172.21.15.5:0/3065807274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c068940 0x7f7b9c068dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009f90 tx=0x7f7b9400b330 comp rx=0 tx=0).stop 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 -- 172.21.15.5:0/3065807274 shutdown_connections 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 --2- 172.21.15.5:0/3065807274 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c068940 0x7f7b9c068dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 --2- 172.21.15.5:0/3065807274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c105de0 0x7f7b9c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 --2- 172.21.15.5:0/3065807274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c103db0 0x7f7b9c104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.440+0000 7f7ba276b700 1 -- 172.21.15.5:0/3065807274 >> 172.21.15.5:0/3065807274 conn(0x7f7b9c076e20 msgr2=0x7f7b9c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.441+0000 7f7ba276b700 1 -- 172.21.15.5:0/3065807274 shutdown_connections 2023-09-23T04:44:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.441+0000 7f7ba276b700 1 -- 172.21.15.5:0/3065807274 wait complete. 2023-09-23T04:44:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.442+0000 7f7ba276b700 1 Processor -- start 2023-09-23T04:44:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 -- start start 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c068940 0x7f7b9c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c103db0 0x7f7b9c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c105de0 0x7f7b9c1a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a25d0 con 0x7f7b9c068940 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a2740 con 0x7f7b9c105de0 2023-09-23T04:44:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.443+0000 7f7ba276b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c1a2900 con 0x7f7b9c103db0 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.444+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c103db0 0x7f7b9c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.444+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c103db0 0x7f7b9c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40708/0 (socket says 172.21.15.5:40708) 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.444+0000 7f7b9b7fe700 1 -- 172.21.15.5:0/1031161535 learned_addr learned my addr 172.21.15.5:0/1031161535 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.444+0000 7f7ba0d08700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c105de0 0x7f7b9c1a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.445+0000 7f7ba0d08700 1 -- 172.21.15.5:0/1031161535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c103db0 msgr2=0x7f7b9c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.445+0000 7f7ba0d08700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b9c103db0 0x7f7b9c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.445+0000 7f7ba0d08700 1 -- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c068940 msgr2=0x7f7b9c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.445+0000 7f7ba0d08700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b9c068940 0x7f7b9c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.445+0000 7f7ba0d08700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b94009c40 con 0x7f7b9c105de0 2023-09-23T04:44:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.446+0000 7f7ba0d08700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c105de0 0x7f7b9c1a1fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009f90 tx=0x7f7b94009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.448+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400e040 con 0x7f7b9c105de0 2023-09-23T04:44:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.448+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b9401d470 con 0x7f7b9c105de0 2023-09-23T04:44:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.448+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c1a2b80 con 0x7f7b9c105de0 2023-09-23T04:44:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.448+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c1aa440 con 0x7f7b9c105de0 2023-09-23T04:44:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.448+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9401b680 con 0x7f7b9c105de0 2023-09-23T04:44:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.450+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b88000ff0 con 0x7f7b9c105de0 2023-09-23T04:44:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f7b9400b9f0 con 0x7f7b9c105de0 2023-09-23T04:44:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b840a17b0 0x7f7b840a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b940ce8d0 con 0x7f7b9c105de0 2023-09-23T04:44:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b9bfff700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b840a17b0 0x7f7b840a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] conn(0x7f7b840ac6a0 0x7f7b840aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b840af210 con 0x7f7b840ac6a0 2023-09-23T04:44:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b940cec40 con 0x7f7b9c105de0 2023-09-23T04:44:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.472+0000 7f7b9bfff700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b840a17b0 0x7f7b840a3c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.479+0000 7f7b9b7fe700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] conn(0x7f7b840ac6a0 0x7f7b840aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.486+0000 7f7b9b7fe700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] conn(0x7f7b840ac6a0 0x7f7b840aeb40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.487+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== osd.4 v2:172.21.15.69:6808/3327136642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7b840af210 con 0x7f7b840ac6a0 2023-09-23T04:44:49.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.508+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 --> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b88002d70 con 0x7f7b840ac6a0 2023-09-23T04:44:49.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.509+0000 7f7b997fa700 1 -- 172.21.15.5:0/1031161535 <== osd.4 v2:172.21.15.69:6808/3327136642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7b88002d70 con 0x7f7b840ac6a0 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.509+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] conn(0x7f7b840ac6a0 msgr2=0x7f7b840aeb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.509+0000 7f7ba276b700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] conn(0x7f7b840ac6a0 0x7f7b840aeb40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b840a17b0 msgr2=0x7f7b840a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b840a17b0 0x7f7b840a3c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c005ee0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c105de0 msgr2=0x7f7b9c1a1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 --2- 172.21.15.5:0/1031161535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b9c105de0 0x7f7b9c1a1fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009f90 tx=0x7f7b94009500 comp rx=0 tx=0).stop 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba0d08700 1 -- 172.21.15.5:0/1031161535 reap_dead start 2023-09-23T04:44:49.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 shutdown_connections 2023-09-23T04:44:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 >> 172.21.15.5:0/1031161535 conn(0x7f7b9c076e20 msgr2=0x7f7b9c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 shutdown_connections 2023-09-23T04:44:49.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.510+0000 7f7ba276b700 1 -- 172.21.15.5:0/1031161535 wait complete. 2023-09-23T04:44:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 -- 172.21.15.5:0/84565858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc8073250 msgr2=0x7efcc8073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 --2- 172.21.15.5:0/84565858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc8073250 0x7efcc8073630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009990 tx=0x7efcb8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 -- 172.21.15.5:0/84565858 shutdown_connections 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 --2- 172.21.15.5:0/84565858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc810d4c0 0x7efcc810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 --2- 172.21.15.5:0/84565858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc8073250 0x7efcc8073630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 --2- 172.21.15.5:0/84565858 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 -- 172.21.15.5:0/84565858 >> 172.21.15.5:0/84565858 conn(0x7efcc80fc7d0 msgr2=0x7efcc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 -- 172.21.15.5:0/84565858 shutdown_connections 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.795+0000 7efccd507700 1 -- 172.21.15.5:0/84565858 wait complete. 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.796+0000 7efccd507700 1 Processor -- start 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.796+0000 7efccd507700 1 -- start start 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc8073250 0x7efcc8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc81994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc810d4c0 0x7efcc819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819df40 con 0x7efcc810d4c0 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819e0b0 con 0x7efcc8073250 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efccd507700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819e270 con 0x7efcc8073b70 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc81994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcccd06700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc810d4c0 0x7efcc819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc81994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40714/0 (socket says 172.21.15.5:40714) 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 -- 172.21.15.5:0/1031898795 learned_addr learned my addr 172.21.15.5:0/1031898795 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc7fff700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc8073250 0x7efcc8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 -- 172.21.15.5:0/1031898795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc8073250 msgr2=0x7efcc8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc8073250 0x7efcc8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 -- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc810d4c0 msgr2=0x7efcc819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc810d4c0 0x7efcc819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc77fe700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80095f0 con 0x7efcc8073b70 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcc7fff700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc8073250 0x7efcc8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.797+0000 7efcccd06700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc810d4c0 0x7efcc819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:49.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efcc77fe700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc81994c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e9a0 tx=0x7efcbc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc009790 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc819e550 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcbc010430 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc018680 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.798+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc81a5db0 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.799+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efcc81130d0 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.801+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7efcbc0105a0 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.802+0000 7efcc57fa700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a16b0 0x7efcb00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efcbc014070 con 0x7efcc8073b70 2023-09-23T04:44:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc7fff700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a16b0 0x7efcb00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc57fa700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] conn(0x7efcb00ac6d0 0x7efcb00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcb00af240 con 0x7efcb00ac6d0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcbc0cdb90 con 0x7efcc8073b70 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcc7fff700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a16b0 0x7efcb00a3b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b2e0 tx=0x7efcb8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.803+0000 7efcccd06700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] conn(0x7efcb00ac6d0 0x7efcb00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.804+0000 7efcccd06700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] conn(0x7efcb00ac6d0 0x7efcb00aeb70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.805+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== osd.2 v2:172.21.15.118:6800/275897405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7efcb00af240 con 0x7efcb00ac6d0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.822+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 --> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efcc804f370 con 0x7efcb00ac6d0 2023-09-23T04:44:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.822+0000 7efcc57fa700 1 -- 172.21.15.5:0/1031898795 <== osd.2 v2:172.21.15.118:6800/275897405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efcc804f370 con 0x7efcb00ac6d0 2023-09-23T04:44:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] conn(0x7efcb00ac6d0 msgr2=0x7efcb00aeb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] conn(0x7efcb00ac6d0 0x7efcb00aeb70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a16b0 msgr2=0x7efcb00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a16b0 0x7efcb00a3b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b2e0 tx=0x7efcb8005fd0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 msgr2=0x7efcc81994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 --2- 172.21.15.5:0/1031898795 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc8073b70 0x7efcc81994c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e9a0 tx=0x7efcbc00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efcccd06700 1 -- 172.21.15.5:0/1031898795 reap_dead start 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 shutdown_connections 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 >> 172.21.15.5:0/1031898795 conn(0x7efcc80fc7d0 msgr2=0x7efcc810f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 shutdown_connections 2023-09-23T04:44:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:49.823+0000 7efccd507700 1 -- 172.21.15.5:0/1031898795 wait complete. 2023-09-23T04:44:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:49 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:44:49.994 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:49 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:44:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:49 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:44:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:197568495662 2023-09-23T04:44:50.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-09-23T04:44:50.365 INFO:teuthology.orchestra.run.smithi005.stdout:283467841569 2023-09-23T04:44:50.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-09-23T04:44:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 -- 172.21.15.5:0/298319237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb340101a50 msgr2=0x7fb340105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 --2- 172.21.15.5:0/298319237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb340101a50 0x7fb340105aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb330009990 tx=0x7fb330009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 -- 172.21.15.5:0/298319237 shutdown_connections 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 --2- 172.21.15.5:0/298319237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb340106180 0x7fb34010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 --2- 172.21.15.5:0/298319237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3401010a0 0x7fb340101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 --2- 172.21.15.5:0/298319237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb340101a50 0x7fb340105aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.234+0000 7fb347e09700 1 -- 172.21.15.5:0/298319237 >> 172.21.15.5:0/298319237 conn(0x7fb3400fc910 msgr2=0x7fb3400fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.235+0000 7fb347e09700 1 -- 172.21.15.5:0/298319237 shutdown_connections 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.235+0000 7fb347e09700 1 -- 172.21.15.5:0/298319237 wait complete. 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 Processor -- start 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 -- start start 2023-09-23T04:44:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 0x7fb340199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb340101a50 0x7fb340199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb340106180 0x7fb34019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e100 con 0x7fb340101a50 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e270 con 0x7fb340106180 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb347e09700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019e430 con 0x7fb3401010a0 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb345ba5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 0x7fb340199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb345ba5700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 0x7fb340199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40730/0 (socket says 172.21.15.5:40730) 2023-09-23T04:44:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb345ba5700 1 -- 172.21.15.5:0/885482389 learned_addr learned my addr 172.21.15.5:0/885482389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.236+0000 7fb3453a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb340101a50 0x7fb340199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb3463a6700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb340106180 0x7fb34019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 -- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb340106180 msgr2=0x7fb34019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb340106180 0x7fb34019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 -- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb340101a50 msgr2=0x7fb340199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb340101a50 0x7fb340199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300095f0 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb345ba5700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 0x7fb340199140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00e9a0 tx=0x7fb33c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c009790 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34019e710 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb33c010430 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.237+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c018680 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.238+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401a5f70 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.238+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3401110c0 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.241+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fb33c0105a0 con 0x7fb3401010a0 2023-09-23T04:44:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb336ffd700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb32c0a1630 0x7fb32c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb33c014070 con 0x7fb3401010a0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb3453a4700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb32c0a1630 0x7fb32c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb336ffd700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] conn(0x7fb32c0ac650 0x7fb32c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb32c0af1c0 con 0x7fb32c0ac650 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.242+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33c0cdc20 con 0x7fb3401010a0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.243+0000 7fb3463a6700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] conn(0x7fb32c0ac650 0x7fb32c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.243+0000 7fb3453a4700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb32c0a1630 0x7fb32c0a3af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb34019a760 tx=0x7fb330005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.243+0000 7fb3463a6700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] conn(0x7fb32c0ac650 0x7fb32c0aeaf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.244+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== osd.10 v2:172.21.15.69:6824/4220981201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb32c0af1c0 con 0x7fb32c0ac650 2023-09-23T04:44:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.262+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 --> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb34004eb20 con 0x7fb32c0ac650 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb336ffd700 1 -- 172.21.15.5:0/885482389 <== osd.10 v2:172.21.15.69:6824/4220981201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb34004eb20 con 0x7fb32c0ac650 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] conn(0x7fb32c0ac650 msgr2=0x7fb32c0aeaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] conn(0x7fb32c0ac650 0x7fb32c0aeaf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb32c0a1630 msgr2=0x7fb32c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb32c0a1630 0x7fb32c0a3af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb34019a760 tx=0x7fb330005fd0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 msgr2=0x7fb340199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 --2- 172.21.15.5:0/885482389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3401010a0 0x7fb340199140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00e9a0 tx=0x7fb33c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb3463a6700 1 -- 172.21.15.5:0/885482389 reap_dead start 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 shutdown_connections 2023-09-23T04:44:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 >> 172.21.15.5:0/885482389 conn(0x7fb3400fc910 msgr2=0x7fb3400fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 shutdown_connections 2023-09-23T04:44:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:50.263+0000 7fb347e09700 1 -- 172.21.15.5:0/885482389 wait complete. 2023-09-23T04:44:50.946 INFO:teuthology.orchestra.run.smithi005.stdout:77309411394 2023-09-23T04:44:50.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-09-23T04:44:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:50 smithi069 ceph-mon[133931]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:44:51.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:50 smithi005 ceph-mon[97638]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:44:51.104 INFO:teuthology.orchestra.run.smithi005.stdout:304942678044 2023-09-23T04:44:51.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-09-23T04:44:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:50 smithi118 ceph-mon[134297]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:44:51.437 INFO:teuthology.orchestra.run.smithi005.stdout:141733920823 2023-09-23T04:44:51.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-09-23T04:44:51.559 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:51.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.417 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.831 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.831 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:52 smithi005 ceph-mon[97638]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:44:52.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:52 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:52 smithi118 ceph-mon[134297]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:44:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:52 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.967 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:52 smithi069 ceph-mon[133931]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:44:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:52.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:52 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:44:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 -- 172.21.15.5:0/3434581195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768106180 msgr2=0x7fc76810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 --2- 172.21.15.5:0/3434581195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768106180 0x7fc76810dd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b0a0 tx=0x7fc76000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 -- 172.21.15.5:0/3434581195 shutdown_connections 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 --2- 172.21.15.5:0/3434581195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768106180 0x7fc76810dd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 --2- 172.21.15.5:0/3434581195 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 0x7fc768101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 --2- 172.21.15.5:0/3434581195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768101a50 0x7fc768105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 -- 172.21.15.5:0/3434581195 >> 172.21.15.5:0/3434581195 conn(0x7fc7680fc910 msgr2=0x7fc7680fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 -- 172.21.15.5:0/3434581195 shutdown_connections 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.648+0000 7fc76e949700 1 -- 172.21.15.5:0/3434581195 wait complete. 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.649+0000 7fc76e949700 1 Processor -- start 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.649+0000 7fc76e949700 1 -- start start 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 0x7fc768073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 0x7fc768073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a5510 con 0x7fc768106180 2023-09-23T04:44:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a5680 con 0x7fc768101a50 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76e949700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a5840 con 0x7fc7681010a0 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc76cee6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 0x7fc768073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56108/0 (socket says 172.21.15.5:56108) 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc7677fe700 1 -- 172.21.15.5:0/3023946173 learned_addr learned my addr 172.21.15.5:0/3023946173 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.650+0000 7fc767fff700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 0x7fc768073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 -- 172.21.15.5:0/3023946173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 msgr2=0x7fc768073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 0x7fc768073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 -- 172.21.15.5:0/3023946173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 msgr2=0x7fc768073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 -- 172.21.15.5:0/3023946173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009d00 con 0x7fc768106180 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc7677fe700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc76cee6700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 0x7fc768073be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc760009fd0 tx=0x7fc760007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.651+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000e040 con 0x7fc768106180 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.652+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76001d470 con 0x7fc768106180 2023-09-23T04:44:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.652+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7681a5ac0 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.652+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600052a0 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.652+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681a5f80 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.653+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76804eb20 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.655+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fc76001d5e0 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.656+0000 7fc7657fa700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 0x7fc7500a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.656+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7600ce610 con 0x7fc768106180 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.656+0000 7fc767fff700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 0x7fc7500a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.657+0000 7fc767fff700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 0x7fc7500a3b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc758009c60 tx=0x7fc758009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.660+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc7600977c0 con 0x7fc768106180 2023-09-23T04:44:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:54.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:54 smithi005 ceph-mon[97638]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.919+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc768111060 con 0x7fc768106180 2023-09-23T04:44:54.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.920+0000 7fc7657fa700 1 -- 172.21.15.5:0/3023946173 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc76009ade0 con 0x7fc768106180 2023-09-23T04:44:54.921 INFO:teuthology.orchestra.run.smithi005.stdout:34359738443 2023-09-23T04:44:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 msgr2=0x7fc7500a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:54.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 0x7fc7500a3b70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc758009c60 tx=0x7fc758009260 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 msgr2=0x7fc768073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 0x7fc768073be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc760009fd0 tx=0x7fc760007930 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 shutdown_connections 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc7500a16b0 0x7fc7500a3b70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc768106180 0x7fc768073be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc7681010a0 0x7fc768073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 --2- 172.21.15.5:0/3023946173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc768101a50 0x7fc768073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.923+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 >> 172.21.15.5:0/3023946173 conn(0x7fc7680fc910 msgr2=0x7fc7680fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.924+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 shutdown_connections 2023-09-23T04:44:54.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:54.924+0000 7fc76e949700 1 -- 172.21.15.5:0/3023946173 wait complete. 2023-09-23T04:44:54.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:54 smithi069 ceph-mon[133931]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:44:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:54 smithi118 ceph-mon[134297]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:55.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:55 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3023946173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T04:44:55.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:55 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3774712647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T04:44:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:55 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3023946173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T04:44:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:55 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3774712647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T04:44:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:55 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3023946173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-23T04:44:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:55 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3774712647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-23T04:44:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.315+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3558077467 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58101a40 msgr2=0x7fec58105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.315+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3558077467 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58101a40 0x7fec58105a90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec44009990 tx=0x7fec44009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3558077467 shutdown_connections 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3558077467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58106170 0x7fec5810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3558077467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101090 0x7fec58101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3558077467 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58101a40 0x7fec58105a90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3558077467 >> 172.21.15.5:0/3558077467 conn(0x7fec580fc8c0 msgr2=0x7fec580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3558077467 shutdown_connections 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.316+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3558077467 wait complete. 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.317+0000 7fec5ee8d700 1 Processor -- start 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.317+0000 7fec5ee8d700 1 -- start start 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 0x7fec58073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58106170 0x7fec58073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a5440 con 0x7fec58101a40 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a55b0 con 0x7fec58101090 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5ee8d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581a5770 con 0x7fec58106170 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58004/0 (socket says 172.21.15.5:58004) 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5cc29700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 0x7fec58073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec57fff700 1 -- 172.21.15.5:0/3774712647 learned_addr learned my addr 172.21.15.5:0/3774712647 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.318+0000 7fec5d42a700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58106170 0x7fec58073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 -- 172.21.15.5:0/3774712647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58106170 msgr2=0x7fec58073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58106170 0x7fec58073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 -- 172.21.15.5:0/3774712647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 msgr2=0x7fec58073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 0x7fec58073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 -- 172.21.15.5:0/3774712647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440095f0 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec5cc29700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 0x7fec58073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec57fff700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec44004730 tx=0x7fec44004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4401d070 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec581a59f0 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec440047d0 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.319+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440053b0 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.320+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a5e00 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.321+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5804eb20 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.323+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fec44004940 con 0x7fec58101a40 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.324+0000 7fec55ffb700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 0x7fec480a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.324+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec440ce5f0 con 0x7fec58101a40 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.324+0000 7fec5cc29700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 0x7fec480a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.325+0000 7fec5cc29700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 0x7fec480a3af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec4c011490 tx=0x7fec4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.328+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fec4405aec0 con 0x7fec58101a40 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.558+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fec58111000 con 0x7fec58101a40 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.558+0000 7fec55ffb700 1 -- 172.21.15.5:0/3774712647 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec440977a0 con 0x7fec58101a40 2023-09-23T04:44:56.563 INFO:teuthology.orchestra.run.smithi005.stdout:55834574918 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 msgr2=0x7fec480a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 0x7fec480a3af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec4c011490 tx=0x7fec4c009260 comp rx=0 tx=0).stop 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 msgr2=0x7fec58073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec44004730 tx=0x7fec44004760 comp rx=0 tx=0).stop 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 shutdown_connections 2023-09-23T04:44:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fec480a1630 0x7fec480a3af0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fec58106170 0x7fec58073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec58101090 0x7fec58073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 --2- 172.21.15.5:0/3774712647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec58101a40 0x7fec58073660 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 >> 172.21.15.5:0/3774712647 conn(0x7fec580fc8c0 msgr2=0x7fec580fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.561+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 shutdown_connections 2023-09-23T04:44:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:55.562+0000 7fec5ee8d700 1 -- 172.21.15.5:0/3774712647 wait complete. 2023-09-23T04:44:56.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:56 smithi005 ceph-mon[97638]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-23T04:44:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:56 smithi069 ceph-mon[133931]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-23T04:44:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:56 smithi118 ceph-mon[134297]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 -- 172.21.15.5:0/2043966647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c068940 msgr2=0x7f207c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 --2- 172.21.15.5:0/2043966647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c068940 0x7f207c068dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2074009f90 tx=0x7f207400b330 comp rx=0 tx=0).stop 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 -- 172.21.15.5:0/2043966647 shutdown_connections 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 --2- 172.21.15.5:0/2043966647 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c068940 0x7f207c068dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 --2- 172.21.15.5:0/2043966647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 0x7f207c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 --2- 172.21.15.5:0/2043966647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c103db0 0x7f207c104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 -- 172.21.15.5:0/2043966647 >> 172.21.15.5:0/2043966647 conn(0x7f207c076e20 msgr2=0x7f207c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 -- 172.21.15.5:0/2043966647 shutdown_connections 2023-09-23T04:44:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.451+0000 7f208268a700 1 -- 172.21.15.5:0/2043966647 wait complete. 2023-09-23T04:44:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.452+0000 7f208268a700 1 Processor -- start 2023-09-23T04:44:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 -- start start 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 0x7f207c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 0x7f207c1a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1a2600 con 0x7f207c105de0 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1a2770 con 0x7f207c068940 2023-09-23T04:44:57.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.453+0000 7f208268a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1a2930 con 0x7f207c103db0 2023-09-23T04:44:57.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:57.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f2080c27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 0x7f207c1a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:57.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:53518/0 (socket says 172.21.15.5:53518) 2023-09-23T04:44:57.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 -- 172.21.15.5:0/2199806808 learned_addr learned my addr 172.21.15.5:0/2199806808 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:57.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207bfff700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 0x7f207c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 -- 172.21.15.5:0/2199806808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 msgr2=0x7f207c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 0x7f207c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 -- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 msgr2=0x7f207c1a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 0x7f207c1a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 -- 172.21.15.5:0/2199806808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700095f0 con 0x7f207c103db0 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207bfff700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 0x7f207c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.454+0000 7f207b7fe700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f207000ebf0 tx=0x7f207000ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.455+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070009a10 con 0x7f207c103db0 2023-09-23T04:44:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.455+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f207000cd40 con 0x7f207c103db0 2023-09-23T04:44:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.455+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074009c40 con 0x7f207c103db0 2023-09-23T04:44:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.455+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070018820 con 0x7f207c103db0 2023-09-23T04:44:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.455+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c1aa2a0 con 0x7f207c103db0 2023-09-23T04:44:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.457+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c04eb20 con 0x7f207c103db0 2023-09-23T04:44:57.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.459+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f2070018980 con 0x7f207c103db0 2023-09-23T04:44:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.461+0000 7f20797fa700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 0x7f20640a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.461+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2070014070 con 0x7f207c103db0 2023-09-23T04:44:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.461+0000 7f207bfff700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 0x7f20640a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.462+0000 7f207bfff700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 0x7f20640a3b70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f206c005fd0 tx=0x7f206c00a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.465+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2070096b70 con 0x7f207c103db0 2023-09-23T04:44:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.686+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f207c19e9a0 con 0x7f207c103db0 2023-09-23T04:44:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.687+0000 7f20797fa700 1 -- 172.21.15.5:0/2199806808 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2070083e60 con 0x7f207c103db0 2023-09-23T04:44:57.688 INFO:teuthology.orchestra.run.smithi005.stdout:335007449115 2023-09-23T04:44:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 msgr2=0x7f20640a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 0x7f20640a3b70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f206c005fd0 tx=0x7f206c00a5e0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 msgr2=0x7f207c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f207000ebf0 tx=0x7f207000ef00 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 shutdown_connections 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f20640a16b0 0x7f20640a3b70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f207c105de0 0x7f207c1a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f207c068940 0x7f207c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 --2- 172.21.15.5:0/2199806808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f207c103db0 0x7f207c19db80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 >> 172.21.15.5:0/2199806808 conn(0x7f207c076e20 msgr2=0x7f207c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 shutdown_connections 2023-09-23T04:44:57.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:57.690+0000 7f208268a700 1 -- 172.21.15.5:0/2199806808 wait complete. 2023-09-23T04:44:57.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:57 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2199806808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T04:44:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:57 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2199806808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T04:44:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:57 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2199806808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-23T04:44:58.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738443 for osd.0 2023-09-23T04:44:58.315 DEBUG:teuthology.parallel:result is None 2023-09-23T04:44:58.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574918 for osd.1 2023-09-23T04:44:58.671 DEBUG:teuthology.parallel:result is None 2023-09-23T04:44:58.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:58 smithi005 ceph-mon[97638]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 -- 172.21.15.5:0/3422227369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898101890 msgr2=0x7f28981058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 --2- 172.21.15.5:0/3422227369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898101890 0x7f28981058e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f288c009990 tx=0x7f288c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 -- 172.21.15.5:0/3422227369 shutdown_connections 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 --2- 172.21.15.5:0/3422227369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898105fc0 0x7f289810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 --2- 172.21.15.5:0/3422227369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898100ee0 0x7f28981012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 --2- 172.21.15.5:0/3422227369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898101890 0x7f28981058e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 -- 172.21.15.5:0/3422227369 >> 172.21.15.5:0/3422227369 conn(0x7f28980fc7d0 msgr2=0x7f28980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.970+0000 7f289d799700 1 -- 172.21.15.5:0/3422227369 shutdown_connections 2023-09-23T04:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.971+0000 7f289d799700 1 -- 172.21.15.5:0/3422227369 wait complete. 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 Processor -- start 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 -- start start 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898100ee0 0x7f28981990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898101890 0x7f2898199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819dff0 con 0x7f2898105fc0 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819e160 con 0x7f2898101890 2023-09-23T04:44:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289d799700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289819e320 con 0x7f2898100ee0 2023-09-23T04:44:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289cf98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.972+0000 7f289cf98700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58036/0 (socket says 172.21.15.5:58036) 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 -- 172.21.15.5:0/1404820333 learned_addr learned my addr 172.21.15.5:0/1404820333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f2897fff700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898100ee0 0x7f28981990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f28977fe700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898101890 0x7f2898199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 -- 172.21.15.5:0/1404820333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898100ee0 msgr2=0x7f28981990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898100ee0 0x7f28981990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 -- 172.21.15.5:0/1404820333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898101890 msgr2=0x7f2898199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898101890 0x7f2898199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 -- 172.21.15.5:0/1404820333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0095f0 con 0x7f2898105fc0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289cf98700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f289000ef30 tx=0x7f289000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890018070 con 0x7f2898105fc0 2023-09-23T04:44:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289819e5a0 con 0x7f2898105fc0 2023-09-23T04:44:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289000f040 con 0x7f2898105fc0 2023-09-23T04:44:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.973+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000cd00 con 0x7f2898105fc0 2023-09-23T04:44:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.974+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28981a5e60 con 0x7f2898105fc0 2023-09-23T04:44:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.975+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289804f370 con 0x7f2898105fc0 2023-09-23T04:44:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.976+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f2890003a00 con 0x7f2898105fc0 2023-09-23T04:44:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.978+0000 7f28957fa700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 0x7f28800a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:44:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.978+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28900ccec0 con 0x7f2898105fc0 2023-09-23T04:44:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.978+0000 7f2897fff700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 0x7f28800a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:44:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.979+0000 7f2897fff700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 0x7f28800a3af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2888009c60 tx=0x7f2888009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:44:58.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:58.982+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2890096070 con 0x7f2898105fc0 2023-09-23T04:44:58.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:58 smithi069 ceph-mon[133931]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:58 smithi118 ceph-mon[134297]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T04:44:59.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449113 got 335007449115 for osd.11 2023-09-23T04:44:59.181 DEBUG:teuthology.parallel:result is None 2023-09-23T04:44:59.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.202+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2898110d50 con 0x7f2898105fc0 2023-09-23T04:44:59.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.202+0000 7f28957fa700 1 -- 172.21.15.5:0/1404820333 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2890099690 con 0x7f2898105fc0 2023-09-23T04:44:59.205 INFO:teuthology.orchestra.run.smithi005.stdout:219043332140 2023-09-23T04:44:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 msgr2=0x7f28800a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 0x7f28800a3af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2888009c60 tx=0x7f2888009260 comp rx=0 tx=0).stop 2023-09-23T04:44:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 msgr2=0x7f289819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:44:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f289000ef30 tx=0x7f289000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:44:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 shutdown_connections 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f28800a1630 0x7f28800a3af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2898105fc0 0x7f289819d9e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2898100ee0 0x7f28981990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 --2- 172.21.15.5:0/1404820333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898101890 0x7f2898199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 >> 172.21.15.5:0/1404820333 conn(0x7f28980fc7d0 msgr2=0x7f28980fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 shutdown_connections 2023-09-23T04:44:59.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.205+0000 7f289d799700 1 -- 172.21.15.5:0/1404820333 wait complete. 2023-09-23T04:44:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:44:59 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1404820333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T04:45:00.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:44:59 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1404820333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T04:45:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 -- 172.21.15.5:0/2633244955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c34101a50 msgr2=0x7f8c34105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 --2- 172.21.15.5:0/2633244955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c34101a50 0x7f8c34105aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009990 tx=0x7f8c28009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 -- 172.21.15.5:0/2633244955 shutdown_connections 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 --2- 172.21.15.5:0/2633244955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34106180 0x7f8c3410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 --2- 172.21.15.5:0/2633244955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c341010a0 0x7f8c34101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 --2- 172.21.15.5:0/2633244955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c34101a50 0x7f8c34105aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 -- 172.21.15.5:0/2633244955 >> 172.21.15.5:0/2633244955 conn(0x7f8c340fc910 msgr2=0x7f8c340fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 -- 172.21.15.5:0/2633244955 shutdown_connections 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.831+0000 7f8c3a871700 1 -- 172.21.15.5:0/2633244955 wait complete. 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.832+0000 7f8c3a871700 1 Processor -- start 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.832+0000 7f8c3a871700 1 -- start start 2023-09-23T04:45:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34101a50 0x7f8c341996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c34106180 0x7f8c3419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3419e160 con 0x7f8c341010a0 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3419e2d0 con 0x7f8c34106180 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c3a871700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3419e490 con 0x7f8c34101a50 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58048/0 (socket says 172.21.15.5:58048) 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c33fff700 1 -- 172.21.15.5:0/659580157 learned_addr learned my addr 172.21.15.5:0/659580157 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c38e0e700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c34106180 0x7f8c3419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.833+0000 7f8c337fe700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34101a50 0x7f8c341996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 -- 172.21.15.5:0/659580157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34101a50 msgr2=0x7f8c341996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34101a50 0x7f8c341996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 -- 172.21.15.5:0/659580157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c34106180 msgr2=0x7f8c3419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c34106180 0x7f8c3419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 -- 172.21.15.5:0/659580157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280095f0 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c33fff700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e9a0 tx=0x7f8c2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24009790 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3419e770 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c24010430 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.834+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24018680 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.835+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c341a5fd0 con 0x7f8c341010a0 2023-09-23T04:45:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.836+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3404eb20 con 0x7f8c341010a0 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.838+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f8c240105a0 con 0x7f8c341010a0 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.840+0000 7f8c317fa700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 0x7f8c1c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.841+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c24014070 con 0x7f8c341010a0 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.841+0000 7f8c337fe700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 0x7f8c1c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.842+0000 7f8c337fe700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 0x7f8c1c0a3af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3419a7c0 tx=0x7f8c28005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:44:59.845+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8c24051ec0 con 0x7f8c341010a0 2023-09-23T04:45:00.101 INFO:teuthology.orchestra.run.smithi005.stdout:163208757300 2023-09-23T04:45:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.099+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8c34110dc0 con 0x7f8c341010a0 2023-09-23T04:45:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.100+0000 7f8c317fa700 1 -- 172.21.15.5:0/659580157 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c240966d0 con 0x7f8c341010a0 2023-09-23T04:45:00.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.103+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 msgr2=0x7f8c1c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.103+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 0x7f8c1c0a3af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3419a7c0 tx=0x7f8c28005fd0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.103+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 msgr2=0x7f8c341991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.103+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e9a0 tx=0x7f8c2400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 shutdown_connections 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8c1c0a1630 0x7f8c1c0a3af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c34106180 0x7f8c3419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c341010a0 0x7f8c341991a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 --2- 172.21.15.5:0/659580157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8c34101a50 0x7f8c341996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 >> 172.21.15.5:0/659580157 conn(0x7f8c340fc910 msgr2=0x7f8c340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 shutdown_connections 2023-09-23T04:45:00.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.104+0000 7f8c3a871700 1 -- 172.21.15.5:0/659580157 wait complete. 2023-09-23T04:45:00.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:44:59 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1404820333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-23T04:45:00.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 -- 172.21.15.5:0/3018831417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d601010a0 msgr2=0x7f9d60101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/3018831417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d601010a0 0x7f9d60101480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009990 tx=0x7f9d5c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 -- 172.21.15.5:0/3018831417 shutdown_connections 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/3018831417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60106180 0x7f9d6010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/3018831417 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d601010a0 0x7f9d60101480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/3018831417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d60101a50 0x7f9d60105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 -- 172.21.15.5:0/3018831417 >> 172.21.15.5:0/3018831417 conn(0x7f9d600fc910 msgr2=0x7f9d600fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 -- 172.21.15.5:0/3018831417 shutdown_connections 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.122+0000 7f9d67f5b700 1 -- 172.21.15.5:0/3018831417 wait complete. 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.123+0000 7f9d67f5b700 1 Processor -- start 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.123+0000 7f9d67f5b700 1 -- start start 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d601010a0 0x7f9d6010cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d60106180 0x7f9d601b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6010d8b0 con 0x7f9d601010a0 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6006a870 con 0x7f9d60101a50 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d67f5b700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6006aa30 con 0x7f9d60106180 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d654f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d654f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56188/0 (socket says 172.21.15.5:56188) 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d654f6700 1 -- 172.21.15.5:0/2637019175 learned_addr learned my addr 172.21.15.5:0/2637019175 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.124+0000 7f9d65cf7700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d601010a0 0x7f9d6010cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d664f8700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d60106180 0x7f9d601b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d654f6700 1 -- 172.21.15.5:0/2637019175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d60106180 msgr2=0x7f9d601b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d654f6700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d60106180 0x7f9d601b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d654f6700 1 -- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d601010a0 msgr2=0x7f9d6010cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d654f6700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d601010a0 0x7f9d6010cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.125+0000 7f9d654f6700 1 -- 172.21.15.5:0/2637019175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0095f0 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d654f6700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d950 tx=0x7f9d5000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50004220 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d50007620 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50005690 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6006ad10 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.126+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6006b210 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.127+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6004eb20 con 0x7f9d60101a50 2023-09-23T04:45:00.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.130+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f9d50004750 con 0x7f9d60101a50 2023-09-23T04:45:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.131+0000 7f9d56ffd700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 0x7f9d4c0a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.131+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9d500cd160 con 0x7f9d60101a50 2023-09-23T04:45:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.132+0000 7f9d65cf7700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 0x7f9d4c0a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.133+0000 7f9d65cf7700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 0x7f9d4c0a3bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b6f0 tx=0x7f9d5c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.135+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9d50096280 con 0x7f9d60101a50 2023-09-23T04:45:00.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.401+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9d6010df10 con 0x7f9d60101a50 2023-09-23T04:45:00.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.401+0000 7f9d56ffd700 1 -- 172.21.15.5:0/2637019175 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9d50016090 con 0x7f9d60101a50 2023-09-23T04:45:00.404 INFO:teuthology.orchestra.run.smithi005.stdout:304942678046 2023-09-23T04:45:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 msgr2=0x7f9d4c0a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 0x7f9d4c0a3bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b6f0 tx=0x7f9d5c005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 msgr2=0x7f9d6010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d950 tx=0x7f9d5000dc60 comp rx=0 tx=0).stop 2023-09-23T04:45:00.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 shutdown_connections 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f9d4c0a1730 0x7f9d4c0a3bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d60106180 0x7f9d601b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d601010a0 0x7f9d6010cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 --2- 172.21.15.5:0/2637019175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d60101a50 0x7f9d6010d110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 >> 172.21.15.5:0/2637019175 conn(0x7f9d600fc910 msgr2=0x7f9d60102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 shutdown_connections 2023-09-23T04:45:00.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.404+0000 7f9d67f5b700 1 -- 172.21.15.5:0/2637019175 wait complete. 2023-09-23T04:45:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.482+0000 7f2631fd3700 1 -- 172.21.15.5:0/2941726536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c068940 msgr2=0x7f262c068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.482+0000 7f2631fd3700 1 --2- 172.21.15.5:0/2941726536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c068940 0x7f262c068dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2620009990 tx=0x7f2620009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 -- 172.21.15.5:0/2941726536 shutdown_connections 2023-09-23T04:45:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 --2- 172.21.15.5:0/2941726536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c069300 0x7f262c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 --2- 172.21.15.5:0/2941726536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c106fb0 0x7f262c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 --2- 172.21.15.5:0/2941726536 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c068940 0x7f262c068dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 -- 172.21.15.5:0/2941726536 >> 172.21.15.5:0/2941726536 conn(0x7f262c076e20 msgr2=0x7f262c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 -- 172.21.15.5:0/2941726536 shutdown_connections 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.483+0000 7f2631fd3700 1 -- 172.21.15.5:0/2941726536 wait complete. 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.484+0000 7f2631fd3700 1 Processor -- start 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.484+0000 7f2631fd3700 1 -- start start 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c069300 0x7f262c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c106fb0 0x7f262c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c19e140 con 0x7f262c069300 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c19e2b0 con 0x7f262c068940 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f2631fd3700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c19e470 con 0x7f262c106fb0 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.485+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56220/0 (socket says 172.21.15.5:56220) 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 -- 172.21.15.5:0/3816385636 learned_addr learned my addr 172.21.15.5:0/3816385636 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262affd700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c069300 0x7f262c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 -- 172.21.15.5:0/3816385636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c106fb0 msgr2=0x7f262c19db30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c106fb0 0x7f262c19db30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 -- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c069300 msgr2=0x7f262c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c069300 0x7f262c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 -- 172.21.15.5:0/3816385636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200095f0 con 0x7f262c068940 2023-09-23T04:45:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.486+0000 7f262b7fe700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f261c00e9a0 tx=0x7f261c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.487+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c009790 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.487+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c19e750 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.487+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f261c00cbd0 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.487+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1a5fb0 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.487+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c018680 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.489+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618005320 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.490+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f261c010430 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.492+0000 7f2628ff9700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 0x7f26140a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.492+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f261c014070 con 0x7f262c068940 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.492+0000 7f262affd700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 0x7f26140a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.493+0000 7f262affd700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 0x7f26140a3b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f262c19a7a0 tx=0x7f262000b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.496+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f261c09a150 con 0x7f262c068940 2023-09-23T04:45:00.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 -- 172.21.15.5:0/3266726334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8106fb0 msgr2=0x7f83b8107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3266726334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8106fb0 0x7f83b8107390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009990 tx=0x7f83a8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 -- 172.21.15.5:0/3266726334 shutdown_connections 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3266726334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8069300 0x7f83b810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3266726334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8106fb0 0x7f83b8107390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3266726334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8068940 0x7f83b8068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.555+0000 7f83be4c9700 1 -- 172.21.15.5:0/3266726334 >> 172.21.15.5:0/3266726334 conn(0x7f83b8076e20 msgr2=0x7f83b8077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.556+0000 7f83be4c9700 1 -- 172.21.15.5:0/3266726334 shutdown_connections 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.556+0000 7f83be4c9700 1 -- 172.21.15.5:0/3266726334 wait complete. 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.557+0000 7f83be4c9700 1 Processor -- start 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 -- start start 2023-09-23T04:45:00.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8068940 0x7f83b8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8069300 0x7f83b8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b819e0f0 con 0x7f83b8069300 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b819e260 con 0x7f83b8106fb0 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.558+0000 7f83be4c9700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b819e420 con 0x7f83b8068940 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.559+0000 7f83bca66700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.559+0000 7f83bca66700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56242/0 (socket says 172.21.15.5:56242) 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.559+0000 7f83bca66700 1 -- 172.21.15.5:0/3465736000 learned_addr learned my addr 172.21.15.5:0/3465736000 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.559+0000 7f83b7fff700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8068940 0x7f83b8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83b77fe700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8069300 0x7f83b8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 -- 172.21.15.5:0/3465736000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8068940 msgr2=0x7f83b8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8068940 0x7f83b8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 -- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8069300 msgr2=0x7f83b8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8069300 0x7f83b8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 -- 172.21.15.5:0/3465736000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80095f0 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.560+0000 7f83bca66700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83b0007f10 tx=0x7f83b000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.561+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0017070 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.561+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b000f040 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.561+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0005740 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.561+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b819e700 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.561+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b81a5f60 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.562+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b804eb20 con 0x7f83b8106fb0 2023-09-23T04:45:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.567+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f83b000dc60 con 0x7f83b8106fb0 2023-09-23T04:45:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.570+0000 7f83b57fa700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 0x7f83a00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.570+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83b00cc8a0 con 0x7f83b8106fb0 2023-09-23T04:45:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.570+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f83b00f8f60 con 0x7f83b8106fb0 2023-09-23T04:45:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.571+0000 7f83b7fff700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 0x7f83a00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.571+0000 7f83b7fff700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 0x7f83a00a3b70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f83a8003800 tx=0x7f83a800b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 -- 172.21.15.5:0/3934683880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64240ffd60 msgr2=0x7f64241001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 --2- 172.21.15.5:0/3934683880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64240ffd60 0x7f64241001e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f64180098f0 tx=0x7f6418009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 -- 172.21.15.5:0/3934683880 shutdown_connections 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 --2- 172.21.15.5:0/3934683880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 0x7f642410cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 --2- 172.21.15.5:0/3934683880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6424105e10 0x7f64241061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 --2- 172.21.15.5:0/3934683880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64240ffd60 0x7f64241001e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 -- 172.21.15.5:0/3934683880 >> 172.21.15.5:0/3934683880 conn(0x7f6424076e20 msgr2=0x7f6424077230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 -- 172.21.15.5:0/3934683880 shutdown_connections 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.395+0000 7f642a908700 1 -- 172.21.15.5:0/3934683880 wait complete. 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.397+0000 7f642a908700 1 Processor -- start 2023-09-23T04:45:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.397+0000 7f642a908700 1 -- start start 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 0x7f64241071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 0x7f6424107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a5400 con 0x7f6424105e10 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a5570 con 0x7f64240ffd60 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f642a908700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64241a5730 con 0x7f6424100720 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f6423fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f6423fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56196/0 (socket says 172.21.15.5:56196) 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f6423fff700 1 -- 172.21.15.5:0/1751787437 learned_addr learned my addr 172.21.15.5:0/1751787437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f64237fe700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 0x7f64241071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.398+0000 7f6428ea5700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 0x7f6424107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 -- 172.21.15.5:0/1751787437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 msgr2=0x7f64241071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 0x7f64241071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 -- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 msgr2=0x7f6424107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 0x7f6424107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 -- 172.21.15.5:0/1751787437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009520 con 0x7f64240ffd60 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6428ea5700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 0x7f6424107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f6423fff700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.399+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400cbd0 con 0x7f64240ffd60 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.400+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64241a5a10 con 0x7f64240ffd60 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.400+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f641400eeb0 con 0x7f64240ffd60 2023-09-23T04:45:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.400+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414005580 con 0x7f64240ffd60 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.400+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64241a5f30 con 0x7f64240ffd60 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.401+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642404eb20 con 0x7f64240ffd60 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.405+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f6414004750 con 0x7f64240ffd60 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.407+0000 7f64217fa700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 0x7f640c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.407+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6414014070 con 0x7f64240ffd60 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.407+0000 7f64237fe700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 0x7f640c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.408+0000 7f64237fe700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 0x7f640c0a3b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6418005e90 tx=0x7f6418019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.410+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f64140967f0 con 0x7f64240ffd60 2023-09-23T04:45:00.749 INFO:teuthology.orchestra.run.smithi005.stdout:249108103207 2023-09-23T04:45:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.692+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f642410fef0 con 0x7f64240ffd60 2023-09-23T04:45:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.693+0000 7f64217fa700 1 -- 172.21.15.5:0/1751787437 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6414099e10 con 0x7f64240ffd60 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 msgr2=0x7f640c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 0x7f640c0a3b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6418005e90 tx=0x7f6418019040 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 msgr2=0x7f6424106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f641400e9c0 tx=0x7f641400ecd0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 shutdown_connections 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f640c0a16b0 0x7f640c0a3b70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424105e10 0x7f6424107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64240ffd60 0x7f6424106c60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 --2- 172.21.15.5:0/1751787437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6424100720 0x7f64241071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 >> 172.21.15.5:0/1751787437 conn(0x7f6424076e20 msgr2=0x7f64240fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.695+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 shutdown_connections 2023-09-23T04:45:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.696+0000 7f642a908700 1 -- 172.21.15.5:0/1751787437 wait complete. 2023-09-23T04:45:00.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.857+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2618005190 con 0x7f262c068940 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.857+0000 7f2628ff9700 1 -- 172.21.15.5:0/3816385636 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f261c099ce0 con 0x7f262c068940 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 msgr2=0x7f26140a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 0x7f26140a3b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f262c19a7a0 tx=0x7f262000b280 comp rx=0 tx=0).stop 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 msgr2=0x7f262c199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f261c00e9a0 tx=0x7f261c00ed60 comp rx=0 tx=0).stop 2023-09-23T04:45:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 shutdown_connections 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f26140a16b0 0x7f26140a3b70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f262c106fb0 0x7f262c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f262c068940 0x7f262c199180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 --2- 172.21.15.5:0/3816385636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f262c069300 0x7f262c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 >> 172.21.15.5:0/3816385636 conn(0x7f262c076e20 msgr2=0x7f262c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 shutdown_connections 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.860+0000 7f2631fd3700 1 -- 172.21.15.5:0/3816385636 wait complete. 2023-09-23T04:45:00.876 INFO:teuthology.orchestra.run.smithi005.stdout:283467841571 2023-09-23T04:45:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.917+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f83b8110e90 con 0x7f83b8106fb0 2023-09-23T04:45:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.917+0000 7f83b57fa700 1 -- 172.21.15.5:0/3465736000 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83b00959c0 con 0x7f83b8106fb0 2023-09-23T04:45:00.918 INFO:teuthology.orchestra.run.smithi005.stdout:107374182462 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 msgr2=0x7f83a00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 0x7f83a00a3b70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f83a8003800 tx=0x7f83a800b580 comp rx=0 tx=0).stop 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 msgr2=0x7f83b819dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83b0007f10 tx=0x7f83b000d3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 shutdown_connections 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83a00a16b0 0x7f83a00a3b70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83b8106fb0 0x7f83b819dae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83b8068940 0x7f83b8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 --2- 172.21.15.5:0/3465736000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8069300 0x7f83b8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 >> 172.21.15.5:0/3465736000 conn(0x7f83b8076e20 msgr2=0x7f83b810bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 shutdown_connections 2023-09-23T04:45:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:00.920+0000 7f83be4c9700 1 -- 172.21.15.5:0/3465736000 wait complete. 2023-09-23T04:45:00.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:00 smithi005 ceph-mon[97638]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:45:00.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/659580157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T04:45:00.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2637019175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T04:45:00.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1751787437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T04:45:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:00 smithi069 ceph-mon[133931]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:45:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/659580157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T04:45:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2637019175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T04:45:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1751787437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T04:45:01.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:00 smithi118 ceph-mon[134297]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2023-09-23T04:45:01.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/659580157' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-23T04:45:01.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2637019175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-23T04:45:01.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1751787437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-23T04:45:01.366 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:01.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332140 for osd.7 2023-09-23T04:45:01.511 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 -- 172.21.15.5:0/153157664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 msgr2=0x7f2d74101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 --2- 172.21.15.5:0/153157664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74101480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009990 tx=0x7f2d68009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 -- 172.21.15.5:0/153157664 shutdown_connections 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 --2- 172.21.15.5:0/153157664 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74106180 0x7f2d7410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 --2- 172.21.15.5:0/153157664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74101480 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 --2- 172.21.15.5:0/153157664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74101a50 0x7f2d74105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 -- 172.21.15.5:0/153157664 >> 172.21.15.5:0/153157664 conn(0x7f2d740fc910 msgr2=0x7f2d740fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 -- 172.21.15.5:0/153157664 shutdown_connections 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.731+0000 7f2d7b188700 1 -- 172.21.15.5:0/153157664 wait complete. 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.732+0000 7f2d7b188700 1 Processor -- start 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 -- start start 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74101a50 0x7f2d74073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 0x7f2d74073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741a5450 con 0x7f2d74106180 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741a55c0 con 0x7f2d741010a0 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d7b188700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741a5780 con 0x7f2d74101a50 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d78f24700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.733+0000 7f2d79725700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 0x7f2d74073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39036/0 (socket says 172.21.15.5:39036) 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 -- 172.21.15.5:0/1014142218 learned_addr learned my addr 172.21.15.5:0/1014142218 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d73fff700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74101a50 0x7f2d74073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 -- 172.21.15.5:0/1014142218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74101a50 msgr2=0x7f2d74073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74101a50 0x7f2d74073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 -- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 msgr2=0x7f2d74073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 0x7f2d74073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 -- 172.21.15.5:0/1014142218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680095f0 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d79725700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 0x7f2d74073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d78f24700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d68005910 tx=0x7f2d68004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.734+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6801d070 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.735+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d741a5a00 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.735+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d68022470 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.735+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68017650 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.735+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d741a5e20 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.736+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7404eb20 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.738+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f2d680225e0 con 0x7f2d741010a0 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.739+0000 7f2d71ffb700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 0x7f2d640a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.739+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d680ce620 con 0x7f2d741010a0 2023-09-23T04:45:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.739+0000 7f2d73fff700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 0x7f2d640a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.740+0000 7f2d73fff700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 0x7f2d640a3af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d74074910 tx=0x7f2d60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.741+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2d680977d0 con 0x7f2d741010a0 2023-09-23T04:45:01.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.965+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2d74110fd0 con 0x7f2d741010a0 2023-09-23T04:45:01.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.966+0000 7f2d71ffb700 1 -- 172.21.15.5:0/1014142218 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d6809adf0 con 0x7f2d741010a0 2023-09-23T04:45:01.968 INFO:teuthology.orchestra.run.smithi005.stdout:77309411396 2023-09-23T04:45:01.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 msgr2=0x7f2d640a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:01.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 0x7f2d640a3af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d74074910 tx=0x7f2d60009260 comp rx=0 tx=0).stop 2023-09-23T04:45:01.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 msgr2=0x7f2d74073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:01.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d68005910 tx=0x7f2d68004710 comp rx=0 tx=0).stop 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 shutdown_connections 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d640a1630 0x7f2d640a3af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74106180 0x7f2d74073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d741010a0 0x7f2d74073120 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 --2- 172.21.15.5:0/1014142218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d74101a50 0x7f2d74073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 >> 172.21.15.5:0/1014142218 conn(0x7f2d740fc910 msgr2=0x7f2d74104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 shutdown_connections 2023-09-23T04:45:01.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:01.969+0000 7f2d7b188700 1 -- 172.21.15.5:0/1014142218 wait complete. 2023-09-23T04:45:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:01 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3816385636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T04:45:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:01 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3465736000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T04:45:01.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:01 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3816385636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T04:45:01.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:01 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3465736000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T04:45:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:01 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3816385636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-23T04:45:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:01 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3465736000' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-23T04:45:02.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678044 got 304942678046 for osd.10 2023-09-23T04:45:02.215 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:02.286 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841569 got 283467841571 for osd.9 2023-09-23T04:45:02.286 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:02.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757297 got 163208757300 for osd.5 2023-09-23T04:45:02.624 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:02.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:02 smithi005 ceph-mon[97638]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-23T04:45:02.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:02 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1014142218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T04:45:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:02 smithi069 ceph-mon[133931]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-23T04:45:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:02 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1014142218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T04:45:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 -- 172.21.15.5:0/1517149215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd000111e50 msgr2=0x7fd000114240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 --2- 172.21.15.5:0/1517149215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd000111e50 0x7fd000114240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b0a0 tx=0x7fcff800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 -- 172.21.15.5:0/1517149215 shutdown_connections 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 --2- 172.21.15.5:0/1517149215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd000111e50 0x7fd000114240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 --2- 172.21.15.5:0/1517149215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd0000731c0 0x7fd0000735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 --2- 172.21.15.5:0/1517149215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000073ae0 0x7fd000111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 -- 172.21.15.5:0/1517149215 >> 172.21.15.5:0/1517149215 conn(0x7fd0000fc910 msgr2=0x7fd0000fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.057+0000 7fd005aee700 1 -- 172.21.15.5:0/1517149215 shutdown_connections 2023-09-23T04:45:03.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.058+0000 7fd005aee700 1 -- 172.21.15.5:0/1517149215 wait complete. 2023-09-23T04:45:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 Processor -- start 2023-09-23T04:45:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 -- start start 2023-09-23T04:45:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd000073ae0 0x7fd00019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000111e50 0x7fd0001a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0001a15b0 con 0x7fd0000731c0 2023-09-23T04:45:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0001a1720 con 0x7fd000073ae0 2023-09-23T04:45:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.059+0000 7fd005aee700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0001a18e0 con 0x7fd000111e50 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41428/0 (socket says 172.21.15.5:41428) 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 -- 172.21.15.5:0/1802903213 learned_addr learned my addr 172.21.15.5:0/1802903213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcffeffd700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd000073ae0 0x7fd00019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcffffff700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000111e50 0x7fd0001a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 -- 172.21.15.5:0/1802903213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000111e50 msgr2=0x7fd0001a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000111e50 0x7fd0001a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 -- 172.21.15.5:0/1802903213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd000073ae0 msgr2=0x7fd00019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd000073ae0 0x7fd00019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.060+0000 7fcfff7fe700 1 -- 172.21.15.5:0/1802903213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0009520 con 0x7fd0000731c0 2023-09-23T04:45:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fcfff7fe700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcff000eb30 tx=0x7fcff000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0009880 con 0x7fd0000731c0 2023-09-23T04:45:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcff00044e0 con 0x7fd0000731c0 2023-09-23T04:45:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff8009d00 con 0x7fd0000731c0 2023-09-23T04:45:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0003d80 con 0x7fd0000731c0 2023-09-23T04:45:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.061+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0001aa340 con 0x7fd0000731c0 2023-09-23T04:45:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.062+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd000104330 con 0x7fd0000731c0 2023-09-23T04:45:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.064+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fcff0003f20 con 0x7fd0000731c0 2023-09-23T04:45:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.065+0000 7fcffcff9700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 0x7fcfe80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.066+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcff0014070 con 0x7fd0000731c0 2023-09-23T04:45:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.066+0000 7fcffeffd700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 0x7fcfe80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.066+0000 7fcffeffd700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 0x7fcfe80a3b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd00019ecd0 tx=0x7fcff4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.069+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fcff0096a10 con 0x7fd0000731c0 2023-09-23T04:45:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:02 smithi118 ceph-mon[134297]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-23T04:45:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:02 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1014142218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-23T04:45:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.286+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd0001173d0 con 0x7fd0000731c0 2023-09-23T04:45:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.286+0000 7fcffcff9700 1 -- 172.21.15.5:0/1802903213 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcff009a030 con 0x7fd0000731c0 2023-09-23T04:45:03.288 INFO:teuthology.orchestra.run.smithi005.stdout:197568495665 2023-09-23T04:45:03.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 msgr2=0x7fcfe80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:03.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 0x7fcfe80a3b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd00019ecd0 tx=0x7fcff4009260 comp rx=0 tx=0).stop 2023-09-23T04:45:03.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 msgr2=0x7fd00019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:03.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcff000eb30 tx=0x7fcff000ee40 comp rx=0 tx=0).stop 2023-09-23T04:45:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 shutdown_connections 2023-09-23T04:45:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcfe80a16b0 0x7fcfe80a3b70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd000111e50 0x7fd0001a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0000731c0 0x7fd00019d6b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 --2- 172.21.15.5:0/1802903213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd000073ae0 0x7fd00019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 >> 172.21.15.5:0/1802903213 conn(0x7fd0000fc910 msgr2=0x7fd0000fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 shutdown_connections 2023-09-23T04:45:03.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:03.290+0000 7fd005aee700 1 -- 172.21.15.5:0/1802903213 wait complete. 2023-09-23T04:45:03.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411396 for osd.2 2023-09-23T04:45:03.612 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:03.824 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:03 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1802903213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T04:45:03.826 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103203 got 249108103207 for osd.8 2023-09-23T04:45:03.826 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:03.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182462 for osd.3 2023-09-23T04:45:03.932 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:03 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1802903213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T04:45:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:03 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1802903213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-23T04:45:04.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495665 for osd.6 2023-09-23T04:45:04.541 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:04.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:04 smithi005 ceph-mon[97638]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-09-23T04:45:04.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:04.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:04 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:04 smithi069 ceph-mon[133931]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-09-23T04:45:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:04 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 -- 172.21.15.5:0/1456958293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c069170 msgr2=0x7f830c0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 --2- 172.21.15.5:0/1456958293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c069170 0x7f830c0695f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009990 tx=0x7f82fc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 -- 172.21.15.5:0/1456958293 shutdown_connections 2023-09-23T04:45:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 --2- 172.21.15.5:0/1456958293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 --2- 172.21.15.5:0/1456958293 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c105d20 0x7f830c106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 --2- 172.21.15.5:0/1456958293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c069170 0x7f830c0695f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 -- 172.21.15.5:0/1456958293 >> 172.21.15.5:0/1456958293 conn(0x7f830c075880 msgr2=0x7f830c075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 -- 172.21.15.5:0/1456958293 shutdown_connections 2023-09-23T04:45:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.016+0000 7f8314223700 1 -- 172.21.15.5:0/1456958293 wait complete. 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.017+0000 7f8314223700 1 Processor -- start 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.017+0000 7f8314223700 1 -- start start 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c069170 0x7f830c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 0x7f830c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a2460 con 0x7f830c105d20 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a25d0 con 0x7f830c069c10 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8314223700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1a2790 con 0x7f830c069170 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f83127c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 0x7f830c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f83117be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f8311fbf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c069170 0x7f830c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f83127c0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 0x7f830c1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41434/0 (socket says 172.21.15.5:41434) 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f83117be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39074/0 (socket says 172.21.15.5:39074) 2023-09-23T04:45:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.018+0000 7f83127c0700 1 -- 172.21.15.5:0/3450024240 learned_addr learned my addr 172.21.15.5:0/3450024240 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 -- 172.21.15.5:0/3450024240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c069170 msgr2=0x7f830c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c069170 0x7f830c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 -- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 msgr2=0x7f830c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 0x7f830c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 -- 172.21.15.5:0/3450024240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc0095f0 con 0x7f830c069c10 2023-09-23T04:45:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f83117be700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82fc000c00 tx=0x7f82fc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01d070 con 0x7f830c069c10 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc021410 con 0x7f830c069c10 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc017650 con 0x7f830c069c10 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.019+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c1a2a10 con 0x7f830c069c10 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.020+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c1aa100 con 0x7f830c069c10 2023-09-23T04:45:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.021+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c04eb20 con 0x7f830c069c10 2023-09-23T04:45:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.022+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f82fc00fad0 con 0x7f830c069c10 2023-09-23T04:45:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.024+0000 7f8302ffd700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 0x7f82f80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.024+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82fc0ce750 con 0x7f830c069c10 2023-09-23T04:45:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.024+0000 7f8311fbf700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 0x7f82f80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.025+0000 7f8311fbf700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 0x7f82f80a3af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8308009be0 tx=0x7f8308009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.026+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f82fc097900 con 0x7f830c069c10 2023-09-23T04:45:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:04 smithi118 ceph-mon[134297]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2023-09-23T04:45:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:04 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.242+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f830c117480 con 0x7f830c069c10 2023-09-23T04:45:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.243+0000 7f8302ffd700 1 -- 172.21.15.5:0/3450024240 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82fc09af20 con 0x7f830c069c10 2023-09-23T04:45:05.244 INFO:teuthology.orchestra.run.smithi005.stdout:141733920826 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 msgr2=0x7f82f80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 0x7f82f80a3af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8308009be0 tx=0x7f8308009260 comp rx=0 tx=0).stop 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 msgr2=0x7f830c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82fc000c00 tx=0x7f82fc005740 comp rx=0 tx=0).stop 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 shutdown_connections 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f82f80a1630 0x7f82f80a3af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f830c105d20 0x7f830c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f830c069170 0x7f830c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 --2- 172.21.15.5:0/3450024240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f830c069c10 0x7f830c19d9e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 >> 172.21.15.5:0/3450024240 conn(0x7f830c075880 msgr2=0x7f830c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 shutdown_connections 2023-09-23T04:45:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:05.246+0000 7f8314223700 1 -- 172.21.15.5:0/3450024240 wait complete. 2023-09-23T04:45:05.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920826 for osd.4 2023-09-23T04:45:05.751 DEBUG:teuthology.parallel:result is None 2023-09-23T04:45:05.751 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-23T04:45:05.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph pg dump --format=json 2023-09-23T04:45:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:45:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:45:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3450024240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T04:45:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:45:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:45:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3450024240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T04:45:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:45:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:45:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' 2023-09-23T04:45:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3450024240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-23T04:45:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:06 smithi069 ceph-mon[133931]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-23T04:45:07.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:06 smithi005 ceph-mon[97638]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-23T04:45:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:06 smithi118 ceph-mon[134297]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-23T04:45:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.915+0000 7f32b342e700 1 -- 172.21.15.5:0/1488488658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac106180 msgr2=0x7f32ac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.915+0000 7f32b342e700 1 --2- 172.21.15.5:0/1488488658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac106180 0x7f32ac10dd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b0a0 tx=0x7f32a400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 -- 172.21.15.5:0/1488488658 shutdown_connections 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 --2- 172.21.15.5:0/1488488658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac106180 0x7f32ac10dd50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 --2- 172.21.15.5:0/1488488658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac1010a0 0x7f32ac101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 --2- 172.21.15.5:0/1488488658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 -- 172.21.15.5:0/1488488658 >> 172.21.15.5:0/1488488658 conn(0x7f32ac0fc910 msgr2=0x7f32ac0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 -- 172.21.15.5:0/1488488658 shutdown_connections 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.916+0000 7f32b342e700 1 -- 172.21.15.5:0/1488488658 wait complete. 2023-09-23T04:45:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 Processor -- start 2023-09-23T04:45:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 -- start start 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 0x7f32ac197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 0x7f32ac19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac19c070 con 0x7f32ac1010a0 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac19c1e0 con 0x7f32ac106180 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.917+0000 7f32b342e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac19c3a0 con 0x7f32ac101a50 2023-09-23T04:45:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b11ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 0x7f32ac197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46366/0 (socket says 172.21.15.5:46366) 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 -- 172.21.15.5:0/1026734898 learned_addr learned my addr 172.21.15.5:0/1026734898 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b19cb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 0x7f32ac19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 -- 172.21.15.5:0/1026734898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 msgr2=0x7f32ac19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 0x7f32ac19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 -- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 msgr2=0x7f32ac197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 0x7f32ac197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b09c9700 1 -- 172.21.15.5:0/1026734898 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009d00 con 0x7f32ac101a50 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b19cb700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 0x7f32ac19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.918+0000 7f32b11ca700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 0x7f32ac197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32b09c9700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d980 tx=0x7f329c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00f890 con 0x7f32ac101a50 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac19c680 con 0x7f32ac101a50 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329c010460 con 0x7f32ac101a50 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00e610 con 0x7f32ac101a50 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.919+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1a3ee0 con 0x7f32ac101a50 2023-09-23T04:45:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.920+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac04eb20 con 0x7f32ac101a50 2023-09-23T04:45:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.922+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f329c0105d0 con 0x7f32ac101a50 2023-09-23T04:45:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.923+0000 7f32a27fc700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 0x7f32980a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.923+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f329c0ccd70 con 0x7f32ac101a50 2023-09-23T04:45:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.924+0000 7f32b11ca700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 0x7f32980a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.924+0000 7f32b11ca700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 0x7f32980a3af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32a8007fd0 tx=0x7f32a80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:07.928+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f329c095e90 con 0x7f32ac101a50 2023-09-23T04:45:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.131+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32ac1110d0 con 0x7f32980a1630 2023-09-23T04:45:08.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.145+0000 7f32a27fc700 1 -- 172.21.15.5:0/1026734898 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427849 (secure 0 0 0) 0x7f32ac1110d0 con 0x7f32980a1630 2023-09-23T04:45:08.154 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:08.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 msgr2=0x7f32980a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:08.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 0x7f32980a3af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f32a8007fd0 tx=0x7f32a80073d0 comp rx=0 tx=0).stop 2023-09-23T04:45:08.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 msgr2=0x7f32ac197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d980 tx=0x7f329c00dc90 comp rx=0 tx=0).stop 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 shutdown_connections 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f32980a1630 0x7f32980a3af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32ac106180 0x7f32ac19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac1010a0 0x7f32ac197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 --2- 172.21.15.5:0/1026734898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f32ac101a50 0x7f32ac197640 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 >> 172.21.15.5:0/1026734898 conn(0x7f32ac0fc910 msgr2=0x7f32ac0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 shutdown_connections 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:08.151+0000 7f32b342e700 1 -- 172.21.15.5:0/1026734898 wait complete. 2023-09-23T04:45:08.158 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:45:08.582 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2023-09-23T04:45:07.650297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658257,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":608,"num_read_kb":427,"num_write":421,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181458,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336248,"kb_used_data":11996,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605576,"statfs":{"total":1151940427776,"available":1151596109824,"internally_reserved":0,"allocated":12283904,"data_stored":7803604,"data_compressed":321123,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003086"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808506+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:44:23.808506+0000","last_peered":"2023-09-23T04:44:23.808506+0000","last_clean":"2023-09-23T04:44:23.808506+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:44:23.808506+0000","last_undegraded":"2023-09-23T04:44:23.808506+0000","last_fullsized":"2023-09-23T04:44:23.808506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805954+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:44:23.805954+0000","last_peered":"2023-09-23T04:44:23.805954+0000","last_clean":"2023-09-23T04:44:23.805954+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:44:23.805954+0000","last_undegraded":"2023-09-23T04:44:23.805954+0000","last_fullsized":"2023-09-23T04:44:23.805954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806021+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:44:23.806021+0000","last_peered":"2023-09-23T04:44:23.806021+0000","last_clean":"2023-09-23T04:44:23.806021+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:44:23.806021+0000","last_undegraded":"2023-09-23T04:44:23.806021+0000","last_fullsized":"2023-09-23T04:44:23.806021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803120+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:44:23.803120+0000","last_peered":"2023-09-23T04:44:23.803120+0000","last_clean":"2023-09-23T04:44:23.803120+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:44:23.803120+0000","last_undegraded":"2023-09-23T04:44:23.803120+0000","last_fullsized":"2023-09-23T04:44:23.803120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803525+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:44:23.803525+0000","last_peered":"2023-09-23T04:44:23.803525+0000","last_clean":"2023-09-23T04:44:23.803525+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:44:23.803525+0000","last_undegraded":"2023-09-23T04:44:23.803525+0000","last_fullsized":"2023-09-23T04:44:23.803525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807910+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:44:23.807910+0000","last_peered":"2023-09-23T04:44:23.807910+0000","last_clean":"2023-09-23T04:44:23.807910+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:44:23.807910+0000","last_undegraded":"2023-09-23T04:44:23.807910+0000","last_fullsized":"2023-09-23T04:44:23.807910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806203+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:44:23.806203+0000","last_peered":"2023-09-23T04:44:23.806203+0000","last_clean":"2023-09-23T04:44:23.806203+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:44:23.806203+0000","last_undegraded":"2023-09-23T04:44:23.806203+0000","last_fullsized":"2023-09-23T04:44:23.806203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806391+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:44:23.806391+0000","last_peered":"2023-09-23T04:44:23.806391+0000","last_clean":"2023-09-23T04:44:23.806391+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:44:23.806391+0000","last_undegraded":"2023-09-23T04:44:23.806391+0000","last_fullsized":"2023-09-23T04:44:23.806391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810224+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:44:23.810224+0000","last_peered":"2023-09-23T04:44:23.810224+0000","last_clean":"2023-09-23T04:44:23.810224+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:44:23.810224+0000","last_undegraded":"2023-09-23T04:44:23.810224+0000","last_fullsized":"2023-09-23T04:44:23.810224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808475+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:44:23.808475+0000","last_peered":"2023-09-23T04:44:23.808475+0000","last_clean":"2023-09-23T04:44:23.808475+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:44:23.808475+0000","last_undegraded":"2023-09-23T04:44:23.808475+0000","last_fullsized":"2023-09-23T04:44:23.808475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807762+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:44:23.807762+0000","last_peered":"2023-09-23T04:44:23.807762+0000","last_clean":"2023-09-23T04:44:23.807762+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:44:23.807762+0000","last_undegraded":"2023-09-23T04:44:23.807762+0000","last_fullsized":"2023-09-23T04:44:23.807762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803160+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:44:23.803160+0000","last_peered":"2023-09-23T04:44:23.803160+0000","last_clean":"2023-09-23T04:44:23.803160+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:44:23.803160+0000","last_undegraded":"2023-09-23T04:44:23.803160+0000","last_fullsized":"2023-09-23T04:44:23.803160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811269+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:44:23.811269+0000","last_peered":"2023-09-23T04:44:23.811269+0000","last_clean":"2023-09-23T04:44:23.811269+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:44:23.811269+0000","last_undegraded":"2023-09-23T04:44:23.811269+0000","last_fullsized":"2023-09-23T04:44:23.811269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805584+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:44:23.805584+0000","last_peered":"2023-09-23T04:44:23.805584+0000","last_clean":"2023-09-23T04:44:23.805584+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:44:23.805584+0000","last_undegraded":"2023-09-23T04:44:23.805584+0000","last_fullsized":"2023-09-23T04:44:23.805584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807939+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:44:23.807939+0000","last_peered":"2023-09-23T04:44:23.807939+0000","last_clean":"2023-09-23T04:44:23.807939+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:44:23.807939+0000","last_undegraded":"2023-09-23T04:44:23.807939+0000","last_fullsized":"2023-09-23T04:44:23.807939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807423+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:44:23.807423+0000","last_peered":"2023-09-23T04:44:23.807423+0000","last_clean":"2023-09-23T04:44:23.807423+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:44:23.807423+0000","last_undegraded":"2023-09-23T04:44:23.807423+0000","last_fullsized":"2023-09-23T04:44:23.807423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808415+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:44:23.808415+0000","last_peered":"2023-09-23T04:44:23.808415+0000","last_clean":"2023-09-23T04:44:23.808415+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:44:23.808415+0000","last_undegraded":"2023-09-23T04:44:23.808415+0000","last_fullsized":"2023-09-23T04:44:23.808415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813088+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:44:23.813088+0000","last_peered":"2023-09-23T04:44:23.813088+0000","last_clean":"2023-09-23T04:44:23.813088+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:44:23.813088+0000","last_undegraded":"2023-09-23T04:44:23.813088+0000","last_fullsized":"2023-09-23T04:44:23.813088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804381+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:44:23.804381+0000","last_peered":"2023-09-23T04:44:23.804381+0000","last_clean":"2023-09-23T04:44:23.804381+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:44:23.804381+0000","last_undegraded":"2023-09-23T04:44:23.804381+0000","last_fullsized":"2023-09-23T04:44:23.804381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804863+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:44:23.804863+0000","last_peered":"2023-09-23T04:44:23.804863+0000","last_clean":"2023-09-23T04:44:23.804863+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:44:23.804863+0000","last_undegraded":"2023-09-23T04:44:23.804863+0000","last_fullsized":"2023-09-23T04:44:23.804863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:01.881038+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:45:01.881038+0000","last_peered":"2023-09-23T04:45:01.881038+0000","last_clean":"2023-09-23T04:45:01.881038+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:45:01.881038+0000","last_undegraded":"2023-09-23T04:45:01.881038+0000","last_fullsized":"2023-09-23T04:45:01.881038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807581+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:44:23.807581+0000","last_peered":"2023-09-23T04:44:23.807581+0000","last_clean":"2023-09-23T04:44:23.807581+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:44:23.807581+0000","last_undegraded":"2023-09-23T04:44:23.807581+0000","last_fullsized":"2023-09-23T04:44:23.807581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802552+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:44:23.802552+0000","last_peered":"2023-09-23T04:44:23.802552+0000","last_clean":"2023-09-23T04:44:23.802552+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:44:23.802552+0000","last_undegraded":"2023-09-23T04:44:23.802552+0000","last_fullsized":"2023-09-23T04:44:23.802552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807732+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:44:23.807732+0000","last_peered":"2023-09-23T04:44:23.807732+0000","last_clean":"2023-09-23T04:44:23.807732+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:44:23.807732+0000","last_undegraded":"2023-09-23T04:44:23.807732+0000","last_fullsized":"2023-09-23T04:44:23.807732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807678+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:44:23.807678+0000","last_peered":"2023-09-23T04:44:23.807678+0000","last_clean":"2023-09-23T04:44:23.807678+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:44:23.807678+0000","last_undegraded":"2023-09-23T04:44:23.807678+0000","last_fullsized":"2023-09-23T04:44:23.807678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802751+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:44:23.802751+0000","last_peered":"2023-09-23T04:44:23.802751+0000","last_clean":"2023-09-23T04:44:23.802751+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:44:23.802751+0000","last_undegraded":"2023-09-23T04:44:23.802751+0000","last_fullsized":"2023-09-23T04:44:23.802751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.805382+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:44:24.805382+0000","last_peered":"2023-09-23T04:44:24.805382+0000","last_clean":"2023-09-23T04:44:24.805382+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:44:24.805382+0000","last_undegraded":"2023-09-23T04:44:24.805382+0000","last_fullsized":"2023-09-23T04:44:24.805382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809275+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:44:23.809275+0000","last_peered":"2023-09-23T04:44:23.809275+0000","last_clean":"2023-09-23T04:44:23.809275+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:44:23.809275+0000","last_undegraded":"2023-09-23T04:44:23.809275+0000","last_fullsized":"2023-09-23T04:44:23.809275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802672+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:44:23.802672+0000","last_peered":"2023-09-23T04:44:23.802672+0000","last_clean":"2023-09-23T04:44:23.802672+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:44:23.802672+0000","last_undegraded":"2023-09-23T04:44:23.802672+0000","last_fullsized":"2023-09-23T04:44:23.802672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803598+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:44:23.803598+0000","last_peered":"2023-09-23T04:44:23.803598+0000","last_clean":"2023-09-23T04:44:23.803598+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:44:23.803598+0000","last_undegraded":"2023-09-23T04:44:23.803598+0000","last_fullsized":"2023-09-23T04:44:23.803598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.816417+0000","last_change":"2023-09-23T04:44:24.816417+0000","last_active":"2023-09-23T04:44:24.816417+0000","last_peered":"2023-09-23T04:44:24.816417+0000","last_clean":"2023-09-23T04:44:24.816417+0000","last_became_active":"2023-09-23T04:44:24.815984+0000","last_became_peered":"2023-09-23T04:44:24.815984+0000","last_unstale":"2023-09-23T04:44:24.816417+0000","last_undegraded":"2023-09-23T04:44:24.816417+0000","last_fullsized":"2023-09-23T04:44:24.816417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:07.080432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804189+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:44:23.804189+0000","last_peered":"2023-09-23T04:44:23.804189+0000","last_clean":"2023-09-23T04:44:23.804189+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:44:23.804189+0000","last_undegraded":"2023-09-23T04:44:23.804189+0000","last_fullsized":"2023-09-23T04:44:23.804189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.520623+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:45:03.520623+0000","last_peered":"2023-09-23T04:45:03.520623+0000","last_clean":"2023-09-23T04:45:03.520623+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:45:03.520623+0000","last_undegraded":"2023-09-23T04:45:03.520623+0000","last_fullsized":"2023-09-23T04:45:03.520623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803119+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:44:23.803119+0000","last_peered":"2023-09-23T04:44:23.803119+0000","last_clean":"2023-09-23T04:44:23.803119+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:44:23.803119+0000","last_undegraded":"2023-09-23T04:44:23.803119+0000","last_fullsized":"2023-09-23T04:44:23.803119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804786+0000","last_change":"2023-09-23T04:43:51.735548+0000","last_active":"2023-09-23T04:44:23.804786+0000","last_peered":"2023-09-23T04:44:23.804786+0000","last_clean":"2023-09-23T04:44:23.804786+0000","last_became_active":"2023-09-23T04:43:51.735048+0000","last_became_peered":"2023-09-23T04:43:51.735048+0000","last_unstale":"2023-09-23T04:44:23.804786+0000","last_undegraded":"2023-09-23T04:44:23.804786+0000","last_fullsized":"2023-09-23T04:44:23.804786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:15:25.552028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.819153+0000","last_change":"2023-09-23T04:44:24.819153+0000","last_active":"2023-09-23T04:44:24.819153+0000","last_peered":"2023-09-23T04:44:24.819153+0000","last_clean":"2023-09-23T04:44:24.819153+0000","last_became_active":"2023-09-23T04:44:24.809881+0000","last_became_peered":"2023-09-23T04:44:24.809881+0000","last_unstale":"2023-09-23T04:44:24.819153+0000","last_undegraded":"2023-09-23T04:44:24.819153+0000","last_fullsized":"2023-09-23T04:44:24.819153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:38.014881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.813256+0000","last_change":"2023-09-23T04:44:24.813256+0000","last_active":"2023-09-23T04:44:24.813256+0000","last_peered":"2023-09-23T04:44:24.813256+0000","last_clean":"2023-09-23T04:44:24.813256+0000","last_became_active":"2023-09-23T04:44:24.812749+0000","last_became_peered":"2023-09-23T04:44:24.812749+0000","last_unstale":"2023-09-23T04:44:24.813256+0000","last_undegraded":"2023-09-23T04:44:24.813256+0000","last_fullsized":"2023-09-23T04:44:24.813256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:54:04.120039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808297+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:44:23.808297+0000","last_peered":"2023-09-23T04:44:23.808297+0000","last_clean":"2023-09-23T04:44:23.808297+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:44:23.808297+0000","last_undegraded":"2023-09-23T04:44:23.808297+0000","last_fullsized":"2023-09-23T04:44:23.808297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810951+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:44:23.810951+0000","last_peered":"2023-09-23T04:44:23.810951+0000","last_clean":"2023-09-23T04:44:23.810951+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:44:23.810951+0000","last_undegraded":"2023-09-23T04:44:23.810951+0000","last_fullsized":"2023-09-23T04:44:23.810951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806585+0000","last_change":"2023-09-23T04:43:47.591680+0000","last_active":"2023-09-23T04:44:23.806585+0000","last_peered":"2023-09-23T04:44:23.806585+0000","last_clean":"2023-09-23T04:44:23.806585+0000","last_became_active":"2023-09-23T04:43:47.590872+0000","last_became_peered":"2023-09-23T04:43:47.590872+0000","last_unstale":"2023-09-23T04:44:23.806585+0000","last_undegraded":"2023-09-23T04:44:23.806585+0000","last_fullsized":"2023-09-23T04:44:23.806585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:49:20.600268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805775+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:44:23.805775+0000","last_peered":"2023-09-23T04:44:23.805775+0000","last_clean":"2023-09-23T04:44:23.805775+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:44:23.805775+0000","last_undegraded":"2023-09-23T04:44:23.805775+0000","last_fullsized":"2023-09-23T04:44:23.805775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808024+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:44:23.808024+0000","last_peered":"2023-09-23T04:44:23.808024+0000","last_clean":"2023-09-23T04:44:23.808024+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:44:23.808024+0000","last_undegraded":"2023-09-23T04:44:23.808024+0000","last_fullsized":"2023-09-23T04:44:23.808024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.812795+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:44:23.812795+0000","last_peered":"2023-09-23T04:44:23.812795+0000","last_clean":"2023-09-23T04:44:23.812795+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:44:23.812795+0000","last_undegraded":"2023-09-23T04:44:23.812795+0000","last_fullsized":"2023-09-23T04:44:23.812795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806555+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:44:23.806555+0000","last_peered":"2023-09-23T04:44:23.806555+0000","last_clean":"2023-09-23T04:44:23.806555+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:44:23.806555+0000","last_undegraded":"2023-09-23T04:44:23.806555+0000","last_fullsized":"2023-09-23T04:44:23.806555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.519668+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:45:03.519668+0000","last_peered":"2023-09-23T04:45:03.519668+0000","last_clean":"2023-09-23T04:45:03.519668+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:45:03.519668+0000","last_undegraded":"2023-09-23T04:45:03.519668+0000","last_fullsized":"2023-09-23T04:45:03.519668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810181+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:44:23.810181+0000","last_peered":"2023-09-23T04:44:23.810181+0000","last_clean":"2023-09-23T04:44:23.810181+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:44:23.810181+0000","last_undegraded":"2023-09-23T04:44:23.810181+0000","last_fullsized":"2023-09-23T04:44:23.810181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813240+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:44:23.813240+0000","last_peered":"2023-09-23T04:44:23.813240+0000","last_clean":"2023-09-23T04:44:23.813240+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:44:23.813240+0000","last_undegraded":"2023-09-23T04:44:23.813240+0000","last_fullsized":"2023-09-23T04:44:23.813240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808498+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:44:23.808498+0000","last_peered":"2023-09-23T04:44:23.808498+0000","last_clean":"2023-09-23T04:44:23.808498+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:44:23.808498+0000","last_undegraded":"2023-09-23T04:44:23.808498+0000","last_fullsized":"2023-09-23T04:44:23.808498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.813806+0000","last_change":"2023-09-23T04:44:24.813806+0000","last_active":"2023-09-23T04:44:24.813806+0000","last_peered":"2023-09-23T04:44:24.813806+0000","last_clean":"2023-09-23T04:44:24.813806+0000","last_became_active":"2023-09-23T04:44:24.813414+0000","last_became_peered":"2023-09-23T04:44:24.813414+0000","last_unstale":"2023-09-23T04:44:24.813806+0000","last_undegraded":"2023-09-23T04:44:24.813806+0000","last_fullsized":"2023-09-23T04:44:24.813806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:05:26.739283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802896+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:44:23.802896+0000","last_peered":"2023-09-23T04:44:23.802896+0000","last_clean":"2023-09-23T04:44:23.802896+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:44:23.802896+0000","last_undegraded":"2023-09-23T04:44:23.802896+0000","last_fullsized":"2023-09-23T04:44:23.802896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808530+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:44:23.808530+0000","last_peered":"2023-09-23T04:44:23.808530+0000","last_clean":"2023-09-23T04:44:23.808530+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:44:23.808530+0000","last_undegraded":"2023-09-23T04:44:23.808530+0000","last_fullsized":"2023-09-23T04:44:23.808530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809154+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:44:23.809154+0000","last_peered":"2023-09-23T04:44:23.809154+0000","last_clean":"2023-09-23T04:44:23.809154+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:44:23.809154+0000","last_undegraded":"2023-09-23T04:44:23.809154+0000","last_fullsized":"2023-09-23T04:44:23.809154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.812779+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:44:23.812779+0000","last_peered":"2023-09-23T04:44:23.812779+0000","last_clean":"2023-09-23T04:44:23.812779+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:44:23.812779+0000","last_undegraded":"2023-09-23T04:44:23.812779+0000","last_fullsized":"2023-09-23T04:44:23.812779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.519651+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:45:03.519651+0000","last_peered":"2023-09-23T04:45:03.519651+0000","last_clean":"2023-09-23T04:45:03.519651+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:45:03.519651+0000","last_undegraded":"2023-09-23T04:45:03.519651+0000","last_fullsized":"2023-09-23T04:45:03.519651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":95,"num_read_kb":106,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803437+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:44:23.803437+0000","last_peered":"2023-09-23T04:44:23.803437+0000","last_clean":"2023-09-23T04:44:23.803437+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:44:23.803437+0000","last_undegraded":"2023-09-23T04:44:23.803437+0000","last_fullsized":"2023-09-23T04:44:23.803437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809584+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:44:23.809584+0000","last_peered":"2023-09-23T04:44:23.809584+0000","last_clean":"2023-09-23T04:44:23.809584+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:44:23.809584+0000","last_undegraded":"2023-09-23T04:44:23.809584+0000","last_fullsized":"2023-09-23T04:44:23.809584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804710+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:44:23.804710+0000","last_peered":"2023-09-23T04:44:23.804710+0000","last_clean":"2023-09-23T04:44:23.804710+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:44:23.804710+0000","last_undegraded":"2023-09-23T04:44:23.804710+0000","last_fullsized":"2023-09-23T04:44:23.804710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811028+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:44:23.811028+0000","last_peered":"2023-09-23T04:44:23.811028+0000","last_clean":"2023-09-23T04:44:23.811028+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:44:23.811028+0000","last_undegraded":"2023-09-23T04:44:23.811028+0000","last_fullsized":"2023-09-23T04:44:23.811028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808601+0000","last_change":"2023-09-23T04:43:51.731564+0000","last_active":"2023-09-23T04:44:23.808601+0000","last_peered":"2023-09-23T04:44:23.808601+0000","last_clean":"2023-09-23T04:44:23.808601+0000","last_became_active":"2023-09-23T04:43:51.731151+0000","last_became_peered":"2023-09-23T04:43:51.731151+0000","last_unstale":"2023-09-23T04:44:23.808601+0000","last_undegraded":"2023-09-23T04:44:23.808601+0000","last_fullsized":"2023-09-23T04:44:23.808601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.954294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808528+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:44:23.808528+0000","last_peered":"2023-09-23T04:44:23.808528+0000","last_clean":"2023-09-23T04:44:23.808528+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:44:23.808528+0000","last_undegraded":"2023-09-23T04:44:23.808528+0000","last_fullsized":"2023-09-23T04:44:23.808528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806413+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:44:23.806413+0000","last_peered":"2023-09-23T04:44:23.806413+0000","last_clean":"2023-09-23T04:44:23.806413+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:44:23.806413+0000","last_undegraded":"2023-09-23T04:44:23.806413+0000","last_fullsized":"2023-09-23T04:44:23.806413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813077+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:44:23.813077+0000","last_peered":"2023-09-23T04:44:23.813077+0000","last_clean":"2023-09-23T04:44:23.813077+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:44:23.813077+0000","last_undegraded":"2023-09-23T04:44:23.813077+0000","last_fullsized":"2023-09-23T04:44:23.813077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:04.538029+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:45:04.538029+0000","last_peered":"2023-09-23T04:45:04.538029+0000","last_clean":"2023-09-23T04:45:04.538029+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:45:04.538029+0000","last_undegraded":"2023-09-23T04:45:04.538029+0000","last_fullsized":"2023-09-23T04:45:04.538029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806042+0000","last_change":"2023-09-23T04:43:47.584835+0000","last_active":"2023-09-23T04:44:23.806042+0000","last_peered":"2023-09-23T04:44:23.806042+0000","last_clean":"2023-09-23T04:44:23.806042+0000","last_became_active":"2023-09-23T04:43:47.584538+0000","last_became_peered":"2023-09-23T04:43:47.584538+0000","last_unstale":"2023-09-23T04:44:23.806042+0000","last_undegraded":"2023-09-23T04:44:23.806042+0000","last_fullsized":"2023-09-23T04:44:23.806042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:16.222318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811681+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:44:23.811681+0000","last_peered":"2023-09-23T04:44:23.811681+0000","last_clean":"2023-09-23T04:44:23.811681+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:44:23.811681+0000","last_undegraded":"2023-09-23T04:44:23.811681+0000","last_fullsized":"2023-09-23T04:44:23.811681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.830212+0000","last_change":"2023-09-23T04:44:24.830212+0000","last_active":"2023-09-23T04:44:24.830212+0000","last_peered":"2023-09-23T04:44:24.830212+0000","last_clean":"2023-09-23T04:44:24.830212+0000","last_became_active":"2023-09-23T04:44:24.809351+0000","last_became_peered":"2023-09-23T04:44:24.809351+0000","last_unstale":"2023-09-23T04:44:24.830212+0000","last_undegraded":"2023-09-23T04:44:24.830212+0000","last_fullsized":"2023-09-23T04:44:24.830212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:07:32.566651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808152+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:44:23.808152+0000","last_peered":"2023-09-23T04:44:23.808152+0000","last_clean":"2023-09-23T04:44:23.808152+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:44:23.808152+0000","last_undegraded":"2023-09-23T04:44:23.808152+0000","last_fullsized":"2023-09-23T04:44:23.808152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808021+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:44:23.808021+0000","last_peered":"2023-09-23T04:44:23.808021+0000","last_clean":"2023-09-23T04:44:23.808021+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:44:23.808021+0000","last_undegraded":"2023-09-23T04:44:23.808021+0000","last_fullsized":"2023-09-23T04:44:23.808021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.834072+0000","last_change":"2023-09-23T04:44:24.833679+0000","last_active":"2023-09-23T04:44:24.834072+0000","last_peered":"2023-09-23T04:44:24.834072+0000","last_clean":"2023-09-23T04:44:24.834072+0000","last_became_active":"2023-09-23T04:44:24.814917+0000","last_became_peered":"2023-09-23T04:44:24.814917+0000","last_unstale":"2023-09-23T04:44:24.834072+0000","last_undegraded":"2023-09-23T04:44:24.834072+0000","last_fullsized":"2023-09-23T04:44:24.834072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:18:28.868416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808392+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:44:23.808392+0000","last_peered":"2023-09-23T04:44:23.808392+0000","last_clean":"2023-09-23T04:44:23.808392+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:44:23.808392+0000","last_undegraded":"2023-09-23T04:44:23.808392+0000","last_fullsized":"2023-09-23T04:44:23.808392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803020+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:44:23.803020+0000","last_peered":"2023-09-23T04:44:23.803020+0000","last_clean":"2023-09-23T04:44:23.803020+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:44:23.803020+0000","last_undegraded":"2023-09-23T04:44:23.803020+0000","last_fullsized":"2023-09-23T04:44:23.803020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813256+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:44:23.813256+0000","last_peered":"2023-09-23T04:44:23.813256+0000","last_clean":"2023-09-23T04:44:23.813256+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:44:23.813256+0000","last_undegraded":"2023-09-23T04:44:23.813256+0000","last_fullsized":"2023-09-23T04:44:23.813256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807982+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:44:23.807982+0000","last_peered":"2023-09-23T04:44:23.807982+0000","last_clean":"2023-09-23T04:44:23.807982+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:44:23.807982+0000","last_undegraded":"2023-09-23T04:44:23.807982+0000","last_fullsized":"2023-09-23T04:44:23.807982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811279+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:44:23.811279+0000","last_peered":"2023-09-23T04:44:23.811279+0000","last_clean":"2023-09-23T04:44:23.811279+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:44:23.811279+0000","last_undegraded":"2023-09-23T04:44:23.811279+0000","last_fullsized":"2023-09-23T04:44:23.811279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.884387+0000","last_change":"2023-09-23T04:44:24.821431+0000","last_active":"2023-09-23T04:45:03.884387+0000","last_peered":"2023-09-23T04:45:03.884387+0000","last_clean":"2023-09-23T04:45:03.884387+0000","last_became_active":"2023-09-23T04:44:24.812602+0000","last_became_peered":"2023-09-23T04:44:24.812602+0000","last_unstale":"2023-09-23T04:45:03.884387+0000","last_undegraded":"2023-09-23T04:45:03.884387+0000","last_fullsized":"2023-09-23T04:45:03.884387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:48:01.507032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808244+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:44:23.808244+0000","last_peered":"2023-09-23T04:44:23.808244+0000","last_clean":"2023-09-23T04:44:23.808244+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:44:23.808244+0000","last_undegraded":"2023-09-23T04:44:23.808244+0000","last_fullsized":"2023-09-23T04:44:23.808244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811902+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:44:23.811902+0000","last_peered":"2023-09-23T04:44:23.811902+0000","last_clean":"2023-09-23T04:44:23.811902+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:44:23.811902+0000","last_undegraded":"2023-09-23T04:44:23.811902+0000","last_fullsized":"2023-09-23T04:44:23.811902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808442+0000","last_change":"2023-09-23T04:43:47.587791+0000","last_active":"2023-09-23T04:44:23.808442+0000","last_peered":"2023-09-23T04:44:23.808442+0000","last_clean":"2023-09-23T04:44:23.808442+0000","last_became_active":"2023-09-23T04:43:47.587346+0000","last_became_peered":"2023-09-23T04:43:47.587346+0000","last_unstale":"2023-09-23T04:44:23.808442+0000","last_undegraded":"2023-09-23T04:44:23.808442+0000","last_fullsized":"2023-09-23T04:44:23.808442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:13:40.267593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:04.537134+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:45:04.537134+0000","last_peered":"2023-09-23T04:45:04.537134+0000","last_clean":"2023-09-23T04:45:04.537134+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:45:04.537134+0000","last_undegraded":"2023-09-23T04:45:04.537134+0000","last_fullsized":"2023-09-23T04:45:04.537134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807433+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:44:23.807433+0000","last_peered":"2023-09-23T04:44:23.807433+0000","last_clean":"2023-09-23T04:44:23.807433+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:44:23.807433+0000","last_undegraded":"2023-09-23T04:44:23.807433+0000","last_fullsized":"2023-09-23T04:44:23.807433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806809+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:44:23.806809+0000","last_peered":"2023-09-23T04:44:23.806809+0000","last_clean":"2023-09-23T04:44:23.806809+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:44:23.806809+0000","last_undegraded":"2023-09-23T04:44:23.806809+0000","last_fullsized":"2023-09-23T04:44:23.806809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802869+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:44:23.802869+0000","last_peered":"2023-09-23T04:44:23.802869+0000","last_clean":"2023-09-23T04:44:23.802869+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:44:23.802869+0000","last_undegraded":"2023-09-23T04:44:23.802869+0000","last_fullsized":"2023-09-23T04:44:23.802869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805888+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:44:23.805888+0000","last_peered":"2023-09-23T04:44:23.805888+0000","last_clean":"2023-09-23T04:44:23.805888+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:44:23.805888+0000","last_undegraded":"2023-09-23T04:44:23.805888+0000","last_fullsized":"2023-09-23T04:44:23.805888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809394+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:44:23.809394+0000","last_peered":"2023-09-23T04:44:23.809394+0000","last_clean":"2023-09-23T04:44:23.809394+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:44:23.809394+0000","last_undegraded":"2023-09-23T04:44:23.809394+0000","last_fullsized":"2023-09-23T04:44:23.809394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810753+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:44:23.810753+0000","last_peered":"2023-09-23T04:44:23.810753+0000","last_clean":"2023-09-23T04:44:23.810753+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:44:23.810753+0000","last_undegraded":"2023-09-23T04:44:23.810753+0000","last_fullsized":"2023-09-23T04:44:23.810753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805984+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:44:23.805984+0000","last_peered":"2023-09-23T04:44:23.805984+0000","last_clean":"2023-09-23T04:44:23.805984+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:44:23.805984+0000","last_undegraded":"2023-09-23T04:44:23.805984+0000","last_fullsized":"2023-09-23T04:44:23.805984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808876+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:44:23.808876+0000","last_peered":"2023-09-23T04:44:23.808876+0000","last_clean":"2023-09-23T04:44:23.808876+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:44:23.808876+0000","last_undegraded":"2023-09-23T04:44:23.808876+0000","last_fullsized":"2023-09-23T04:44:23.808876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803819+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:44:23.803819+0000","last_peered":"2023-09-23T04:44:23.803819+0000","last_clean":"2023-09-23T04:44:23.803819+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:44:23.803819+0000","last_undegraded":"2023-09-23T04:44:23.803819+0000","last_fullsized":"2023-09-23T04:44:23.803819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811394+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:44:23.811394+0000","last_peered":"2023-09-23T04:44:23.811394+0000","last_clean":"2023-09-23T04:44:23.811394+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:44:23.811394+0000","last_undegraded":"2023-09-23T04:44:23.811394+0000","last_fullsized":"2023-09-23T04:44:23.811394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806486+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:44:23.806486+0000","last_peered":"2023-09-23T04:44:23.806486+0000","last_clean":"2023-09-23T04:44:23.806486+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:44:23.806486+0000","last_undegraded":"2023-09-23T04:44:23.806486+0000","last_fullsized":"2023-09-23T04:44:23.806486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:25.060323+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:44:25.060323+0000","last_peered":"2023-09-23T04:44:25.060323+0000","last_clean":"2023-09-23T04:44:25.060323+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:44:25.060323+0000","last_undegraded":"2023-09-23T04:44:25.060323+0000","last_fullsized":"2023-09-23T04:44:25.060323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808062+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:44:23.808062+0000","last_peered":"2023-09-23T04:44:23.808062+0000","last_clean":"2023-09-23T04:44:23.808062+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:44:23.808062+0000","last_undegraded":"2023-09-23T04:44:23.808062+0000","last_fullsized":"2023-09-23T04:44:23.808062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804969+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:44:23.804969+0000","last_peered":"2023-09-23T04:44:23.804969+0000","last_clean":"2023-09-23T04:44:23.804969+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:44:23.804969+0000","last_undegraded":"2023-09-23T04:44:23.804969+0000","last_fullsized":"2023-09-23T04:44:23.804969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803382+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:44:23.803382+0000","last_peered":"2023-09-23T04:44:23.803382+0000","last_clean":"2023-09-23T04:44:23.803382+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:44:23.803382+0000","last_undegraded":"2023-09-23T04:44:23.803382+0000","last_fullsized":"2023-09-23T04:44:23.803382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805867+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:44:23.805867+0000","last_peered":"2023-09-23T04:44:23.805867+0000","last_clean":"2023-09-23T04:44:23.805867+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:44:23.805867+0000","last_undegraded":"2023-09-23T04:44:23.805867+0000","last_fullsized":"2023-09-23T04:44:23.805867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803159+0000","last_change":"2023-09-23T04:43:51.737574+0000","last_active":"2023-09-23T04:44:23.803159+0000","last_peered":"2023-09-23T04:44:23.803159+0000","last_clean":"2023-09-23T04:44:23.803159+0000","last_became_active":"2023-09-23T04:43:51.736981+0000","last_became_peered":"2023-09-23T04:43:51.736981+0000","last_unstale":"2023-09-23T04:44:23.803159+0000","last_undegraded":"2023-09-23T04:44:23.803159+0000","last_fullsized":"2023-09-23T04:44:23.803159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:59:43.525900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807565+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:44:23.807565+0000","last_peered":"2023-09-23T04:44:23.807565+0000","last_clean":"2023-09-23T04:44:23.807565+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:44:23.807565+0000","last_undegraded":"2023-09-23T04:44:23.807565+0000","last_fullsized":"2023-09-23T04:44:23.807565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802897+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:44:23.802897+0000","last_peered":"2023-09-23T04:44:23.802897+0000","last_clean":"2023-09-23T04:44:23.802897+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:44:23.802897+0000","last_undegraded":"2023-09-23T04:44:23.802897+0000","last_fullsized":"2023-09-23T04:44:23.802897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813276+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:44:23.813276+0000","last_peered":"2023-09-23T04:44:23.813276+0000","last_clean":"2023-09-23T04:44:23.813276+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:44:23.813276+0000","last_undegraded":"2023-09-23T04:44:23.813276+0000","last_fullsized":"2023-09-23T04:44:23.813276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806648+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:44:23.806648+0000","last_peered":"2023-09-23T04:44:23.806648+0000","last_clean":"2023-09-23T04:44:23.806648+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:44:23.806648+0000","last_undegraded":"2023-09-23T04:44:23.806648+0000","last_fullsized":"2023-09-23T04:44:23.806648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807472+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:44:23.807472+0000","last_peered":"2023-09-23T04:44:23.807472+0000","last_clean":"2023-09-23T04:44:23.807472+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:44:23.807472+0000","last_undegraded":"2023-09-23T04:44:23.807472+0000","last_fullsized":"2023-09-23T04:44:23.807472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802987+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:44:23.802987+0000","last_peered":"2023-09-23T04:44:23.802987+0000","last_clean":"2023-09-23T04:44:23.802987+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:44:23.802987+0000","last_undegraded":"2023-09-23T04:44:23.802987+0000","last_fullsized":"2023-09-23T04:44:23.802987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804290+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:44:23.804290+0000","last_peered":"2023-09-23T04:44:23.804290+0000","last_clean":"2023-09-23T04:44:23.804290+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:44:23.804290+0000","last_undegraded":"2023-09-23T04:44:23.804290+0000","last_fullsized":"2023-09-23T04:44:23.804290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808053+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:44:23.808053+0000","last_peered":"2023-09-23T04:44:23.808053+0000","last_clean":"2023-09-23T04:44:23.808053+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:44:23.808053+0000","last_undegraded":"2023-09-23T04:44:23.808053+0000","last_fullsized":"2023-09-23T04:44:23.808053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803036+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:44:23.803036+0000","last_peered":"2023-09-23T04:44:23.803036+0000","last_clean":"2023-09-23T04:44:23.803036+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:44:23.803036+0000","last_undegraded":"2023-09-23T04:44:23.803036+0000","last_fullsized":"2023-09-23T04:44:23.803036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.833845+0000","last_change":"2023-09-23T04:44:24.833845+0000","last_active":"2023-09-23T04:44:24.833845+0000","last_peered":"2023-09-23T04:44:24.833845+0000","last_clean":"2023-09-23T04:44:24.833845+0000","last_became_active":"2023-09-23T04:44:24.814306+0000","last_became_peered":"2023-09-23T04:44:24.814306+0000","last_unstale":"2023-09-23T04:44:24.833845+0000","last_undegraded":"2023-09-23T04:44:24.833845+0000","last_fullsized":"2023-09-23T04:44:24.833845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:59.493234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807044+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:44:23.807044+0000","last_peered":"2023-09-23T04:44:23.807044+0000","last_clean":"2023-09-23T04:44:23.807044+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:44:23.807044+0000","last_undegraded":"2023-09-23T04:44:23.807044+0000","last_fullsized":"2023-09-23T04:44:23.807044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807777+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:44:23.807777+0000","last_peered":"2023-09-23T04:44:23.807777+0000","last_clean":"2023-09-23T04:44:23.807777+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:44:23.807777+0000","last_undegraded":"2023-09-23T04:44:23.807777+0000","last_fullsized":"2023-09-23T04:44:23.807777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:01.517562+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:45:01.517562+0000","last_peered":"2023-09-23T04:45:01.517562+0000","last_clean":"2023-09-23T04:45:01.517562+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:45:01.517562+0000","last_undegraded":"2023-09-23T04:45:01.517562+0000","last_fullsized":"2023-09-23T04:45:01.517562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807690+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:44:23.807690+0000","last_peered":"2023-09-23T04:44:23.807690+0000","last_clean":"2023-09-23T04:44:23.807690+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:44:23.807690+0000","last_undegraded":"2023-09-23T04:44:23.807690+0000","last_fullsized":"2023-09-23T04:44:23.807690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807186+0000","last_change":"2023-09-23T04:43:51.741159+0000","last_active":"2023-09-23T04:44:23.807186+0000","last_peered":"2023-09-23T04:44:23.807186+0000","last_clean":"2023-09-23T04:44:23.807186+0000","last_became_active":"2023-09-23T04:43:51.740696+0000","last_became_peered":"2023-09-23T04:43:51.740696+0000","last_unstale":"2023-09-23T04:44:23.807186+0000","last_undegraded":"2023-09-23T04:44:23.807186+0000","last_fullsized":"2023-09-23T04:44:23.807186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:34.832089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808879+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:44:23.808879+0000","last_peered":"2023-09-23T04:44:23.808879+0000","last_clean":"2023-09-23T04:44:23.808879+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:44:23.808879+0000","last_undegraded":"2023-09-23T04:44:23.808879+0000","last_fullsized":"2023-09-23T04:44:23.808879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808173+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:44:23.808173+0000","last_peered":"2023-09-23T04:44:23.808173+0000","last_clean":"2023-09-23T04:44:23.808173+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:44:23.808173+0000","last_undegraded":"2023-09-23T04:44:23.808173+0000","last_fullsized":"2023-09-23T04:44:23.808173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806675+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:44:23.806675+0000","last_peered":"2023-09-23T04:44:23.806675+0000","last_clean":"2023-09-23T04:44:23.806675+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:44:23.806675+0000","last_undegraded":"2023-09-23T04:44:23.806675+0000","last_fullsized":"2023-09-23T04:44:23.806675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811590+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:44:23.811590+0000","last_peered":"2023-09-23T04:44:23.811590+0000","last_clean":"2023-09-23T04:44:23.811590+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:44:23.811590+0000","last_undegraded":"2023-09-23T04:44:23.811590+0000","last_fullsized":"2023-09-23T04:44:23.811590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808552+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:44:23.808552+0000","last_peered":"2023-09-23T04:44:23.808552+0000","last_clean":"2023-09-23T04:44:23.808552+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:44:23.808552+0000","last_undegraded":"2023-09-23T04:44:23.808552+0000","last_fullsized":"2023-09-23T04:44:23.808552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807578+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:44:23.807578+0000","last_peered":"2023-09-23T04:44:23.807578+0000","last_clean":"2023-09-23T04:44:23.807578+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:44:23.807578+0000","last_undegraded":"2023-09-23T04:44:23.807578+0000","last_fullsized":"2023-09-23T04:44:23.807578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804735+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:44:23.804735+0000","last_peered":"2023-09-23T04:44:23.804735+0000","last_clean":"2023-09-23T04:44:23.804735+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:44:23.804735+0000","last_undegraded":"2023-09-23T04:44:23.804735+0000","last_fullsized":"2023-09-23T04:44:23.804735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804408+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:44:23.804408+0000","last_peered":"2023-09-23T04:44:23.804408+0000","last_clean":"2023-09-23T04:44:23.804408+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:44:23.804408+0000","last_undegraded":"2023-09-23T04:44:23.804408+0000","last_fullsized":"2023-09-23T04:44:23.804408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807418+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:44:23.807418+0000","last_peered":"2023-09-23T04:44:23.807418+0000","last_clean":"2023-09-23T04:44:23.807418+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:44:23.807418+0000","last_undegraded":"2023-09-23T04:44:23.807418+0000","last_fullsized":"2023-09-23T04:44:23.807418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811488+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:44:23.811488+0000","last_peered":"2023-09-23T04:44:23.811488+0000","last_clean":"2023-09-23T04:44:23.811488+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:44:23.811488+0000","last_undegraded":"2023-09-23T04:44:23.811488+0000","last_fullsized":"2023-09-23T04:44:23.811488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808209+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:44:23.808209+0000","last_peered":"2023-09-23T04:44:23.808209+0000","last_clean":"2023-09-23T04:44:23.808209+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:44:23.808209+0000","last_undegraded":"2023-09-23T04:44:23.808209+0000","last_fullsized":"2023-09-23T04:44:23.808209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803958+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:44:23.803958+0000","last_peered":"2023-09-23T04:44:23.803958+0000","last_clean":"2023-09-23T04:44:23.803958+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:44:23.803958+0000","last_undegraded":"2023-09-23T04:44:23.803958+0000","last_fullsized":"2023-09-23T04:44:23.803958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.812717+0000","last_change":"2023-09-23T04:44:24.812717+0000","last_active":"2023-09-23T04:44:24.812717+0000","last_peered":"2023-09-23T04:44:24.812717+0000","last_clean":"2023-09-23T04:44:24.812717+0000","last_became_active":"2023-09-23T04:44:24.812307+0000","last_became_peered":"2023-09-23T04:44:24.812307+0000","last_unstale":"2023-09-23T04:44:24.812717+0000","last_undegraded":"2023-09-23T04:44:24.812717+0000","last_fullsized":"2023-09-23T04:44:24.812717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:43:34.537044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.999803+0000","last_change":"2023-09-23T04:44:24.999461+0000","last_active":"2023-09-23T04:44:24.999803+0000","last_peered":"2023-09-23T04:44:24.999803+0000","last_clean":"2023-09-23T04:44:24.999803+0000","last_became_active":"2023-09-23T04:44:24.999082+0000","last_became_peered":"2023-09-23T04:44:24.999082+0000","last_unstale":"2023-09-23T04:44:24.999803+0000","last_undegraded":"2023-09-23T04:44:24.999803+0000","last_fullsized":"2023-09-23T04:44:24.999803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:20.840200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806736+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:44:23.806736+0000","last_peered":"2023-09-23T04:44:23.806736+0000","last_clean":"2023-09-23T04:44:23.806736+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:44:23.806736+0000","last_undegraded":"2023-09-23T04:44:23.806736+0000","last_fullsized":"2023-09-23T04:44:23.806736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808628+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:44:23.808628+0000","last_peered":"2023-09-23T04:44:23.808628+0000","last_clean":"2023-09-23T04:44:23.808628+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:44:23.808628+0000","last_undegraded":"2023-09-23T04:44:23.808628+0000","last_fullsized":"2023-09-23T04:44:23.808628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811182+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:44:23.811182+0000","last_peered":"2023-09-23T04:44:23.811182+0000","last_clean":"2023-09-23T04:44:23.811182+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:44:23.811182+0000","last_undegraded":"2023-09-23T04:44:23.811182+0000","last_fullsized":"2023-09-23T04:44:23.811182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808770+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:44:23.808770+0000","last_peered":"2023-09-23T04:44:23.808770+0000","last_clean":"2023-09-23T04:44:23.808770+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:44:23.808770+0000","last_undegraded":"2023-09-23T04:44:23.808770+0000","last_fullsized":"2023-09-23T04:44:23.808770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":472,"num_read_kb":308,"num_write":238,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449117,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489038,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.71899999999999997,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55300000000000005}]},{"osd":2,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":0.748},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92500000000000004,"5min":1.034,"15min":1.034},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.238,"15min":1.238},"last":0.59999999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80400000000000005,"5min":1.125,"15min":1.125},"last":0.433},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.80300000000000005,"15min":0.73099999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":10.571999999999999,"5min":10.571999999999999,"15min":10.571999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":10.398999999999999,"5min":10.398999999999999,"15min":10.398999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.92700000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48999999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":0.88900000000000001,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":9.734,"5min":9.734,"15min":9.734},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.278,"5min":0.89000000000000001,"15min":0.82599999999999996},"min":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":10.288,"5min":10.288,"15min":10.288},"last":0.57599999999999996}]},{"osd":10,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98599999999999999,"5min":1.272,"15min":1.272},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.71,"15min":1.71},"last":0.75900000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678047,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1136075,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.59899999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.46000000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.81799999999999995,"15min":0.79800000000000004},"min":{"1min":0.45000000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.573,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.234,"5min":0.75600000000000001,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.0960000000000001,"5min":8.0960000000000001,"15min":8.0960000000000001},"last":0.505}]},{"osd":4,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81899999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.443,"5min":0.433,"15min":0.433},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.76500000000000001,"15min":0.72699999999999998},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.77900000000000003,"15min":0.76200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96599999999999997,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95699999999999996},"min":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.121,"5min":1.0169999999999999,"15min":1},"min":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":5.3760000000000003,"5min":5.3760000000000003,"15min":5.3760000000000003},"last":0.71799999999999997}]},{"osd":11,"last update":"Sat Sep 23 04:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73099999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":492454,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.64100000000000001,"15min":0.60599999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4669999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":0.504},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.75,"15min":0.71399999999999997},"min":{"1min":0.42699999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.749,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":0.85699999999999998,"15min":0.76300000000000001},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":13.364000000000001,"5min":13.364000000000001,"15min":13.364000000000001},"last":0.627},{"interface":"front","average":{"1min":1.2030000000000001,"5min":0.82299999999999995,"15min":0.75900000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.83399999999999996,"15min":0.81999999999999995},"min":{"1min":0.61199999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.3340000000000001,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86099999999999999,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.82499999999999996}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.626,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.91500000000000004,"15min":0.91300000000000003},"min":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.80800000000000005}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.628}]},{"osd":7,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":0.96999999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.89700000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.65200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.71799999999999997}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.997,"15min":0.99099999999999999},"min":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.8330000000000002,"5min":3.8330000000000002,"15min":3.8330000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":1},"min":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.93400000000000005}]},{"osd":11,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.97099999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103208,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":488714,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73499999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47199999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.59399999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.68400000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.57099999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.68100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.284,"15min":1.284},"last":0.749}]},{"osd":9,"last update":"Sat Sep 23 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.748,"15min":0.748},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.874}]},{"osd":10,"last update":"Sat Sep 23 04:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.78500000000000003,"15min":0.76500000000000001},"min":{"1min":0.57499999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.84499999999999997}]},{"osd":11,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.89500000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1084761,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55900000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45100000000000001,"15min":0.442},"min":{"1min":0.29499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81100000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.626}]},{"osd":3,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72999999999999998}]},{"osd":4,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.30199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.68700000000000006,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.65200000000000002},"min":{"1min":0.47799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746}]},{"osd":6,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98199999999999998,"5min":1,"15min":1},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97799999999999998,"5min":1.036,"15min":1.036},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89200000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.746,"15min":0.745},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.004,"5min":1.125,"15min":1.125},"last":0.79600000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95799999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67000000000000004}]},{"osd":11,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86299999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483862,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.59899999999999998,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.2750000000000004,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.57999999999999996,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.748,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.2480000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.79900000000000004,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.627},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.88100000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.83099999999999996,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.60399999999999998}]},{"osd":7,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.81899999999999995,"15min":0.77700000000000002},"min":{"1min":0.68899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.7829999999999999,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.3029999999999999,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.94199999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.998,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.056}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.109,"15min":0.98799999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":15.34,"5min":15.34,"15min":15.34},"last":1.014},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.99099999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.016,"15min":1.012},"min":{"1min":0.81000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":1.1000000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":8.3249999999999993,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":1.036}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":545060,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.72999999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":1.042,"15min":1.042},"last":0.44800000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58799999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.61799999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.30499999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.64000000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":1.014,"15min":1.014},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64400000000000002}]},{"osd":7,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86299999999999999,"5min":0.873,"15min":0.873},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.621,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96299999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71299999999999997}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":0.997,"15min":0.997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0669999999999999,"5min":1.103,"15min":1.103},"last":0.78900000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0840000000000001,"5min":1.232,"15min":1.232},"last":0.755}]},{"osd":11,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.82399999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485318,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.629,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96799999999999997,"5min":1.345,"15min":1.345},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.59199999999999997},"min":{"1min":0.51700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":1.306,"15min":1.306},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59299999999999997},"min":{"1min":0.54000000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66800000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Sep 23 04:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67100000000000004,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88300000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.628,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97799999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.753,"15min":0.755},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.85399999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.96099999999999997},"min":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1060000000000001,"5min":1.871,"15min":1.871},"last":0.69799999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.147,"5min":1.153,"15min":1.153},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89200000000000002,"15min":0.89100000000000001},"min":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2070000000000001,"5min":1.3,"15min":1.3},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84999999999999998,"15min":0.84899999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0069999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.88300000000000001,"15min":0.872},"min":{"1min":0.76800000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.069}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545831,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.86499999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":14.076000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Sep 23 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81999999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68300000000000005}]},{"osd":5,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.46300000000000002,"15min":0.441},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.47099999999999997},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80700000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.90400000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":14.143000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.86899999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.94599999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483239,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.434,"15min":0.39300000000000002},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.38}]},{"osd":1,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65900000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.52200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73799999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86599999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68100000000000005}]},{"osd":5,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.52300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":0.999,"15min":0.999},"last":0.623}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.214,"15min":1.214},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.77400000000000002,"15min":0.76000000000000001},"min":{"1min":0.57999999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.089,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.90200000000000002,"15min":0.90600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":1.115}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.73999999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0700000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.86799999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.88600000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":488438,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.37,"15min":0.34200000000000003},"min":{"1min":0.31,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.56100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.371},"min":{"1min":0.317,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61099999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.874,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58999999999999997,"15min":0.53900000000000003},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89700000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79000000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.59199999999999997},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68999999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80000000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76000000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1080814,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.51500000000000001},"min":{"1min":0.46800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91300000000000003,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.624,"5min":1.246,"15min":1.246},"last":0.55000000000000004}]},{"osd":1,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":1.115,"15min":1.115},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.071,"15min":1.071},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.371},"min":{"1min":0.27300000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.33600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79900000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40100000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.753,"5min":1.212,"15min":1.212},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68999999999999995,"5min":1.042,"15min":1.042},"last":0.94299999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.48199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":6,"last update":"Sat Sep 23 04:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82099999999999995,"5min":1.292,"15min":1.292},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.84899999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95599999999999996,"5min":1.105,"15min":1.105},"last":0.5},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85499999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.58899999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89100000000000001,"5min":1.262,"15min":1.262},"last":1.325},{"interface":"front","average":{"1min":0.75,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.99099999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.282}]},{"osd":10,"last update":"Sat Sep 23 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.67700000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:45:08.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph pg dump --format=json 2023-09-23T04:45:08.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:08 smithi005 ceph-mon[97638]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:08 smithi118 ceph-mon[134297]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:08 smithi069 ceph-mon[133931]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:09.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:09 smithi005 ceph-mon[97638]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:10.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:09 smithi118 ceph-mon[134297]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:09 smithi069 ceph-mon[133931]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 -- 172.21.15.5:0/3936842321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 msgr2=0x7f0c7c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 --2- 172.21.15.5:0/3936842321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c101480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009990 tx=0x7f0c6c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 -- 172.21.15.5:0/3936842321 shutdown_connections 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 --2- 172.21.15.5:0/3936842321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c106180 0x7f0c7c10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 --2- 172.21.15.5:0/3936842321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c101480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 --2- 172.21.15.5:0/3936842321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c101a50 0x7f0c7c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 -- 172.21.15.5:0/3936842321 >> 172.21.15.5:0/3936842321 conn(0x7f0c7c0fc910 msgr2=0x7f0c7c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:10.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 -- 172.21.15.5:0/3936842321 shutdown_connections 2023-09-23T04:45:10.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.857+0000 7f0c8092a700 1 -- 172.21.15.5:0/3936842321 wait complete. 2023-09-23T04:45:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.858+0000 7f0c8092a700 1 Processor -- start 2023-09-23T04:45:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 -- start start 2023-09-23T04:45:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c101a50 0x7f0c7c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c106180 0x7f0c7c073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a53c0 con 0x7f0c7c1010a0 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a5530 con 0x7f0c7c101a50 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c8092a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a56f0 con 0x7f0c7c106180 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49592/0 (socket says 172.21.15.5:49592) 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c7a59c700 1 -- 172.21.15.5:0/2412550696 learned_addr learned my addr 172.21.15.5:0/2412550696 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.859+0000 7f0c79d9b700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c101a50 0x7f0c7c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7ad9d700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c106180 0x7f0c7c073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 -- 172.21.15.5:0/2412550696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c106180 msgr2=0x7f0c7c073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c106180 0x7f0c7c073be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 -- 172.21.15.5:0/2412550696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c101a50 msgr2=0x7f0c7c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c101a50 0x7f0c7c073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 -- 172.21.15.5:0/2412550696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0095f0 con 0x7f0c7c1010a0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c7a59c700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005910 tx=0x7f0c6c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c01d070 con 0x7f0c7c1010a0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c1a5970 con 0x7f0c7c1010a0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c022470 con 0x7f0c7c1010a0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.860+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c017650 con 0x7f0c7c1010a0 2023-09-23T04:45:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.861+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c1a5d80 con 0x7f0c7c1010a0 2023-09-23T04:45:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.862+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c04eb20 con 0x7f0c7c1010a0 2023-09-23T04:45:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.863+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f0c6c0225e0 con 0x7f0c7c1010a0 2023-09-23T04:45:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.865+0000 7f0c6b7fe700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 0x7f0c640a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.865+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0c6c0ce710 con 0x7f0c7c1010a0 2023-09-23T04:45:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.865+0000 7f0c79d9b700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 0x7f0c640a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.866+0000 7f0c79d9b700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 0x7f0c640a3af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c074910 tx=0x7f0c70009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:10.869+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0c6c0978c0 con 0x7f0c7c1010a0 2023-09-23T04:45:10.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:10 smithi005 ceph-mon[97638]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.074+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c7c111090 con 0x7f0c640a1630 2023-09-23T04:45:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.087+0000 7f0c6b7fe700 1 -- 172.21.15.5:0/2412550696 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427808 (secure 0 0 0) 0x7f0c7c111090 con 0x7f0c640a1630 2023-09-23T04:45:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.092+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 msgr2=0x7f0c640a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.092+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 0x7f0c640a3af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c074910 tx=0x7f0c70009260 comp rx=0 tx=0).stop 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 msgr2=0x7f0c7c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005910 tx=0x7f0c6c005720 comp rx=0 tx=0).stop 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 shutdown_connections 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0c640a1630 0x7f0c640a3af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0c7c106180 0x7f0c7c073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c7c1010a0 0x7f0c7c073120 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 --2- 172.21.15.5:0/2412550696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c7c101a50 0x7f0c7c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 >> 172.21.15.5:0/2412550696 conn(0x7f0c7c0fc910 msgr2=0x7f0c7c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 shutdown_connections 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:11.093+0000 7f0c8092a700 1 -- 172.21.15.5:0/2412550696 wait complete. 2023-09-23T04:45:11.101 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:45:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:10 smithi118 ceph-mon[134297]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:10 smithi069 ceph-mon[133931]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:11.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2023-09-23T04:45:09.650879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658257,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":638,"num_read_kb":447,"num_write":421,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181458,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336248,"kb_used_data":11996,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605576,"statfs":{"total":1151940427776,"available":1151596109824,"internally_reserved":0,"allocated":12283904,"data_stored":7803604,"data_compressed":321123,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808506+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:44:23.808506+0000","last_peered":"2023-09-23T04:44:23.808506+0000","last_clean":"2023-09-23T04:44:23.808506+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:44:23.808506+0000","last_undegraded":"2023-09-23T04:44:23.808506+0000","last_fullsized":"2023-09-23T04:44:23.808506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805954+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:44:23.805954+0000","last_peered":"2023-09-23T04:44:23.805954+0000","last_clean":"2023-09-23T04:44:23.805954+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:44:23.805954+0000","last_undegraded":"2023-09-23T04:44:23.805954+0000","last_fullsized":"2023-09-23T04:44:23.805954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806021+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:44:23.806021+0000","last_peered":"2023-09-23T04:44:23.806021+0000","last_clean":"2023-09-23T04:44:23.806021+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:44:23.806021+0000","last_undegraded":"2023-09-23T04:44:23.806021+0000","last_fullsized":"2023-09-23T04:44:23.806021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803120+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:44:23.803120+0000","last_peered":"2023-09-23T04:44:23.803120+0000","last_clean":"2023-09-23T04:44:23.803120+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:44:23.803120+0000","last_undegraded":"2023-09-23T04:44:23.803120+0000","last_fullsized":"2023-09-23T04:44:23.803120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803525+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:44:23.803525+0000","last_peered":"2023-09-23T04:44:23.803525+0000","last_clean":"2023-09-23T04:44:23.803525+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:44:23.803525+0000","last_undegraded":"2023-09-23T04:44:23.803525+0000","last_fullsized":"2023-09-23T04:44:23.803525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807910+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:44:23.807910+0000","last_peered":"2023-09-23T04:44:23.807910+0000","last_clean":"2023-09-23T04:44:23.807910+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:44:23.807910+0000","last_undegraded":"2023-09-23T04:44:23.807910+0000","last_fullsized":"2023-09-23T04:44:23.807910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806203+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:44:23.806203+0000","last_peered":"2023-09-23T04:44:23.806203+0000","last_clean":"2023-09-23T04:44:23.806203+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:44:23.806203+0000","last_undegraded":"2023-09-23T04:44:23.806203+0000","last_fullsized":"2023-09-23T04:44:23.806203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806391+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:44:23.806391+0000","last_peered":"2023-09-23T04:44:23.806391+0000","last_clean":"2023-09-23T04:44:23.806391+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:44:23.806391+0000","last_undegraded":"2023-09-23T04:44:23.806391+0000","last_fullsized":"2023-09-23T04:44:23.806391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810224+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:44:23.810224+0000","last_peered":"2023-09-23T04:44:23.810224+0000","last_clean":"2023-09-23T04:44:23.810224+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:44:23.810224+0000","last_undegraded":"2023-09-23T04:44:23.810224+0000","last_fullsized":"2023-09-23T04:44:23.810224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808475+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:44:23.808475+0000","last_peered":"2023-09-23T04:44:23.808475+0000","last_clean":"2023-09-23T04:44:23.808475+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:44:23.808475+0000","last_undegraded":"2023-09-23T04:44:23.808475+0000","last_fullsized":"2023-09-23T04:44:23.808475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807762+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:44:23.807762+0000","last_peered":"2023-09-23T04:44:23.807762+0000","last_clean":"2023-09-23T04:44:23.807762+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:44:23.807762+0000","last_undegraded":"2023-09-23T04:44:23.807762+0000","last_fullsized":"2023-09-23T04:44:23.807762+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803160+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:44:23.803160+0000","last_peered":"2023-09-23T04:44:23.803160+0000","last_clean":"2023-09-23T04:44:23.803160+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:44:23.803160+0000","last_undegraded":"2023-09-23T04:44:23.803160+0000","last_fullsized":"2023-09-23T04:44:23.803160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811269+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:44:23.811269+0000","last_peered":"2023-09-23T04:44:23.811269+0000","last_clean":"2023-09-23T04:44:23.811269+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:44:23.811269+0000","last_undegraded":"2023-09-23T04:44:23.811269+0000","last_fullsized":"2023-09-23T04:44:23.811269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805584+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:44:23.805584+0000","last_peered":"2023-09-23T04:44:23.805584+0000","last_clean":"2023-09-23T04:44:23.805584+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:44:23.805584+0000","last_undegraded":"2023-09-23T04:44:23.805584+0000","last_fullsized":"2023-09-23T04:44:23.805584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807939+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:44:23.807939+0000","last_peered":"2023-09-23T04:44:23.807939+0000","last_clean":"2023-09-23T04:44:23.807939+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:44:23.807939+0000","last_undegraded":"2023-09-23T04:44:23.807939+0000","last_fullsized":"2023-09-23T04:44:23.807939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807423+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:44:23.807423+0000","last_peered":"2023-09-23T04:44:23.807423+0000","last_clean":"2023-09-23T04:44:23.807423+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:44:23.807423+0000","last_undegraded":"2023-09-23T04:44:23.807423+0000","last_fullsized":"2023-09-23T04:44:23.807423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808415+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:44:23.808415+0000","last_peered":"2023-09-23T04:44:23.808415+0000","last_clean":"2023-09-23T04:44:23.808415+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:44:23.808415+0000","last_undegraded":"2023-09-23T04:44:23.808415+0000","last_fullsized":"2023-09-23T04:44:23.808415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813088+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:44:23.813088+0000","last_peered":"2023-09-23T04:44:23.813088+0000","last_clean":"2023-09-23T04:44:23.813088+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:44:23.813088+0000","last_undegraded":"2023-09-23T04:44:23.813088+0000","last_fullsized":"2023-09-23T04:44:23.813088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804381+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:44:23.804381+0000","last_peered":"2023-09-23T04:44:23.804381+0000","last_clean":"2023-09-23T04:44:23.804381+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:44:23.804381+0000","last_undegraded":"2023-09-23T04:44:23.804381+0000","last_fullsized":"2023-09-23T04:44:23.804381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804863+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:44:23.804863+0000","last_peered":"2023-09-23T04:44:23.804863+0000","last_clean":"2023-09-23T04:44:23.804863+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:44:23.804863+0000","last_undegraded":"2023-09-23T04:44:23.804863+0000","last_fullsized":"2023-09-23T04:44:23.804863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:06.888055+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:45:06.888055+0000","last_peered":"2023-09-23T04:45:06.888055+0000","last_clean":"2023-09-23T04:45:06.888055+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:45:06.888055+0000","last_undegraded":"2023-09-23T04:45:06.888055+0000","last_fullsized":"2023-09-23T04:45:06.888055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807581+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:44:23.807581+0000","last_peered":"2023-09-23T04:44:23.807581+0000","last_clean":"2023-09-23T04:44:23.807581+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:44:23.807581+0000","last_undegraded":"2023-09-23T04:44:23.807581+0000","last_fullsized":"2023-09-23T04:44:23.807581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802552+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:44:23.802552+0000","last_peered":"2023-09-23T04:44:23.802552+0000","last_clean":"2023-09-23T04:44:23.802552+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:44:23.802552+0000","last_undegraded":"2023-09-23T04:44:23.802552+0000","last_fullsized":"2023-09-23T04:44:23.802552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807732+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:44:23.807732+0000","last_peered":"2023-09-23T04:44:23.807732+0000","last_clean":"2023-09-23T04:44:23.807732+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:44:23.807732+0000","last_undegraded":"2023-09-23T04:44:23.807732+0000","last_fullsized":"2023-09-23T04:44:23.807732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807678+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:44:23.807678+0000","last_peered":"2023-09-23T04:44:23.807678+0000","last_clean":"2023-09-23T04:44:23.807678+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:44:23.807678+0000","last_undegraded":"2023-09-23T04:44:23.807678+0000","last_fullsized":"2023-09-23T04:44:23.807678+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802751+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:44:23.802751+0000","last_peered":"2023-09-23T04:44:23.802751+0000","last_clean":"2023-09-23T04:44:23.802751+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:44:23.802751+0000","last_undegraded":"2023-09-23T04:44:23.802751+0000","last_fullsized":"2023-09-23T04:44:23.802751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.805382+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:44:24.805382+0000","last_peered":"2023-09-23T04:44:24.805382+0000","last_clean":"2023-09-23T04:44:24.805382+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:44:24.805382+0000","last_undegraded":"2023-09-23T04:44:24.805382+0000","last_fullsized":"2023-09-23T04:44:24.805382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809275+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:44:23.809275+0000","last_peered":"2023-09-23T04:44:23.809275+0000","last_clean":"2023-09-23T04:44:23.809275+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:44:23.809275+0000","last_undegraded":"2023-09-23T04:44:23.809275+0000","last_fullsized":"2023-09-23T04:44:23.809275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802672+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:44:23.802672+0000","last_peered":"2023-09-23T04:44:23.802672+0000","last_clean":"2023-09-23T04:44:23.802672+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:44:23.802672+0000","last_undegraded":"2023-09-23T04:44:23.802672+0000","last_fullsized":"2023-09-23T04:44:23.802672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803598+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:44:23.803598+0000","last_peered":"2023-09-23T04:44:23.803598+0000","last_clean":"2023-09-23T04:44:23.803598+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:44:23.803598+0000","last_undegraded":"2023-09-23T04:44:23.803598+0000","last_fullsized":"2023-09-23T04:44:23.803598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.816417+0000","last_change":"2023-09-23T04:44:24.816417+0000","last_active":"2023-09-23T04:44:24.816417+0000","last_peered":"2023-09-23T04:44:24.816417+0000","last_clean":"2023-09-23T04:44:24.816417+0000","last_became_active":"2023-09-23T04:44:24.815984+0000","last_became_peered":"2023-09-23T04:44:24.815984+0000","last_unstale":"2023-09-23T04:44:24.816417+0000","last_undegraded":"2023-09-23T04:44:24.816417+0000","last_fullsized":"2023-09-23T04:44:24.816417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:07.080432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804189+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:44:23.804189+0000","last_peered":"2023-09-23T04:44:23.804189+0000","last_clean":"2023-09-23T04:44:23.804189+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:44:23.804189+0000","last_undegraded":"2023-09-23T04:44:23.804189+0000","last_fullsized":"2023-09-23T04:44:23.804189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.520623+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:45:03.520623+0000","last_peered":"2023-09-23T04:45:03.520623+0000","last_clean":"2023-09-23T04:45:03.520623+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:45:03.520623+0000","last_undegraded":"2023-09-23T04:45:03.520623+0000","last_fullsized":"2023-09-23T04:45:03.520623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803119+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:44:23.803119+0000","last_peered":"2023-09-23T04:44:23.803119+0000","last_clean":"2023-09-23T04:44:23.803119+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:44:23.803119+0000","last_undegraded":"2023-09-23T04:44:23.803119+0000","last_fullsized":"2023-09-23T04:44:23.803119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804786+0000","last_change":"2023-09-23T04:43:51.735548+0000","last_active":"2023-09-23T04:44:23.804786+0000","last_peered":"2023-09-23T04:44:23.804786+0000","last_clean":"2023-09-23T04:44:23.804786+0000","last_became_active":"2023-09-23T04:43:51.735048+0000","last_became_peered":"2023-09-23T04:43:51.735048+0000","last_unstale":"2023-09-23T04:44:23.804786+0000","last_undegraded":"2023-09-23T04:44:23.804786+0000","last_fullsized":"2023-09-23T04:44:23.804786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:15:25.552028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.819153+0000","last_change":"2023-09-23T04:44:24.819153+0000","last_active":"2023-09-23T04:44:24.819153+0000","last_peered":"2023-09-23T04:44:24.819153+0000","last_clean":"2023-09-23T04:44:24.819153+0000","last_became_active":"2023-09-23T04:44:24.809881+0000","last_became_peered":"2023-09-23T04:44:24.809881+0000","last_unstale":"2023-09-23T04:44:24.819153+0000","last_undegraded":"2023-09-23T04:44:24.819153+0000","last_fullsized":"2023-09-23T04:44:24.819153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:38.014881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.813256+0000","last_change":"2023-09-23T04:44:24.813256+0000","last_active":"2023-09-23T04:44:24.813256+0000","last_peered":"2023-09-23T04:44:24.813256+0000","last_clean":"2023-09-23T04:44:24.813256+0000","last_became_active":"2023-09-23T04:44:24.812749+0000","last_became_peered":"2023-09-23T04:44:24.812749+0000","last_unstale":"2023-09-23T04:44:24.813256+0000","last_undegraded":"2023-09-23T04:44:24.813256+0000","last_fullsized":"2023-09-23T04:44:24.813256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:54:04.120039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808297+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:44:23.808297+0000","last_peered":"2023-09-23T04:44:23.808297+0000","last_clean":"2023-09-23T04:44:23.808297+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:44:23.808297+0000","last_undegraded":"2023-09-23T04:44:23.808297+0000","last_fullsized":"2023-09-23T04:44:23.808297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810951+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:44:23.810951+0000","last_peered":"2023-09-23T04:44:23.810951+0000","last_clean":"2023-09-23T04:44:23.810951+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:44:23.810951+0000","last_undegraded":"2023-09-23T04:44:23.810951+0000","last_fullsized":"2023-09-23T04:44:23.810951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806585+0000","last_change":"2023-09-23T04:43:47.591680+0000","last_active":"2023-09-23T04:44:23.806585+0000","last_peered":"2023-09-23T04:44:23.806585+0000","last_clean":"2023-09-23T04:44:23.806585+0000","last_became_active":"2023-09-23T04:43:47.590872+0000","last_became_peered":"2023-09-23T04:43:47.590872+0000","last_unstale":"2023-09-23T04:44:23.806585+0000","last_undegraded":"2023-09-23T04:44:23.806585+0000","last_fullsized":"2023-09-23T04:44:23.806585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:49:20.600268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805775+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:44:23.805775+0000","last_peered":"2023-09-23T04:44:23.805775+0000","last_clean":"2023-09-23T04:44:23.805775+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:44:23.805775+0000","last_undegraded":"2023-09-23T04:44:23.805775+0000","last_fullsized":"2023-09-23T04:44:23.805775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808024+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:44:23.808024+0000","last_peered":"2023-09-23T04:44:23.808024+0000","last_clean":"2023-09-23T04:44:23.808024+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:44:23.808024+0000","last_undegraded":"2023-09-23T04:44:23.808024+0000","last_fullsized":"2023-09-23T04:44:23.808024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.812795+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:44:23.812795+0000","last_peered":"2023-09-23T04:44:23.812795+0000","last_clean":"2023-09-23T04:44:23.812795+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:44:23.812795+0000","last_undegraded":"2023-09-23T04:44:23.812795+0000","last_fullsized":"2023-09-23T04:44:23.812795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806555+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:44:23.806555+0000","last_peered":"2023-09-23T04:44:23.806555+0000","last_clean":"2023-09-23T04:44:23.806555+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:44:23.806555+0000","last_undegraded":"2023-09-23T04:44:23.806555+0000","last_fullsized":"2023-09-23T04:44:23.806555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:08.527406+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:45:08.527406+0000","last_peered":"2023-09-23T04:45:08.527406+0000","last_clean":"2023-09-23T04:45:08.527406+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:45:08.527406+0000","last_undegraded":"2023-09-23T04:45:08.527406+0000","last_fullsized":"2023-09-23T04:45:08.527406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810181+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:44:23.810181+0000","last_peered":"2023-09-23T04:44:23.810181+0000","last_clean":"2023-09-23T04:44:23.810181+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:44:23.810181+0000","last_undegraded":"2023-09-23T04:44:23.810181+0000","last_fullsized":"2023-09-23T04:44:23.810181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813240+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:44:23.813240+0000","last_peered":"2023-09-23T04:44:23.813240+0000","last_clean":"2023-09-23T04:44:23.813240+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:44:23.813240+0000","last_undegraded":"2023-09-23T04:44:23.813240+0000","last_fullsized":"2023-09-23T04:44:23.813240+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808498+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:44:23.808498+0000","last_peered":"2023-09-23T04:44:23.808498+0000","last_clean":"2023-09-23T04:44:23.808498+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:44:23.808498+0000","last_undegraded":"2023-09-23T04:44:23.808498+0000","last_fullsized":"2023-09-23T04:44:23.808498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.813806+0000","last_change":"2023-09-23T04:44:24.813806+0000","last_active":"2023-09-23T04:44:24.813806+0000","last_peered":"2023-09-23T04:44:24.813806+0000","last_clean":"2023-09-23T04:44:24.813806+0000","last_became_active":"2023-09-23T04:44:24.813414+0000","last_became_peered":"2023-09-23T04:44:24.813414+0000","last_unstale":"2023-09-23T04:44:24.813806+0000","last_undegraded":"2023-09-23T04:44:24.813806+0000","last_fullsized":"2023-09-23T04:44:24.813806+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:05:26.739283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802896+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:44:23.802896+0000","last_peered":"2023-09-23T04:44:23.802896+0000","last_clean":"2023-09-23T04:44:23.802896+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:44:23.802896+0000","last_undegraded":"2023-09-23T04:44:23.802896+0000","last_fullsized":"2023-09-23T04:44:23.802896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808530+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:44:23.808530+0000","last_peered":"2023-09-23T04:44:23.808530+0000","last_clean":"2023-09-23T04:44:23.808530+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:44:23.808530+0000","last_undegraded":"2023-09-23T04:44:23.808530+0000","last_fullsized":"2023-09-23T04:44:23.808530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809154+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:44:23.809154+0000","last_peered":"2023-09-23T04:44:23.809154+0000","last_clean":"2023-09-23T04:44:23.809154+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:44:23.809154+0000","last_undegraded":"2023-09-23T04:44:23.809154+0000","last_fullsized":"2023-09-23T04:44:23.809154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.812779+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:44:23.812779+0000","last_peered":"2023-09-23T04:44:23.812779+0000","last_clean":"2023-09-23T04:44:23.812779+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:44:23.812779+0000","last_undegraded":"2023-09-23T04:44:23.812779+0000","last_fullsized":"2023-09-23T04:44:23.812779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:08.527406+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:45:08.527406+0000","last_peered":"2023-09-23T04:45:08.527406+0000","last_clean":"2023-09-23T04:45:08.527406+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:45:08.527406+0000","last_undegraded":"2023-09-23T04:45:08.527406+0000","last_fullsized":"2023-09-23T04:45:08.527406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":111,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803437+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:44:23.803437+0000","last_peered":"2023-09-23T04:44:23.803437+0000","last_clean":"2023-09-23T04:44:23.803437+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:44:23.803437+0000","last_undegraded":"2023-09-23T04:44:23.803437+0000","last_fullsized":"2023-09-23T04:44:23.803437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809584+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:44:23.809584+0000","last_peered":"2023-09-23T04:44:23.809584+0000","last_clean":"2023-09-23T04:44:23.809584+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:44:23.809584+0000","last_undegraded":"2023-09-23T04:44:23.809584+0000","last_fullsized":"2023-09-23T04:44:23.809584+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804710+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:44:23.804710+0000","last_peered":"2023-09-23T04:44:23.804710+0000","last_clean":"2023-09-23T04:44:23.804710+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:44:23.804710+0000","last_undegraded":"2023-09-23T04:44:23.804710+0000","last_fullsized":"2023-09-23T04:44:23.804710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811028+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:44:23.811028+0000","last_peered":"2023-09-23T04:44:23.811028+0000","last_clean":"2023-09-23T04:44:23.811028+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:44:23.811028+0000","last_undegraded":"2023-09-23T04:44:23.811028+0000","last_fullsized":"2023-09-23T04:44:23.811028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808601+0000","last_change":"2023-09-23T04:43:51.731564+0000","last_active":"2023-09-23T04:44:23.808601+0000","last_peered":"2023-09-23T04:44:23.808601+0000","last_clean":"2023-09-23T04:44:23.808601+0000","last_became_active":"2023-09-23T04:43:51.731151+0000","last_became_peered":"2023-09-23T04:43:51.731151+0000","last_unstale":"2023-09-23T04:44:23.808601+0000","last_undegraded":"2023-09-23T04:44:23.808601+0000","last_fullsized":"2023-09-23T04:44:23.808601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.954294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808528+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:44:23.808528+0000","last_peered":"2023-09-23T04:44:23.808528+0000","last_clean":"2023-09-23T04:44:23.808528+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:44:23.808528+0000","last_undegraded":"2023-09-23T04:44:23.808528+0000","last_fullsized":"2023-09-23T04:44:23.808528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806413+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:44:23.806413+0000","last_peered":"2023-09-23T04:44:23.806413+0000","last_clean":"2023-09-23T04:44:23.806413+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:44:23.806413+0000","last_undegraded":"2023-09-23T04:44:23.806413+0000","last_fullsized":"2023-09-23T04:44:23.806413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813077+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:44:23.813077+0000","last_peered":"2023-09-23T04:44:23.813077+0000","last_clean":"2023-09-23T04:44:23.813077+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:44:23.813077+0000","last_undegraded":"2023-09-23T04:44:23.813077+0000","last_fullsized":"2023-09-23T04:44:23.813077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:04.538029+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:45:04.538029+0000","last_peered":"2023-09-23T04:45:04.538029+0000","last_clean":"2023-09-23T04:45:04.538029+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:45:04.538029+0000","last_undegraded":"2023-09-23T04:45:04.538029+0000","last_fullsized":"2023-09-23T04:45:04.538029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806042+0000","last_change":"2023-09-23T04:43:47.584835+0000","last_active":"2023-09-23T04:44:23.806042+0000","last_peered":"2023-09-23T04:44:23.806042+0000","last_clean":"2023-09-23T04:44:23.806042+0000","last_became_active":"2023-09-23T04:43:47.584538+0000","last_became_peered":"2023-09-23T04:43:47.584538+0000","last_unstale":"2023-09-23T04:44:23.806042+0000","last_undegraded":"2023-09-23T04:44:23.806042+0000","last_fullsized":"2023-09-23T04:44:23.806042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:16.222318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811681+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:44:23.811681+0000","last_peered":"2023-09-23T04:44:23.811681+0000","last_clean":"2023-09-23T04:44:23.811681+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:44:23.811681+0000","last_undegraded":"2023-09-23T04:44:23.811681+0000","last_fullsized":"2023-09-23T04:44:23.811681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.830212+0000","last_change":"2023-09-23T04:44:24.830212+0000","last_active":"2023-09-23T04:44:24.830212+0000","last_peered":"2023-09-23T04:44:24.830212+0000","last_clean":"2023-09-23T04:44:24.830212+0000","last_became_active":"2023-09-23T04:44:24.809351+0000","last_became_peered":"2023-09-23T04:44:24.809351+0000","last_unstale":"2023-09-23T04:44:24.830212+0000","last_undegraded":"2023-09-23T04:44:24.830212+0000","last_fullsized":"2023-09-23T04:44:24.830212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:07:32.566651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808152+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:44:23.808152+0000","last_peered":"2023-09-23T04:44:23.808152+0000","last_clean":"2023-09-23T04:44:23.808152+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:44:23.808152+0000","last_undegraded":"2023-09-23T04:44:23.808152+0000","last_fullsized":"2023-09-23T04:44:23.808152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808021+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:44:23.808021+0000","last_peered":"2023-09-23T04:44:23.808021+0000","last_clean":"2023-09-23T04:44:23.808021+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:44:23.808021+0000","last_undegraded":"2023-09-23T04:44:23.808021+0000","last_fullsized":"2023-09-23T04:44:23.808021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.834072+0000","last_change":"2023-09-23T04:44:24.833679+0000","last_active":"2023-09-23T04:44:24.834072+0000","last_peered":"2023-09-23T04:44:24.834072+0000","last_clean":"2023-09-23T04:44:24.834072+0000","last_became_active":"2023-09-23T04:44:24.814917+0000","last_became_peered":"2023-09-23T04:44:24.814917+0000","last_unstale":"2023-09-23T04:44:24.834072+0000","last_undegraded":"2023-09-23T04:44:24.834072+0000","last_fullsized":"2023-09-23T04:44:24.834072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:18:28.868416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808392+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:44:23.808392+0000","last_peered":"2023-09-23T04:44:23.808392+0000","last_clean":"2023-09-23T04:44:23.808392+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:44:23.808392+0000","last_undegraded":"2023-09-23T04:44:23.808392+0000","last_fullsized":"2023-09-23T04:44:23.808392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803020+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:44:23.803020+0000","last_peered":"2023-09-23T04:44:23.803020+0000","last_clean":"2023-09-23T04:44:23.803020+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:44:23.803020+0000","last_undegraded":"2023-09-23T04:44:23.803020+0000","last_fullsized":"2023-09-23T04:44:23.803020+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813256+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:44:23.813256+0000","last_peered":"2023-09-23T04:44:23.813256+0000","last_clean":"2023-09-23T04:44:23.813256+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:44:23.813256+0000","last_undegraded":"2023-09-23T04:44:23.813256+0000","last_fullsized":"2023-09-23T04:44:23.813256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807982+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:44:23.807982+0000","last_peered":"2023-09-23T04:44:23.807982+0000","last_clean":"2023-09-23T04:44:23.807982+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:44:23.807982+0000","last_undegraded":"2023-09-23T04:44:23.807982+0000","last_fullsized":"2023-09-23T04:44:23.807982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811279+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:44:23.811279+0000","last_peered":"2023-09-23T04:44:23.811279+0000","last_clean":"2023-09-23T04:44:23.811279+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:44:23.811279+0000","last_undegraded":"2023-09-23T04:44:23.811279+0000","last_fullsized":"2023-09-23T04:44:23.811279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:03.884387+0000","last_change":"2023-09-23T04:44:24.821431+0000","last_active":"2023-09-23T04:45:03.884387+0000","last_peered":"2023-09-23T04:45:03.884387+0000","last_clean":"2023-09-23T04:45:03.884387+0000","last_became_active":"2023-09-23T04:44:24.812602+0000","last_became_peered":"2023-09-23T04:44:24.812602+0000","last_unstale":"2023-09-23T04:45:03.884387+0000","last_undegraded":"2023-09-23T04:45:03.884387+0000","last_fullsized":"2023-09-23T04:45:03.884387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:48:01.507032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808244+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:44:23.808244+0000","last_peered":"2023-09-23T04:44:23.808244+0000","last_clean":"2023-09-23T04:44:23.808244+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:44:23.808244+0000","last_undegraded":"2023-09-23T04:44:23.808244+0000","last_fullsized":"2023-09-23T04:44:23.808244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811902+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:44:23.811902+0000","last_peered":"2023-09-23T04:44:23.811902+0000","last_clean":"2023-09-23T04:44:23.811902+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:44:23.811902+0000","last_undegraded":"2023-09-23T04:44:23.811902+0000","last_fullsized":"2023-09-23T04:44:23.811902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808442+0000","last_change":"2023-09-23T04:43:47.587791+0000","last_active":"2023-09-23T04:44:23.808442+0000","last_peered":"2023-09-23T04:44:23.808442+0000","last_clean":"2023-09-23T04:44:23.808442+0000","last_became_active":"2023-09-23T04:43:47.587346+0000","last_became_peered":"2023-09-23T04:43:47.587346+0000","last_unstale":"2023-09-23T04:44:23.808442+0000","last_undegraded":"2023-09-23T04:44:23.808442+0000","last_fullsized":"2023-09-23T04:44:23.808442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:13:40.267593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:04.537134+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:45:04.537134+0000","last_peered":"2023-09-23T04:45:04.537134+0000","last_clean":"2023-09-23T04:45:04.537134+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:45:04.537134+0000","last_undegraded":"2023-09-23T04:45:04.537134+0000","last_fullsized":"2023-09-23T04:45:04.537134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807433+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:44:23.807433+0000","last_peered":"2023-09-23T04:44:23.807433+0000","last_clean":"2023-09-23T04:44:23.807433+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:44:23.807433+0000","last_undegraded":"2023-09-23T04:44:23.807433+0000","last_fullsized":"2023-09-23T04:44:23.807433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806809+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:44:23.806809+0000","last_peered":"2023-09-23T04:44:23.806809+0000","last_clean":"2023-09-23T04:44:23.806809+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:44:23.806809+0000","last_undegraded":"2023-09-23T04:44:23.806809+0000","last_fullsized":"2023-09-23T04:44:23.806809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802869+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:44:23.802869+0000","last_peered":"2023-09-23T04:44:23.802869+0000","last_clean":"2023-09-23T04:44:23.802869+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:44:23.802869+0000","last_undegraded":"2023-09-23T04:44:23.802869+0000","last_fullsized":"2023-09-23T04:44:23.802869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805888+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:44:23.805888+0000","last_peered":"2023-09-23T04:44:23.805888+0000","last_clean":"2023-09-23T04:44:23.805888+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:44:23.805888+0000","last_undegraded":"2023-09-23T04:44:23.805888+0000","last_fullsized":"2023-09-23T04:44:23.805888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.809394+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:44:23.809394+0000","last_peered":"2023-09-23T04:44:23.809394+0000","last_clean":"2023-09-23T04:44:23.809394+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:44:23.809394+0000","last_undegraded":"2023-09-23T04:44:23.809394+0000","last_fullsized":"2023-09-23T04:44:23.809394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.810753+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:44:23.810753+0000","last_peered":"2023-09-23T04:44:23.810753+0000","last_clean":"2023-09-23T04:44:23.810753+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:44:23.810753+0000","last_undegraded":"2023-09-23T04:44:23.810753+0000","last_fullsized":"2023-09-23T04:44:23.810753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805984+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:44:23.805984+0000","last_peered":"2023-09-23T04:44:23.805984+0000","last_clean":"2023-09-23T04:44:23.805984+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:44:23.805984+0000","last_undegraded":"2023-09-23T04:44:23.805984+0000","last_fullsized":"2023-09-23T04:44:23.805984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808876+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:44:23.808876+0000","last_peered":"2023-09-23T04:44:23.808876+0000","last_clean":"2023-09-23T04:44:23.808876+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:44:23.808876+0000","last_undegraded":"2023-09-23T04:44:23.808876+0000","last_fullsized":"2023-09-23T04:44:23.808876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803819+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:44:23.803819+0000","last_peered":"2023-09-23T04:44:23.803819+0000","last_clean":"2023-09-23T04:44:23.803819+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:44:23.803819+0000","last_undegraded":"2023-09-23T04:44:23.803819+0000","last_fullsized":"2023-09-23T04:44:23.803819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811394+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:44:23.811394+0000","last_peered":"2023-09-23T04:44:23.811394+0000","last_clean":"2023-09-23T04:44:23.811394+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:44:23.811394+0000","last_undegraded":"2023-09-23T04:44:23.811394+0000","last_fullsized":"2023-09-23T04:44:23.811394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806486+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:44:23.806486+0000","last_peered":"2023-09-23T04:44:23.806486+0000","last_clean":"2023-09-23T04:44:23.806486+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:44:23.806486+0000","last_undegraded":"2023-09-23T04:44:23.806486+0000","last_fullsized":"2023-09-23T04:44:23.806486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:25.060323+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:44:25.060323+0000","last_peered":"2023-09-23T04:44:25.060323+0000","last_clean":"2023-09-23T04:44:25.060323+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:44:25.060323+0000","last_undegraded":"2023-09-23T04:44:25.060323+0000","last_fullsized":"2023-09-23T04:44:25.060323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808062+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:44:23.808062+0000","last_peered":"2023-09-23T04:44:23.808062+0000","last_clean":"2023-09-23T04:44:23.808062+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:44:23.808062+0000","last_undegraded":"2023-09-23T04:44:23.808062+0000","last_fullsized":"2023-09-23T04:44:23.808062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804969+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:44:23.804969+0000","last_peered":"2023-09-23T04:44:23.804969+0000","last_clean":"2023-09-23T04:44:23.804969+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:44:23.804969+0000","last_undegraded":"2023-09-23T04:44:23.804969+0000","last_fullsized":"2023-09-23T04:44:23.804969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803382+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:44:23.803382+0000","last_peered":"2023-09-23T04:44:23.803382+0000","last_clean":"2023-09-23T04:44:23.803382+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:44:23.803382+0000","last_undegraded":"2023-09-23T04:44:23.803382+0000","last_fullsized":"2023-09-23T04:44:23.803382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.805867+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:44:23.805867+0000","last_peered":"2023-09-23T04:44:23.805867+0000","last_clean":"2023-09-23T04:44:23.805867+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:44:23.805867+0000","last_undegraded":"2023-09-23T04:44:23.805867+0000","last_fullsized":"2023-09-23T04:44:23.805867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803159+0000","last_change":"2023-09-23T04:43:51.737574+0000","last_active":"2023-09-23T04:44:23.803159+0000","last_peered":"2023-09-23T04:44:23.803159+0000","last_clean":"2023-09-23T04:44:23.803159+0000","last_became_active":"2023-09-23T04:43:51.736981+0000","last_became_peered":"2023-09-23T04:43:51.736981+0000","last_unstale":"2023-09-23T04:44:23.803159+0000","last_undegraded":"2023-09-23T04:44:23.803159+0000","last_fullsized":"2023-09-23T04:44:23.803159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:59:43.525900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807565+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:44:23.807565+0000","last_peered":"2023-09-23T04:44:23.807565+0000","last_clean":"2023-09-23T04:44:23.807565+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:44:23.807565+0000","last_undegraded":"2023-09-23T04:44:23.807565+0000","last_fullsized":"2023-09-23T04:44:23.807565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802897+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:44:23.802897+0000","last_peered":"2023-09-23T04:44:23.802897+0000","last_clean":"2023-09-23T04:44:23.802897+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:44:23.802897+0000","last_undegraded":"2023-09-23T04:44:23.802897+0000","last_fullsized":"2023-09-23T04:44:23.802897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.813276+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:44:23.813276+0000","last_peered":"2023-09-23T04:44:23.813276+0000","last_clean":"2023-09-23T04:44:23.813276+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:44:23.813276+0000","last_undegraded":"2023-09-23T04:44:23.813276+0000","last_fullsized":"2023-09-23T04:44:23.813276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806648+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:44:23.806648+0000","last_peered":"2023-09-23T04:44:23.806648+0000","last_clean":"2023-09-23T04:44:23.806648+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:44:23.806648+0000","last_undegraded":"2023-09-23T04:44:23.806648+0000","last_fullsized":"2023-09-23T04:44:23.806648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807472+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:44:23.807472+0000","last_peered":"2023-09-23T04:44:23.807472+0000","last_clean":"2023-09-23T04:44:23.807472+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:44:23.807472+0000","last_undegraded":"2023-09-23T04:44:23.807472+0000","last_fullsized":"2023-09-23T04:44:23.807472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.802987+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:44:23.802987+0000","last_peered":"2023-09-23T04:44:23.802987+0000","last_clean":"2023-09-23T04:44:23.802987+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:44:23.802987+0000","last_undegraded":"2023-09-23T04:44:23.802987+0000","last_fullsized":"2023-09-23T04:44:23.802987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804290+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:44:23.804290+0000","last_peered":"2023-09-23T04:44:23.804290+0000","last_clean":"2023-09-23T04:44:23.804290+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:44:23.804290+0000","last_undegraded":"2023-09-23T04:44:23.804290+0000","last_fullsized":"2023-09-23T04:44:23.804290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808053+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:44:23.808053+0000","last_peered":"2023-09-23T04:44:23.808053+0000","last_clean":"2023-09-23T04:44:23.808053+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:44:23.808053+0000","last_undegraded":"2023-09-23T04:44:23.808053+0000","last_fullsized":"2023-09-23T04:44:23.808053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803036+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:44:23.803036+0000","last_peered":"2023-09-23T04:44:23.803036+0000","last_clean":"2023-09-23T04:44:23.803036+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:44:23.803036+0000","last_undegraded":"2023-09-23T04:44:23.803036+0000","last_fullsized":"2023-09-23T04:44:23.803036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.833845+0000","last_change":"2023-09-23T04:44:24.833845+0000","last_active":"2023-09-23T04:44:24.833845+0000","last_peered":"2023-09-23T04:44:24.833845+0000","last_clean":"2023-09-23T04:44:24.833845+0000","last_became_active":"2023-09-23T04:44:24.814306+0000","last_became_peered":"2023-09-23T04:44:24.814306+0000","last_unstale":"2023-09-23T04:44:24.833845+0000","last_undegraded":"2023-09-23T04:44:24.833845+0000","last_fullsized":"2023-09-23T04:44:24.833845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:59.493234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807044+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:44:23.807044+0000","last_peered":"2023-09-23T04:44:23.807044+0000","last_clean":"2023-09-23T04:44:23.807044+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:44:23.807044+0000","last_undegraded":"2023-09-23T04:44:23.807044+0000","last_fullsized":"2023-09-23T04:44:23.807044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807777+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:44:23.807777+0000","last_peered":"2023-09-23T04:44:23.807777+0000","last_clean":"2023-09-23T04:44:23.807777+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:44:23.807777+0000","last_undegraded":"2023-09-23T04:44:23.807777+0000","last_fullsized":"2023-09-23T04:44:23.807777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":74,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:45:06.525051+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:45:06.525051+0000","last_peered":"2023-09-23T04:45:06.525051+0000","last_clean":"2023-09-23T04:45:06.525051+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:45:06.525051+0000","last_undegraded":"2023-09-23T04:45:06.525051+0000","last_fullsized":"2023-09-23T04:45:06.525051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807690+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:44:23.807690+0000","last_peered":"2023-09-23T04:44:23.807690+0000","last_clean":"2023-09-23T04:44:23.807690+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:44:23.807690+0000","last_undegraded":"2023-09-23T04:44:23.807690+0000","last_fullsized":"2023-09-23T04:44:23.807690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807186+0000","last_change":"2023-09-23T04:43:51.741159+0000","last_active":"2023-09-23T04:44:23.807186+0000","last_peered":"2023-09-23T04:44:23.807186+0000","last_clean":"2023-09-23T04:44:23.807186+0000","last_became_active":"2023-09-23T04:43:51.740696+0000","last_became_peered":"2023-09-23T04:43:51.740696+0000","last_unstale":"2023-09-23T04:44:23.807186+0000","last_undegraded":"2023-09-23T04:44:23.807186+0000","last_fullsized":"2023-09-23T04:44:23.807186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:34.832089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808879+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:44:23.808879+0000","last_peered":"2023-09-23T04:44:23.808879+0000","last_clean":"2023-09-23T04:44:23.808879+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:44:23.808879+0000","last_undegraded":"2023-09-23T04:44:23.808879+0000","last_fullsized":"2023-09-23T04:44:23.808879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808173+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:44:23.808173+0000","last_peered":"2023-09-23T04:44:23.808173+0000","last_clean":"2023-09-23T04:44:23.808173+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:44:23.808173+0000","last_undegraded":"2023-09-23T04:44:23.808173+0000","last_fullsized":"2023-09-23T04:44:23.808173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806675+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:44:23.806675+0000","last_peered":"2023-09-23T04:44:23.806675+0000","last_clean":"2023-09-23T04:44:23.806675+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:44:23.806675+0000","last_undegraded":"2023-09-23T04:44:23.806675+0000","last_fullsized":"2023-09-23T04:44:23.806675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811590+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:44:23.811590+0000","last_peered":"2023-09-23T04:44:23.811590+0000","last_clean":"2023-09-23T04:44:23.811590+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:44:23.811590+0000","last_undegraded":"2023-09-23T04:44:23.811590+0000","last_fullsized":"2023-09-23T04:44:23.811590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808552+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:44:23.808552+0000","last_peered":"2023-09-23T04:44:23.808552+0000","last_clean":"2023-09-23T04:44:23.808552+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:44:23.808552+0000","last_undegraded":"2023-09-23T04:44:23.808552+0000","last_fullsized":"2023-09-23T04:44:23.808552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807578+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:44:23.807578+0000","last_peered":"2023-09-23T04:44:23.807578+0000","last_clean":"2023-09-23T04:44:23.807578+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:44:23.807578+0000","last_undegraded":"2023-09-23T04:44:23.807578+0000","last_fullsized":"2023-09-23T04:44:23.807578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804735+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:44:23.804735+0000","last_peered":"2023-09-23T04:44:23.804735+0000","last_clean":"2023-09-23T04:44:23.804735+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:44:23.804735+0000","last_undegraded":"2023-09-23T04:44:23.804735+0000","last_fullsized":"2023-09-23T04:44:23.804735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.804408+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:44:23.804408+0000","last_peered":"2023-09-23T04:44:23.804408+0000","last_clean":"2023-09-23T04:44:23.804408+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:44:23.804408+0000","last_undegraded":"2023-09-23T04:44:23.804408+0000","last_fullsized":"2023-09-23T04:44:23.804408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.807418+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:44:23.807418+0000","last_peered":"2023-09-23T04:44:23.807418+0000","last_clean":"2023-09-23T04:44:23.807418+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:44:23.807418+0000","last_undegraded":"2023-09-23T04:44:23.807418+0000","last_fullsized":"2023-09-23T04:44:23.807418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811488+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:44:23.811488+0000","last_peered":"2023-09-23T04:44:23.811488+0000","last_clean":"2023-09-23T04:44:23.811488+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:44:23.811488+0000","last_undegraded":"2023-09-23T04:44:23.811488+0000","last_fullsized":"2023-09-23T04:44:23.811488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808209+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:44:23.808209+0000","last_peered":"2023-09-23T04:44:23.808209+0000","last_clean":"2023-09-23T04:44:23.808209+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:44:23.808209+0000","last_undegraded":"2023-09-23T04:44:23.808209+0000","last_fullsized":"2023-09-23T04:44:23.808209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.803958+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:44:23.803958+0000","last_peered":"2023-09-23T04:44:23.803958+0000","last_clean":"2023-09-23T04:44:23.803958+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:44:23.803958+0000","last_undegraded":"2023-09-23T04:44:23.803958+0000","last_fullsized":"2023-09-23T04:44:23.803958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.812717+0000","last_change":"2023-09-23T04:44:24.812717+0000","last_active":"2023-09-23T04:44:24.812717+0000","last_peered":"2023-09-23T04:44:24.812717+0000","last_clean":"2023-09-23T04:44:24.812717+0000","last_became_active":"2023-09-23T04:44:24.812307+0000","last_became_peered":"2023-09-23T04:44:24.812307+0000","last_unstale":"2023-09-23T04:44:24.812717+0000","last_undegraded":"2023-09-23T04:44:24.812717+0000","last_fullsized":"2023-09-23T04:44:24.812717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:43:34.537044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-23T04:44:24.999803+0000","last_change":"2023-09-23T04:44:24.999461+0000","last_active":"2023-09-23T04:44:24.999803+0000","last_peered":"2023-09-23T04:44:24.999803+0000","last_clean":"2023-09-23T04:44:24.999803+0000","last_became_active":"2023-09-23T04:44:24.999082+0000","last_became_peered":"2023-09-23T04:44:24.999082+0000","last_unstale":"2023-09-23T04:44:24.999803+0000","last_undegraded":"2023-09-23T04:44:24.999803+0000","last_fullsized":"2023-09-23T04:44:24.999803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:20.840200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.806736+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:44:23.806736+0000","last_peered":"2023-09-23T04:44:23.806736+0000","last_clean":"2023-09-23T04:44:23.806736+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:44:23.806736+0000","last_undegraded":"2023-09-23T04:44:23.806736+0000","last_fullsized":"2023-09-23T04:44:23.806736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808628+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:44:23.808628+0000","last_peered":"2023-09-23T04:44:23.808628+0000","last_clean":"2023-09-23T04:44:23.808628+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:44:23.808628+0000","last_undegraded":"2023-09-23T04:44:23.808628+0000","last_fullsized":"2023-09-23T04:44:23.808628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.811182+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:44:23.811182+0000","last_peered":"2023-09-23T04:44:23.811182+0000","last_clean":"2023-09-23T04:44:23.811182+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:44:23.811182+0000","last_undegraded":"2023-09-23T04:44:23.811182+0000","last_fullsized":"2023-09-23T04:44:23.811182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-23T04:44:23.808770+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:44:23.808770+0000","last_peered":"2023-09-23T04:44:23.808770+0000","last_clean":"2023-09-23T04:44:23.808770+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:44:23.808770+0000","last_undegraded":"2023-09-23T04:44:23.808770+0000","last_fullsized":"2023-09-23T04:44:23.808770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":502,"num_read_kb":328,"num_write":238,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449117,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":489038,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.71899999999999997,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.55300000000000005}]},{"osd":2,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":0.748},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92500000000000004,"5min":1.034,"15min":1.034},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.238,"15min":1.238},"last":0.59999999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80400000000000005,"5min":1.125,"15min":1.125},"last":0.433},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.80300000000000005,"15min":0.73099999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":10.571999999999999,"5min":10.571999999999999,"15min":10.571999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":10.398999999999999,"5min":10.398999999999999,"15min":10.398999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.92700000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48999999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":0.88900000000000001,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":9.734,"5min":9.734,"15min":9.734},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.278,"5min":0.89000000000000001,"15min":0.82599999999999996},"min":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":10.288,"5min":10.288,"15min":10.288},"last":0.57599999999999996}]},{"osd":10,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98599999999999999,"5min":1.272,"15min":1.272},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.71,"15min":1.71},"last":0.75900000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":1136075,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44800000000000001,"15min":0.45000000000000001},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.433}]},{"osd":2,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.81799999999999995,"15min":0.79800000000000004},"min":{"1min":0.45000000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.573,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.234,"5min":0.75600000000000001,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.0960000000000001,"5min":8.0960000000000001,"15min":8.0960000000000001},"last":0.52200000000000002}]},{"osd":4,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81899999999999995,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.443,"5min":0.433,"15min":0.433},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.76500000000000001,"15min":0.72699999999999998},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":5.5289999999999999,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.77900000000000003,"15min":0.76200000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.748}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91300000000000003,"5min":0.995,"15min":0.995},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96599999999999997,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72299999999999998}]},{"osd":9,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95699999999999996},"min":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.121,"5min":1.0169999999999999,"15min":1},"min":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":5.3760000000000003,"5min":5.3760000000000003,"15min":5.3760000000000003},"last":0.66100000000000003}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82699999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":492454,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.64100000000000001,"15min":0.60599999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4669999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.75,"15min":0.71399999999999997},"min":{"1min":0.42699999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.6310000000000002,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.749,"15min":0.72999999999999998},"min":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.751}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.53900000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":0.85699999999999998,"15min":0.76300000000000001},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":13.364000000000001,"5min":13.364000000000001,"15min":13.364000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.2030000000000001,"5min":0.82299999999999995,"15min":0.75900000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.83399999999999996,"15min":0.81999999999999995},"min":{"1min":0.61199999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.3340000000000001,"5min":3.3340000000000001,"15min":3.3340000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86099999999999999,"15min":0.84799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.84999999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.90500000000000003,"15min":0.88700000000000001},"min":{"1min":0.626,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.91500000000000004,"15min":0.91300000000000003},"min":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.68700000000000006}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.628}]},{"osd":7,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.96699999999999997,"15min":0.96999999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0469999999999999}]},{"osd":8,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.069,"5min":1.04,"15min":1.0349999999999999},"min":{"1min":0.65200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.89300000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.997,"15min":0.99099999999999999},"min":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.8330000000000002,"5min":3.8330000000000002,"15min":3.8330000000000002},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":1},"min":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.99299999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.91300000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":488714,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73499999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47199999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.59399999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.68400000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.57099999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.68100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.85999999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.284,"15min":1.284},"last":0.749}]},{"osd":9,"last update":"Sat Sep 23 04:44:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.748,"15min":0.748},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.874}]},{"osd":10,"last update":"Sat Sep 23 04:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.78500000000000003,"15min":0.76500000000000001},"min":{"1min":0.57499999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.84499999999999997}]},{"osd":11,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.89500000000000002}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1084761,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55900000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45100000000000001,"15min":0.442},"min":{"1min":0.29499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81100000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.626}]},{"osd":3,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72999999999999998}]},{"osd":4,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.30199999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.68700000000000006,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65600000000000003,"15min":0.66800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81200000000000006,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.65200000000000002},"min":{"1min":0.47799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746}]},{"osd":6,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98199999999999998,"5min":1,"15min":1},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.72399999999999998,"15min":0.73299999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97799999999999998,"5min":1.036,"15min":1.036},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89200000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82099999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.746,"15min":0.745},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.004,"5min":1.125,"15min":1.125},"last":0.79600000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.51200000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95799999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67000000000000004}]},{"osd":11,"last update":"Sat Sep 23 04:44:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86299999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483862,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.59899999999999998,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.2750000000000004,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.57999999999999996,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.748,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.2480000000000002,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.9460000000000002,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.79900000000000004,"15min":0.76400000000000001},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":4.3819999999999997,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.627},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.41099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73499999999999999,"15min":0.72399999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.88100000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.83099999999999996,"15min":0.80300000000000005},"min":{"1min":0.54200000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.60399999999999998}]},{"osd":7,"last update":"Sat Sep 23 04:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.81899999999999995,"15min":0.77700000000000002},"min":{"1min":0.68899999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.7829999999999999,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.3029999999999999,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.94199999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.998,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.056}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.109,"15min":0.98799999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":15.34,"5min":15.34,"15min":15.34},"last":1.014},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.96499999999999997,"15min":0.93700000000000006},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.99099999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82899999999999996,"15min":0.82299999999999995},"min":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.016,"15min":1.012},"min":{"1min":0.81000000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":1.1000000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":6.8470000000000004,"5min":6.8470000000000004,"15min":6.8470000000000004},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":8.3249999999999993,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":1.036}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":545060,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.56699999999999995},"min":{"1min":0.377,"5min":0.31,"15min":0.31},"max":{"1min":0.72999999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54400000000000004,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88500000000000001,"5min":1.042,"15min":1.042},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67700000000000005}]},{"osd":3,"last update":"Sat Sep 23 04:44:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91000000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.77100000000000002}]},{"osd":4,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.30499999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.32500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.47699999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.64000000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":1.0069999999999999,"5min":1.087,"15min":1.087},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69699999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61199999999999999,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.97299999999999998,"5min":1.014,"15min":1.014},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.55400000000000005,"15min":0.52600000000000002},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86299999999999999,"5min":0.873,"15min":0.873},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.621,"15min":0.58799999999999997},"min":{"1min":0.434,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96299999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:45:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.755},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72299999999999998,"15min":0.68200000000000005},"min":{"1min":0.621,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79500000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98899999999999999,"5min":0.997,"15min":0.997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.81299999999999994,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0669999999999999,"5min":1.103,"15min":1.103},"last":0.88500000000000001}]},{"osd":10,"last update":"Sat Sep 23 04:44:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0840000000000001,"5min":1.232,"15min":1.232},"last":0.90500000000000003}]},{"osd":11,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.85099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":485318,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59799999999999998,"15min":0.56399999999999995},"min":{"1min":0.434,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80000000000000004,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.629,"15min":0.57099999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96799999999999997,"5min":1.345,"15min":1.345},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.66600000000000004,"15min":0.59199999999999997},"min":{"1min":0.51700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92100000000000004,"5min":1.306,"15min":1.306},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.626,"15min":0.59299999999999997},"min":{"1min":0.54000000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66800000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:44:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Sep 23 04:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67100000000000004,"15min":0.61799999999999999},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88300000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.628,"15min":0.56699999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97799999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.877,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.753,"15min":0.755},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93400000000000005,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.85399999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.96099999999999997},"min":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1060000000000001,"5min":1.871,"15min":1.871},"last":0.69799999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.92400000000000004,"15min":0.93799999999999994},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.147,"5min":1.153,"15min":1.153},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89200000000000002,"15min":0.89100000000000001},"min":{"1min":0.61899999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2070000000000001,"5min":1.3,"15min":1.3},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84999999999999998,"15min":0.84899999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0069999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.88300000000000001,"15min":0.872},"min":{"1min":0.76800000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.069}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545831,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.86499999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.747},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.41399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.83199999999999996}]},{"osd":3,"last update":"Sat Sep 23 04:44:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60599999999999998}]},{"osd":4,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.92000000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.46300000000000002,"15min":0.441},"min":{"1min":0.29999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.47099999999999997},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68700000000000006}]},{"osd":6,"last update":"Sat Sep 23 04:44:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.94599999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.76600000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.73299999999999998}]},{"osd":9,"last update":"Sat Sep 23 04:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71599999999999997,"15min":0.70899999999999996},"min":{"1min":0.53100000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.90600000000000003}]},{"osd":10,"last update":"Sat Sep 23 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85899999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:44:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.94199999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483239,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.434,"15min":0.39300000000000002},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.62}]},{"osd":2,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64100000000000001,"15min":0.59099999999999997},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.52200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:44:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86599999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67600000000000005,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56200000000000006}]},{"osd":5,"last update":"Sat Sep 23 04:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.52300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95799999999999996,"5min":0.999,"15min":0.999},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Sep 23 04:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":1.214,"15min":1.214},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.48499999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.77400000000000002,"15min":0.76000000000000001},"min":{"1min":0.57999999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.089,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.69799999999999995,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60399999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.90200000000000002,"15min":0.90600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.747},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.89800000000000002,"15min":0.91400000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.58399999999999996}]},{"osd":10,"last update":"Sat Sep 23 04:44:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.73999999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.0700000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:44:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.81799999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":488438,"data_compressed":17696,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.37,"15min":0.34200000000000003},"min":{"1min":0.31,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.56100000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.371},"min":{"1min":0.317,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61099999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.36399999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64000000000000001,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.874,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 23 04:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58999999999999997,"15min":0.53900000000000003},"min":{"1min":0.41699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.63400000000000001,"15min":0.57199999999999995},"min":{"1min":0.49199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89700000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Sep 23 04:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.498,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.746},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61699999999999999},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79000000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.59199999999999997},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68999999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80000000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81599999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Sat Sep 23 04:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76000000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:44:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1080814,"data_compressed":53953,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.51500000000000001},"min":{"1min":0.46800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91300000000000003,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.624,"5min":1.246,"15min":1.246},"last":0.55000000000000004}]},{"osd":1,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.50700000000000001},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":1.115,"15min":1.115},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.071,"15min":1.071},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39700000000000002,"15min":0.371},"min":{"1min":0.27300000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42699999999999999,"15min":0.40300000000000002},"min":{"1min":0.33600000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79900000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.40100000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.753,"5min":1.212,"15min":1.212},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57099999999999995,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.68999999999999995,"5min":1.042,"15min":1.042},"last":0.94299999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.48199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":6,"last update":"Sat Sep 23 04:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.874,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76100000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82099999999999995,"5min":1.292,"15min":1.292},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.65800000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.84899999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.625},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95599999999999996,"5min":1.105,"15min":1.105},"last":0.5},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85499999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.58899999999999997}]},{"osd":9,"last update":"Sat Sep 23 04:44:23 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89100000000000001,"5min":1.262,"15min":1.262},"last":1.325},{"interface":"front","average":{"1min":0.75,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.99099999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.282}]},{"osd":10,"last update":"Sat Sep 23 04:44:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.67700000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92800000000000005,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88700000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:45:11.544 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-23T04:45:11.545 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-23T04:45:11.545 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-23T04:45:11.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph health --format=json 2023-09-23T04:45:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:12.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:12 smithi005 ceph-mon[97638]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:12.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:12 smithi005 ceph-mon[97638]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:12 smithi118 ceph-mon[134297]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:13.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:12 smithi118 ceph-mon[134297]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:12 smithi069 ceph-mon[133931]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:45:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:12 smithi069 ceph-mon[133931]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.805+0000 7f77c8b73700 1 -- 172.21.15.5:0/3402061778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 msgr2=0x7f77c410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.805+0000 7f77c8b73700 1 --2- 172.21.15.5:0/3402061778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c410dd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00b0a0 tx=0x7f77bc00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 -- 172.21.15.5:0/3402061778 shutdown_connections 2023-09-23T04:45:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 --2- 172.21.15.5:0/3402061778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c410dd50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 --2- 172.21.15.5:0/3402061778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 0x7f77c4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 --2- 172.21.15.5:0/3402061778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 -- 172.21.15.5:0/3402061778 >> 172.21.15.5:0/3402061778 conn(0x7f77c40fc910 msgr2=0x7f77c40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 -- 172.21.15.5:0/3402061778 shutdown_connections 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.806+0000 7f77c8b73700 1 -- 172.21.15.5:0/3402061778 wait complete. 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 Processor -- start 2023-09-23T04:45:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 -- start start 2023-09-23T04:45:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 0x7f77c4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419e200 con 0x7f77c4106180 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419e370 con 0x7f77c41010a0 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.807+0000 7f77c8b73700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c419e530 con 0x7f77c4101a50 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c1d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c1d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59248/0 (socket says 172.21.15.5:59248) 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c1d9b700 1 -- 172.21.15.5:0/1869763162 learned_addr learned my addr 172.21.15.5:0/1869763162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c259c700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 0x7f77c4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 -- 172.21.15.5:0/1869763162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 msgr2=0x7f77c41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 -- 172.21.15.5:0/1869763162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 msgr2=0x7f77c4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 0x7f77c4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 -- 172.21.15.5:0/1869763162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc009d00 con 0x7f77c4106180 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c1d9b700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.808+0000 7f77c2d9d700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c419dbf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00b0a0 tx=0x7f77bc007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.809+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00e040 con 0x7f77c4106180 2023-09-23T04:45:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.809+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77bc01d470 con 0x7f77c4106180 2023-09-23T04:45:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.809+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c419e7b0 con 0x7f77c4106180 2023-09-23T04:45:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.809+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0052a0 con 0x7f77c4106180 2023-09-23T04:45:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.809+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c41a6070 con 0x7f77c4106180 2023-09-23T04:45:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.810+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c404eb20 con 0x7f77c4106180 2023-09-23T04:45:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.812+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f77bc01d5e0 con 0x7f77c4106180 2023-09-23T04:45:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.813+0000 7f77b37fe700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 0x7f77ac0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.813+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77bc0ce860 con 0x7f77c4106180 2023-09-23T04:45:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.813+0000 7f77c259c700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 0x7f77ac0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.814+0000 7f77c259c700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 0x7f77ac0a3b70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009c60 tx=0x7f77b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:13.817+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f77bc097a10 con 0x7f77c4106180 2023-09-23T04:45:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.073+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f77c4110dc0 con 0x7f77c4106180 2023-09-23T04:45:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.073+0000 7f77b37fe700 1 -- 172.21.15.5:0/1869763162 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f77bc09b030 con 0x7f77c4106180 2023-09-23T04:45:14.167 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:14.167 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 msgr2=0x7f77ac0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 0x7f77ac0a3b70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009c60 tx=0x7f77b4009260 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 msgr2=0x7f77c419dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c419dbf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00b0a0 tx=0x7f77bc007990 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 shutdown_connections 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f77ac0a16b0 0x7f77ac0a3b70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c4106180 0x7f77c419dbf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77c41010a0 0x7f77c4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 --2- 172.21.15.5:0/1869763162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77c4101a50 0x7f77c41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 >> 172.21.15.5:0/1869763162 conn(0x7f77c40fc910 msgr2=0x7f77c40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 shutdown_connections 2023-09-23T04:45:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.076+0000 7f77c8b73700 1 -- 172.21.15.5:0/1869763162 wait complete. 2023-09-23T04:45:14.579 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-23T04:45:14.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 -- 172.21.15.5:0/814660803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 msgr2=0x7f7eac10b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/814660803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac10b540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009fc0 tx=0x7f7ea400b330 comp rx=0 tx=0).stop 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 -- 172.21.15.5:0/814660803 shutdown_connections 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/814660803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac10b540 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/814660803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 0x7f7eac103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/814660803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 -- 172.21.15.5:0/814660803 >> 172.21.15.5:0/814660803 conn(0x7f7eac0fe6b0 msgr2=0x7f7eac100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:14.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 -- 172.21.15.5:0/814660803 shutdown_connections 2023-09-23T04:45:14.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.793+0000 7f7eb3bff700 1 -- 172.21.15.5:0/814660803 wait complete. 2023-09-23T04:45:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 Processor -- start 2023-09-23T04:45:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 -- start start 2023-09-23T04:45:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 0x7f7eac198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac1a0a80 con 0x7f7eac1042f0 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac1a0bc0 con 0x7f7eac104c90 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.794+0000 7f7eb3bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac1a0d50 con 0x7f7eac103100 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb199b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 0x7f7eac198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49632/0 (socket says 172.21.15.5:49632) 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 -- 172.21.15.5:0/1664029613 learned_addr learned my addr 172.21.15.5:0/1664029613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb219c700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 -- 172.21.15.5:0/1664029613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 msgr2=0x7f7eac198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 0x7f7eac198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 -- 172.21.15.5:0/1664029613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 msgr2=0x7f7eac1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 -- 172.21.15.5:0/1664029613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4009c40 con 0x7f7eac1042f0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb219c700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.795+0000 7f7eb119a700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9c0 tx=0x7f7e9c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.796+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c004a30 con 0x7f7eac1042f0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.796+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac1a0fd0 con 0x7f7eac1042f0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.796+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e9c00cc80 con 0x7f7eac1042f0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.796+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0054e0 con 0x7f7eac1042f0 2023-09-23T04:45:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.796+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac1a14b0 con 0x7f7eac1042f0 2023-09-23T04:45:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.797+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac10a650 con 0x7f7eac1042f0 2023-09-23T04:45:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.799+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f7e9c004b90 con 0x7f7eac1042f0 2023-09-23T04:45:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.800+0000 7f7ea2ffd700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 0x7f7e980a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.800+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7e9c014070 con 0x7f7eac1042f0 2023-09-23T04:45:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.801+0000 7f7eb199b700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 0x7f7e980a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.801+0000 7f7eb199b700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 0x7f7e980a3af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7eac104150 tx=0x7f7ea8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:14.804+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7e9c0965c0 con 0x7f7eac1042f0 2023-09-23T04:45:14.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:14 smithi005 ceph-mon[97638]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:14.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:14 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1869763162' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T04:45:15.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.017+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eac10ebc0 con 0x7f7eac1042f0 2023-09-23T04:45:15.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.019+0000 7f7ea2ffd700 1 -- 172.21.15.5:0/1664029613 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24393 (secure 0 0 0) 0x7f7e9c099be0 con 0x7f7eac1042f0 2023-09-23T04:45:15.019 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:15.019 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:44:24.800265+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.021+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 msgr2=0x7f7e980a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.021+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 0x7f7e980a3af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7eac104150 tx=0x7f7ea8009260 comp rx=0 tx=0).stop 2023-09-23T04:45:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.021+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 msgr2=0x7f7eac199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.021+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e9c0 tx=0x7f7e9c00ecd0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 shutdown_connections 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7e980a1630 0x7f7e980a3af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eac104c90 0x7f7eac1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7eac103100 0x7f7eac198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 --2- 172.21.15.5:0/1664029613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eac1042f0 0x7f7eac199440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 >> 172.21.15.5:0/1664029613 conn(0x7f7eac0fe6b0 msgr2=0x7f7eac107790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 shutdown_connections 2023-09-23T04:45:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.022+0000 7f7eb3bff700 1 -- 172.21.15.5:0/1664029613 wait complete. 2023-09-23T04:45:15.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:15.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:14 smithi118 ceph-mon[134297]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:14 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1869763162' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T04:45:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:14 smithi069 ceph-mon[133931]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:14 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1869763162' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-23T04:45:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 -- 172.21.15.5:0/4056994131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8073880 msgr2=0x7f3aa8073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 --2- 172.21.15.5:0/4056994131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8073880 0x7f3aa8073d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009990 tx=0x7f3a90009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 -- 172.21.15.5:0/4056994131 shutdown_connections 2023-09-23T04:45:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 --2- 172.21.15.5:0/4056994131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8108dc0 0x7f3aa810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 --2- 172.21.15.5:0/4056994131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8074e60 0x7f3aa80732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 --2- 172.21.15.5:0/4056994131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8073880 0x7f3aa8073d00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 -- 172.21.15.5:0/4056994131 >> 172.21.15.5:0/4056994131 conn(0x7f3aa80fc470 msgr2=0x7f3aa80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 -- 172.21.15.5:0/4056994131 shutdown_connections 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.250+0000 7f3aaddff700 1 -- 172.21.15.5:0/4056994131 wait complete. 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.251+0000 7f3aaddff700 1 Processor -- start 2023-09-23T04:45:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 -- start start 2023-09-23T04:45:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 0x7f3aa8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8074e60 0x7f3aa81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81a09e0 con 0x7f3aa8108dc0 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81a0b20 con 0x7f3aa8073880 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aaddff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81a0cb0 con 0x7f3aa8074e60 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aa77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 0x7f3aa8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.252+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49652/0 (socket says 172.21.15.5:49652) 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa7fff700 1 -- 172.21.15.5:0/2643118308 learned_addr learned my addr 172.21.15.5:0/2643118308 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3a9ffff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8074e60 0x7f3aa81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 -- 172.21.15.5:0/2643118308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8074e60 msgr2=0x7f3aa81993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8074e60 0x7f3aa81993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 -- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 msgr2=0x7f3aa81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 -- 172.21.15.5:0/2643118308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900095f0 con 0x7f3aa8073880 2023-09-23T04:45:15.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa7fff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa77fe700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 0x7f3aa8198e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e9f0 tx=0x7f3a9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.253+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800c9c0 con 0x7f3aa8073880 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.254+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa81a0f90 con 0x7f3aa8073880 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.254+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a98010430 con 0x7f3aa8073880 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.254+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98018680 con 0x7f3aa8073880 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.254+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa81a13b0 con 0x7f3aa8073880 2023-09-23T04:45:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.255+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa8107470 con 0x7f3aa8073880 2023-09-23T04:45:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.257+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f3a98010860 con 0x7f3aa8073880 2023-09-23T04:45:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.258+0000 7f3aa57fa700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 0x7f3a940a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.258+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a98014070 con 0x7f3aa8073880 2023-09-23T04:45:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.258+0000 7f3a9ffff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 0x7f3a940a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.259+0000 7f3a9ffff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 0x7f3a940a3af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000b2e0 tx=0x7f3a90005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.260+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3a980967d0 con 0x7f3aa8073880 2023-09-23T04:45:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.473+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3aa810ebd0 con 0x7f3aa8073880 2023-09-23T04:45:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.474+0000 7f3aa57fa700 1 -- 172.21.15.5:0/2643118308 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a98099df0 con 0x7f3aa8073880 2023-09-23T04:45:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 msgr2=0x7f3a940a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 0x7f3a940a3af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000b2e0 tx=0x7f3a90005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 msgr2=0x7f3aa8198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 0x7f3aa8198e60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e9f0 tx=0x7f3a9800ed00 comp rx=0 tx=0).stop 2023-09-23T04:45:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 shutdown_connections 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3a940a1630 0x7f3a940a3af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3aa8108dc0 0x7f3aa81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3aa8073880 0x7f3aa8198e60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 --2- 172.21.15.5:0/2643118308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3aa8074e60 0x7f3aa81993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 >> 172.21.15.5:0/2643118308 conn(0x7f3aa80fc470 msgr2=0x7f3aa80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 shutdown_connections 2023-09-23T04:45:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.476+0000 7f3aaddff700 1 -- 172.21.15.5:0/2643118308 wait complete. 2023-09-23T04:45:15.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1811136483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0102a50 msgr2=0x7f8fa0109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1811136483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0102a50 0x7f8fa0109370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800b0a0 tx=0x7f8f9800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1811136483 shutdown_connections 2023-09-23T04:45:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1811136483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0102a50 0x7f8fa0109370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1811136483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0100eb0 0x7f8fa01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1811136483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 0x7f8fa0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1811136483 >> 172.21.15.5:0/1811136483 conn(0x7f8fa00fc470 msgr2=0x7f8fa00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1811136483 shutdown_connections 2023-09-23T04:45:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.705+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1811136483 wait complete. 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.706+0000 7f8fa75ff700 1 Processor -- start 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.706+0000 7f8fa75ff700 1 -- start start 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.706+0000 7f8fa75ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa75ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 0x7f8fa0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa75ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0102a50 0x7f8fa01a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa75ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01a0a90 con 0x7f8fa0100eb0 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa75ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01a0bd0 con 0x7f8fa0102090 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa75ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01a0d60 con 0x7f8fa0102a50 2023-09-23T04:45:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49684/0 (socket says 172.21.15.5:49684) 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 -- 172.21.15.5:0/1046161830 learned_addr learned my addr 172.21.15.5:0/1046161830 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa5b9c700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0102a50 0x7f8fa01a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa4b9a700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 0x7f8fa0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 -- 172.21.15.5:0/1046161830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0102a50 msgr2=0x7f8fa01a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0102a50 0x7f8fa01a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 -- 172.21.15.5:0/1046161830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 msgr2=0x7f8fa0199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 0x7f8fa0199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.707+0000 7f8fa539b700 1 -- 172.21.15.5:0/1046161830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98009d00 con 0x7f8fa0100eb0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8fa539b700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e970 tx=0x7f8f9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c009790 con 0x7f8fa0100eb0 2023-09-23T04:45:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9c00c7e0 con 0x7f8fa0100eb0 2023-09-23T04:45:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa01a0fe0 con 0x7f8fa0100eb0 2023-09-23T04:45:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c003c70 con 0x7f8fa0100eb0 2023-09-23T04:45:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.708+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa01a1460 con 0x7f8fa0100eb0 2023-09-23T04:45:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.709+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa010c770 con 0x7f8fa0100eb0 2023-09-23T04:45:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.711+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f8f9c00cc50 con 0x7f8fa0100eb0 2023-09-23T04:45:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.712+0000 7f8f967fc700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 0x7f8f8c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.713+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8f9c014070 con 0x7f8fa0100eb0 2023-09-23T04:45:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.713+0000 7f8fa4b9a700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 0x7f8f8c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:15.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.713+0000 7f8fa4b9a700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 0x7f8f8c0a3af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0100bd0 tx=0x7f8f90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:15.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.716+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8f9c0966c0 con 0x7f8fa0100eb0 2023-09-23T04:45:15.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.928+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fa01a16b0 con 0x7f8fa0100eb0 2023-09-23T04:45:15.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.929+0000 7f8f967fc700 1 -- 172.21.15.5:0/1046161830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f9c003dd0 con 0x7f8fa0100eb0 2023-09-23T04:45:15.930 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:15.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 msgr2=0x7f8f8c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 0x7f8f8c0a3af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0100bd0 tx=0x7f8f90009260 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 msgr2=0x7f8fa0198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00e970 tx=0x7f8f9c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 shutdown_connections 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.931+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f8f8c0a1630 0x7f8f8c0a3af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fa0102a50 0x7f8fa01a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fa0100eb0 0x7f8fa0198f10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 --2- 172.21.15.5:0/1046161830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fa0102090 0x7f8fa0199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 >> 172.21.15.5:0/1046161830 conn(0x7f8fa00fc470 msgr2=0x7f8fa00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 shutdown_connections 2023-09-23T04:45:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:15.932+0000 7f8fa75ff700 1 -- 172.21.15.5:0/1046161830 wait complete. 2023-09-23T04:45:15.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:45:16.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1664029613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:16.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:15 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2643118308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1705664623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec102a50 msgr2=0x7fbbec109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1705664623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec102a50 0x7fbbec109370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400b0a0 tx=0x7fbbe400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1705664623 shutdown_connections 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1705664623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec102a50 0x7fbbec109370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1705664623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec100eb0 0x7fbbec1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.157+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1705664623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 0x7fbbec102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.158+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1705664623 >> 172.21.15.5:0/1705664623 conn(0x7fbbec0fc470 msgr2=0x7fbbec0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.158+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1705664623 shutdown_connections 2023-09-23T04:45:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.158+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1705664623 wait complete. 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 Processor -- start 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 -- start start 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 0x7fbbec073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec102a50 0x7fbbec073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec073ef0 con 0x7fbbec100eb0 2023-09-23T04:45:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec074060 con 0x7fbbec102a50 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf3fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec1a0d30 con 0x7fbbec102090 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49694/0 (socket says 172.21.15.5:49694) 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf1d9b700 1 -- 172.21.15.5:0/1614411459 learned_addr learned my addr 172.21.15.5:0/1614411459 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf259c700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec102a50 0x7fbbec073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.159+0000 7fbbf159a700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 0x7fbbec073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 -- 172.21.15.5:0/1614411459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 msgr2=0x7fbbec073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 0x7fbbec073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 -- 172.21.15.5:0/1614411459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec102a50 msgr2=0x7fbbec073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec102a50 0x7fbbec073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 -- 172.21.15.5:0/1614411459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009d00 con 0x7fbbec100eb0 2023-09-23T04:45:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf1d9b700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800e970 tx=0x7fbbe800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8009790 con 0x7fbbec100eb0 2023-09-23T04:45:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe800c7e0 con 0x7fbbec100eb0 2023-09-23T04:45:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec1a0fb0 con 0x7fbbec100eb0 2023-09-23T04:45:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8003c70 con 0x7fbbec100eb0 2023-09-23T04:45:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.160+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec1a1430 con 0x7fbbec100eb0 2023-09-23T04:45:16.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:15 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1664029613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:16.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:15 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2643118308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.161+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec107e10 con 0x7fbbec100eb0 2023-09-23T04:45:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.163+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fbbe800cc50 con 0x7fbbec100eb0 2023-09-23T04:45:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.165+0000 7fbbe2ffd700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 0x7fbbd80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.165+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbbe8014070 con 0x7fbbec100eb0 2023-09-23T04:45:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.165+0000 7fbbf159a700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 0x7fbbd80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.165+0000 7fbbf159a700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 0x7fbbd80a3b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbec100bd0 tx=0x7fbbdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.168+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbbe8096680 con 0x7fbbec100eb0 2023-09-23T04:45:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:15 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1664029613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:15 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2643118308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.380+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbec10ca20 con 0x7fbbec100eb0 2023-09-23T04:45:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.380+0000 7fbbe2ffd700 1 -- 172.21.15.5:0/1614411459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbbe8003dd0 con 0x7fbbec100eb0 2023-09-23T04:45:16.380 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 msgr2=0x7fbbd80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 0x7fbbd80a3b70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbec100bd0 tx=0x7fbbdc009260 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 msgr2=0x7fbbec074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800e970 tx=0x7fbbe800ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 shutdown_connections 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbbd80a16b0 0x7fbbd80a3b70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbec102a50 0x7fbbec073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbec100eb0 0x7fbbec074bd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1614411459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbec102090 0x7fbbec073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 >> 172.21.15.5:0/1614411459 conn(0x7fbbec0fc470 msgr2=0x7fbbec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 shutdown_connections 2023-09-23T04:45:16.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.383+0000 7fbbf3fff700 1 -- 172.21.15.5:0/1614411459 wait complete. 2023-09-23T04:45:16.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:45:16.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 -- 172.21.15.5:0/348991523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102060 msgr2=0x7fab381024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 --2- 172.21.15.5:0/348991523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102060 0x7fab381024e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab28009990 tx=0x7fab28009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 -- 172.21.15.5:0/348991523 shutdown_connections 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 --2- 172.21.15.5:0/348991523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102a20 0x7fab38109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 --2- 172.21.15.5:0/348991523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 --2- 172.21.15.5:0/348991523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102060 0x7fab381024e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.608+0000 7fab403ff700 1 -- 172.21.15.5:0/348991523 >> 172.21.15.5:0/348991523 conn(0x7fab380fc480 msgr2=0x7fab380fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.609+0000 7fab403ff700 1 -- 172.21.15.5:0/348991523 shutdown_connections 2023-09-23T04:45:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.609+0000 7fab403ff700 1 -- 172.21.15.5:0/348991523 wait complete. 2023-09-23T04:45:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.609+0000 7fab403ff700 1 Processor -- start 2023-09-23T04:45:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 -- start start 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 0x7fab38199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102a20 0x7fab381a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381a0960 con 0x7fab38102a20 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381a0aa0 con 0x7fab38100f10 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab403ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab381a0c30 con 0x7fab38102060 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab3e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab3e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39600/0 (socket says 172.21.15.5:39600) 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab3e19b700 1 -- 172.21.15.5:0/766396470 learned_addr learned my addr 172.21.15.5:0/766396470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab3e99c700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102a20 0x7fab381a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.610+0000 7fab3d99a700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 0x7fab38199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 -- 172.21.15.5:0/766396470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 msgr2=0x7fab38199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 0x7fab38199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 -- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102a20 msgr2=0x7fab381a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102a20 0x7fab381a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 -- 172.21.15.5:0/766396470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280095f0 con 0x7fab38100f10 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3d99a700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 0x7fab38199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab3e19b700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab3400d980 tx=0x7fab3400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34004220 con 0x7fab38100f10 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab34007640 con 0x7fab38100f10 2023-09-23T04:45:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab381a0f10 con 0x7fab38100f10 2023-09-23T04:45:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34005610 con 0x7fab38100f10 2023-09-23T04:45:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.611+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381a1330 con 0x7fab38100f10 2023-09-23T04:45:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.612+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3810c760 con 0x7fab38100f10 2023-09-23T04:45:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.614+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fab34044830 con 0x7fab38100f10 2023-09-23T04:45:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.616+0000 7fab2f7fe700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 0x7fab240a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.616+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fab340cd500 con 0x7fab38100f10 2023-09-23T04:45:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.616+0000 7fab3d99a700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 0x7fab240a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.617+0000 7fab3d99a700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 0x7fab240a3af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab28005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.618+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fab34096620 con 0x7fab38100f10 2023-09-23T04:45:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.830+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fab381a1520 con 0x7fab38100f10 2023-09-23T04:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.831+0000 7fab2f7fe700 1 -- 172.21.15.5:0/766396470 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fab34016020 con 0x7fab38100f10 2023-09-23T04:45:16.831 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 msgr2=0x7fab240a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 0x7fab240a3af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab28005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 msgr2=0x7fab38198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab3400d980 tx=0x7fab3400dc90 comp rx=0 tx=0).stop 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 shutdown_connections 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fab240a1630 0x7fab240a3af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab38102a20 0x7fab381a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab38100f10 0x7fab38198de0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 --2- 172.21.15.5:0/766396470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab38102060 0x7fab38199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 >> 172.21.15.5:0/766396470 conn(0x7fab380fc480 msgr2=0x7fab380fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 shutdown_connections 2023-09-23T04:45:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:16.834+0000 7fab403ff700 1 -- 172.21.15.5:0/766396470 wait complete. 2023-09-23T04:45:16.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:45:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.057+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/2701741646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 msgr2=0x7f2d84102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.057+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/2701741646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d84102500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098f0 tx=0x7f2d74009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/2701741646 shutdown_connections 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/2701741646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84102a40 0x7f2d84109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/2701741646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84100ea0 0x7f2d841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/2701741646 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d84102500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/2701741646 >> 172.21.15.5:0/2701741646 conn(0x7f2d840fc480 msgr2=0x7f2d840fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/2701741646 shutdown_connections 2023-09-23T04:45:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.058+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/2701741646 wait complete. 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 Processor -- start 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 -- start start 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d841993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84102a40 0x7f2d841a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841a0a10 con 0x7f2d84100ea0 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841a0b50 con 0x7f2d84102a40 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8c5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841a0ce0 con 0x7f2d84102080 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49726/0 (socket says 172.21.15.5:49726) 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8a39b700 1 -- 172.21.15.5:0/4027159726 learned_addr learned my addr 172.21.15.5:0/4027159726 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d89b9a700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d841993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.059+0000 7f2d8ab9c700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84102a40 0x7f2d841a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 -- 172.21.15.5:0/4027159726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 msgr2=0x7f2d841993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d841993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 -- 172.21.15.5:0/4027159726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84102a40 msgr2=0x7f2d841a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84102a40 0x7f2d841a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 -- 172.21.15.5:0/4027159726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800095f0 con 0x7f2d84100ea0 2023-09-23T04:45:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8a39b700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000ebf0 tx=0x7f2d8000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80009a10 con 0x7f2d84100ea0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8000cc80 con 0x7f2d84100ea0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c009c40 con 0x7f2d84100ea0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80018710 con 0x7f2d84100ea0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.060+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d74009520 con 0x7f2d84100ea0 2023-09-23T04:45:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.061+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d84193550 con 0x7f2d84100ea0 2023-09-23T04:45:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.063+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f2d80010430 con 0x7f2d84100ea0 2023-09-23T04:45:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.065+0000 7f2d7b7fe700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 0x7f2d700a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.066+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d80014070 con 0x7f2d84100ea0 2023-09-23T04:45:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.066+0000 7f2d89b9a700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 0x7f2d700a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.066+0000 7f2d89b9a700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 0x7f2d700a3af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009fd0 tx=0x7f2d74019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.070+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2d80096990 con 0x7f2d84100ea0 2023-09-23T04:45:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:16 smithi005 ceph-mon[97638]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:16 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1046161830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:16 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1614411459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:16 smithi118 ceph-mon[134297]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:16 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1046161830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:16 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1614411459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:16 smithi069 ceph-mon[133931]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:16 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1046161830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:16 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1614411459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.328+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d8410c9f0 con 0x7f2d84100ea0 2023-09-23T04:45:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.329+0000 7f2d7b7fe700 1 -- 172.21.15.5:0/4027159726 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f2d80099fb0 con 0x7f2d84100ea0 2023-09-23T04:45:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:17.329 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 msgr2=0x7f2d700a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 0x7f2d700a3af0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009fd0 tx=0x7f2d74019040 comp rx=0 tx=0).stop 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 msgr2=0x7f2d84198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000ebf0 tx=0x7f2d8000ef00 comp rx=0 tx=0).stop 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 shutdown_connections 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f2d700a1630 0x7f2d700a3af0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d84102a40 0x7f2d841a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d84100ea0 0x7f2d84198e90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 --2- 172.21.15.5:0/4027159726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2d84102080 0x7f2d841993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 >> 172.21.15.5:0/4027159726 conn(0x7f2d840fc480 msgr2=0x7f2d84107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 shutdown_connections 2023-09-23T04:45:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.331+0000 7f2d8c5ff700 1 -- 172.21.15.5:0/4027159726 wait complete. 2023-09-23T04:45:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:17.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 -- 172.21.15.5:0/2420806263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102070 msgr2=0x7f43cc1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 --2- 172.21.15.5:0/2420806263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102070 0x7f43cc1024f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0098f0 tx=0x7f43bc009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 -- 172.21.15.5:0/2420806263 shutdown_connections 2023-09-23T04:45:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 --2- 172.21.15.5:0/2420806263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102a30 0x7f43cc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 --2- 172.21.15.5:0/2420806263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 --2- 172.21.15.5:0/2420806263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102070 0x7f43cc1024f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 -- 172.21.15.5:0/2420806263 >> 172.21.15.5:0/2420806263 conn(0x7f43cc0fc450 msgr2=0x7f43cc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 -- 172.21.15.5:0/2420806263 shutdown_connections 2023-09-23T04:45:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.553+0000 7f43d37ff700 1 -- 172.21.15.5:0/2420806263 wait complete. 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.554+0000 7f43d37ff700 1 Processor -- start 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.554+0000 7f43d37ff700 1 -- start start 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.554+0000 7f43d37ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.554+0000 7f43d37ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.554+0000 7f43d37ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102a30 0x7f43cc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d37ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc073ef0 con 0x7f43cc102070 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d37ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc074060 con 0x7f43cc100e90 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d37ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc1a0c80 con 0x7f43cc102a30 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49744/0 (socket says 172.21.15.5:49744) 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 -- 172.21.15.5:0/78666817 learned_addr learned my addr 172.21.15.5:0/78666817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d1d9c700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102a30 0x7f43cc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d159b700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 -- 172.21.15.5:0/78666817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102a30 msgr2=0x7f43cc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102a30 0x7f43cc073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 -- 172.21.15.5:0/78666817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 msgr2=0x7f43cc074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 -- 172.21.15.5:0/78666817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009d00 con 0x7f43cc102070 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d159b700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.555+0000 7f43d0d9a700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.556+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc01c070 con 0x7f43cc102070 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.556+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43bc00f7d0 con 0x7f43cc102070 2023-09-23T04:45:17.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.556+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc009520 con 0x7f43cc102070 2023-09-23T04:45:17.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.556+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003ae0 con 0x7f43cc102070 2023-09-23T04:45:17.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.556+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc1a1190 con 0x7f43cc102070 2023-09-23T04:45:17.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.557+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc107ef0 con 0x7f43cc102070 2023-09-23T04:45:17.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.559+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f43bc020030 con 0x7f43cc102070 2023-09-23T04:45:17.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.560+0000 7f43c27fc700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 0x7f43b80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:17.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.561+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43bc0ce400 con 0x7f43cc102070 2023-09-23T04:45:17.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.561+0000 7f43d159b700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 0x7f43b80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.561+0000 7f43d159b700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 0x7f43b80a3af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43cc101be0 tx=0x7f43c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.565+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f43bc097610 con 0x7f43cc102070 2023-09-23T04:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.778+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43cc10c940 con 0x7f43cc102070 2023-09-23T04:45:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.780+0000 7f43c27fc700 1 -- 172.21.15.5:0/78666817 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24393 (secure 0 0 0) 0x7f43bc09ac30 con 0x7f43cc102070 2023-09-23T04:45:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:17.782 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:44:24.800265+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 msgr2=0x7f43b80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 0x7f43b80a3af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43cc101be0 tx=0x7f43c8009260 comp rx=0 tx=0).stop 2023-09-23T04:45:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 msgr2=0x7f43cc073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f43bc000c00 tx=0x7f43bc00b790 comp rx=0 tx=0).stop 2023-09-23T04:45:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 shutdown_connections 2023-09-23T04:45:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43b80a1630 0x7f43b80a3af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f43cc102a30 0x7f43cc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f43cc100e90 0x7f43cc074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 --2- 172.21.15.5:0/78666817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43cc102070 0x7f43cc073210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 >> 172.21.15.5:0/78666817 conn(0x7f43cc0fc450 msgr2=0x7f43cc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 shutdown_connections 2023-09-23T04:45:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:17.782+0000 7f43d37ff700 1 -- 172.21.15.5:0/78666817 wait complete. 2023-09-23T04:45:17.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:17.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 -- 172.21.15.5:0/1423874630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 msgr2=0x7f836810b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 --2- 172.21.15.5:0/1423874630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f836810b260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f836000b0a0 tx=0x7f836000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 -- 172.21.15.5:0/1423874630 shutdown_connections 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 --2- 172.21.15.5:0/1423874630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f836810b260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 --2- 172.21.15.5:0/1423874630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368074e60 0x7f83680732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 --2- 172.21.15.5:0/1423874630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368073880 0x7f8368073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 -- 172.21.15.5:0/1423874630 >> 172.21.15.5:0/1423874630 conn(0x7f83680fc480 msgr2=0x7f83680fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 -- 172.21.15.5:0/1423874630 shutdown_connections 2023-09-23T04:45:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.009+0000 7f836cdff700 1 -- 172.21.15.5:0/1423874630 wait complete. 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.010+0000 7f836cdff700 1 Processor -- start 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.010+0000 7f836cdff700 1 -- start start 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368073880 0x7f8368198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368074e60 0x7f83681993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681a09e0 con 0x7f8368108dd0 2023-09-23T04:45:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681a0b20 con 0x7f8368074e60 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836cdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681a0cb0 con 0x7f8368073880 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49776/0 (socket says 172.21.15.5:49776) 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f836659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368073880 0x7f8368198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 -- 172.21.15.5:0/499083565 learned_addr learned my addr 172.21.15.5:0/499083565 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8365d9b700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368074e60 0x7f83681993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 -- 172.21.15.5:0/499083565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368073880 msgr2=0x7f8368198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368073880 0x7f8368198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 -- 172.21.15.5:0/499083565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368074e60 msgr2=0x7f83681993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368074e60 0x7f83681993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.011+0000 7f8366d9d700 1 -- 172.21.15.5:0/499083565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009d00 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f8366d9d700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f836000b780 tx=0x7f836000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000e040 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8360004c50 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83681a0f30 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360005280 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.012+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83681a1350 con 0x7f8368108dd0 2023-09-23T04:45:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.013+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8368107b90 con 0x7f8368108dd0 2023-09-23T04:45:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.015+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f836001d470 con 0x7f8368108dd0 2023-09-23T04:45:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.017+0000 7f83577fe700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 0x7f83500a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.018+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83600ce930 con 0x7f8368108dd0 2023-09-23T04:45:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.018+0000 7f836659c700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 0x7f83500a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.018+0000 7f836659c700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 0x7f83500a3af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c60 tx=0x7f8358009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.022+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8360097ae0 con 0x7f8368108dd0 2023-09-23T04:45:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:17 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/766396470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:17 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4027159726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:17 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/766396470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:17 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4027159726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:18.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.234+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f836810eb20 con 0x7f8368108dd0 2023-09-23T04:45:18.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.235+0000 7f83577fe700 1 -- 172.21.15.5:0/499083565 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f836009b100 con 0x7f8368108dd0 2023-09-23T04:45:18.235 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 msgr2=0x7f83500a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 0x7f83500a3af0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8358009c60 tx=0x7f8358009260 comp rx=0 tx=0).stop 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 msgr2=0x7f83681a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f836000b780 tx=0x7f836000bab0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 shutdown_connections 2023-09-23T04:45:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83500a1630 0x7f83500a3af0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8368108dd0 0x7f83681a0410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8368073880 0x7f8368198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 --2- 172.21.15.5:0/499083565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8368074e60 0x7f83681993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 >> 172.21.15.5:0/499083565 conn(0x7f83680fc480 msgr2=0x7f8368104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 shutdown_connections 2023-09-23T04:45:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.237+0000 7f836cdff700 1 -- 172.21.15.5:0/499083565 wait complete. 2023-09-23T04:45:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:17 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/766396470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:17 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4027159726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:18.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 -- 172.21.15.5:0/2900874989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100eb0 msgr2=0x7fcedc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 --2- 172.21.15.5:0/2900874989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100eb0 0x7fcedc1012d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009990 tx=0x7fcecc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 -- 172.21.15.5:0/2900874989 shutdown_connections 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 --2- 172.21.15.5:0/2900874989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102a50 0x7fcedc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 --2- 172.21.15.5:0/2900874989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100eb0 0x7fcedc1012d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 --2- 172.21.15.5:0/2900874989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc102090 0x7fcedc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 -- 172.21.15.5:0/2900874989 >> 172.21.15.5:0/2900874989 conn(0x7fcedc0fc470 msgr2=0x7fcedc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 -- 172.21.15.5:0/2900874989 shutdown_connections 2023-09-23T04:45:18.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.463+0000 7fcee21ff700 1 -- 172.21.15.5:0/2900874989 wait complete. 2023-09-23T04:45:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.464+0000 7fcee21ff700 1 Processor -- start 2023-09-23T04:45:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.464+0000 7fcee21ff700 1 -- start start 2023-09-23T04:45:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc100eb0 0x7fcedc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102090 0x7fcedc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1a0a30 con 0x7fcedc102a50 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1a0b70 con 0x7fcedc102090 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcee21ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1a0d00 con 0x7fcedc100eb0 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49790/0 (socket says 172.21.15.5:49790) 2023-09-23T04:45:18.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 -- 172.21.15.5:0/1333959548 learned_addr learned my addr 172.21.15.5:0/1333959548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedaffd700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102090 0x7fcedc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc100eb0 0x7fcedc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 -- 172.21.15.5:0/1333959548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc100eb0 msgr2=0x7fcedc198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc100eb0 0x7fcedc198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 -- 172.21.15.5:0/1333959548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102090 msgr2=0x7fcedc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102090 0x7fcedc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.465+0000 7fcedbfff700 1 -- 172.21.15.5:0/1333959548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0095f0 con 0x7fcedc102a50 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fcedbfff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fced400ef30 tx=0x7fced400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4018070 con 0x7fcedc102a50 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced400f040 con 0x7fcedc102a50 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc1a0f80 con 0x7fcedc102a50 2023-09-23T04:45:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced400cd00 con 0x7fcedc102a50 2023-09-23T04:45:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.466+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1a1400 con 0x7fcedc102a50 2023-09-23T04:45:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.467+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc193560 con 0x7fcedc102a50 2023-09-23T04:45:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.469+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fced4003a00 con 0x7fcedc102a50 2023-09-23T04:45:18.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.471+0000 7fced8ff9700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 0x7fcec40a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.472+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fced40cce60 con 0x7fcedc102a50 2023-09-23T04:45:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.472+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 0x7fcec40a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.472+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 0x7fcec40a3b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00b2e0 tx=0x7fcecc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.476+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fced4096010 con 0x7fcedc102a50 2023-09-23T04:45:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.691+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcedc10c990 con 0x7fcedc102a50 2023-09-23T04:45:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.691+0000 7fced8ff9700 1 -- 172.21.15.5:0/1333959548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fced4099630 con 0x7fcedc102a50 2023-09-23T04:45:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 msgr2=0x7fcec40a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 0x7fcec40a3b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00b2e0 tx=0x7fcecc005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 msgr2=0x7fcedc1a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fced400ef30 tx=0x7fced400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 shutdown_connections 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a16b0 0x7fcec40a3b70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc102a50 0x7fcedc1a0460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcedc100eb0 0x7fcedc198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 --2- 172.21.15.5:0/1333959548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcedc102090 0x7fcedc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 >> 172.21.15.5:0/1333959548 conn(0x7fcedc0fc470 msgr2=0x7fcedc107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:18.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 shutdown_connections 2023-09-23T04:45:18.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.694+0000 7fcee21ff700 1 -- 172.21.15.5:0/1333959548 wait complete. 2023-09-23T04:45:18.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:45:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/2199996763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 msgr2=0x7fb3f4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/2199996763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4102510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009990 tx=0x7fb3e4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/2199996763 shutdown_connections 2023-09-23T04:45:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/2199996763 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 0x7fb3f4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/2199996763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/2199996763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4102510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/2199996763 >> 172.21.15.5:0/2199996763 conn(0x7fb3f40fc470 msgr2=0x7fb3f40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/2199996763 shutdown_connections 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.920+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/2199996763 wait complete. 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.921+0000 7fb3fc1ff700 1 Processor -- start 2023-09-23T04:45:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.921+0000 7fb3fc1ff700 1 -- start start 2023-09-23T04:45:18.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 0x7fb3f41a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41a0980 con 0x7fb3f4102090 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41a0ac0 con 0x7fb3f4100eb0 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fc1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f41a0c50 con 0x7fb3f4102a50 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49800/0 (socket says 172.21.15.5:49800) 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 -- 172.21.15.5:0/3581635919 learned_addr learned my addr 172.21.15.5:0/3581635919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f9f9b700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3fa79c700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 0x7fb3f41a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 -- 172.21.15.5:0/3581635919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 msgr2=0x7fb3f41a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 0x7fb3f41a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 -- 172.21.15.5:0/3581635919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 msgr2=0x7fb3f4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f979a700 1 -- 172.21.15.5:0/3581635919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40095f0 con 0x7fb3f4102090 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.922+0000 7fb3f9f9b700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3f979a700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b2e0 tx=0x7fb3e4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e401d070 con 0x7fb3f4102090 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e40047e0 con 0x7fb3f4102090 2023-09-23T04:45:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4005550 con 0x7fb3f4102090 2023-09-23T04:45:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f41a0ed0 con 0x7fb3f4102090 2023-09-23T04:45:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.923+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f41a1240 con 0x7fb3f4102090 2023-09-23T04:45:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.924+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f410c770 con 0x7fb3f4102090 2023-09-23T04:45:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.926+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fb3e4004950 con 0x7fb3f4102090 2023-09-23T04:45:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.927+0000 7fb3eaffd700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 0x7fb3e00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.928+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb3e40ceac0 con 0x7fb3f4102090 2023-09-23T04:45:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.928+0000 7fb3f9f9b700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 0x7fb3e00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.928+0000 7fb3f9f9b700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 0x7fb3e00a3b70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4101c00 tx=0x7fb3f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:18.930+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb3e4097c70 con 0x7fb3f4102090 2023-09-23T04:45:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.144+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3f41a1490 con 0x7fb3f4102090 2023-09-23T04:45:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.144+0000 7fb3eaffd700 1 -- 172.21.15.5:0/3581635919 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3e409b290 con 0x7fb3f4102090 2023-09-23T04:45:19.144 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:45:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.146+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 msgr2=0x7fb3e00a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.146+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 0x7fb3e00a3b70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4101c00 tx=0x7fb3f0009260 comp rx=0 tx=0).stop 2023-09-23T04:45:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 msgr2=0x7fb3f4199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b2e0 tx=0x7fb3e4005e80 comp rx=0 tx=0).stop 2023-09-23T04:45:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 shutdown_connections 2023-09-23T04:45:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb3e00a16b0 0x7fb3e00a3b70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3f4102a50 0x7fb3f41a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3f4100eb0 0x7fb3f4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 --2- 172.21.15.5:0/3581635919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f4102090 0x7fb3f4199340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 >> 172.21.15.5:0/3581635919 conn(0x7fb3f40fc470 msgr2=0x7fb3f40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 shutdown_connections 2023-09-23T04:45:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.147+0000 7fb3fc1ff700 1 -- 172.21.15.5:0/3581635919 wait complete. 2023-09-23T04:45:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:18 smithi005 ceph-mon[97638]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/78666817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/499083565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1333959548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:18 smithi118 ceph-mon[134297]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/78666817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/499083565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1333959548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:45:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:18 smithi069 ceph-mon[133931]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/78666817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/499083565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1333959548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.380+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2027544580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102090 msgr2=0x7f1e28102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.380+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2027544580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102090 0x7f1e28102510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009990 tx=0x7f1e14009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2027544580 shutdown_connections 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2027544580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28102a50 0x7f1e28109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2027544580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28100eb0 0x7f1e281012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2027544580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102090 0x7f1e28102510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2027544580 >> 172.21.15.5:0/2027544580 conn(0x7f1e280fc470 msgr2=0x7f1e280fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2027544580 shutdown_connections 2023-09-23T04:45:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.381+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2027544580 wait complete. 2023-09-23T04:45:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 Processor -- start 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 -- start start 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 0x7f1e281971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102a50 0x7f1e2819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2819e820 con 0x7f1e28100eb0 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2819e990 con 0x7f1e28102090 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2f1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2819eb50 con 0x7f1e28102a50 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.382+0000 7f1e2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49814/0 (socket says 172.21.15.5:49814) 2023-09-23T04:45:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 -- 172.21.15.5:0/2186815279 learned_addr learned my addr 172.21.15.5:0/2186815279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2d79c700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102a50 0x7f1e2819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e27fff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 0x7f1e281971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 -- 172.21.15.5:0/2186815279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102a50 msgr2=0x7f1e2819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102a50 0x7f1e2819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 -- 172.21.15.5:0/2186815279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 msgr2=0x7f1e281971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 0x7f1e281971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 -- 172.21.15.5:0/2186815279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140095f0 con 0x7f1e28100eb0 2023-09-23T04:45:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e27fff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 0x7f1e281971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2cf9b700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00e970 tx=0x7f1e1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c009790 con 0x7f1e28100eb0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2819ee30 con 0x7f1e28100eb0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1c0044e0 con 0x7f1e28100eb0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.383+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003e30 con 0x7f1e28100eb0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.384+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2819f260 con 0x7f1e28100eb0 2023-09-23T04:45:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.385+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e281084c0 con 0x7f1e28100eb0 2023-09-23T04:45:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.386+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f1e1c004000 con 0x7f1e28100eb0 2023-09-23T04:45:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.388+0000 7f1e25ffb700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 0x7f1e180a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.388+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e1c014070 con 0x7f1e28100eb0 2023-09-23T04:45:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.388+0000 7f1e27fff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 0x7f1e180a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.389+0000 7f1e27fff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 0x7f1e180a3af0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e281981b0 tx=0x7f1e14005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.392+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1e1c096810 con 0x7f1e28100eb0 2023-09-23T04:45:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.606+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e2810c9e0 con 0x7f1e28100eb0 2023-09-23T04:45:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.606+0000 7f1e25ffb700 1 -- 172.21.15.5:0/2186815279 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1e1c099e30 con 0x7f1e28100eb0 2023-09-23T04:45:19.606 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.608+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 msgr2=0x7f1e180a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.608+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 0x7f1e180a3af0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e281981b0 tx=0x7f1e14005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.608+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 msgr2=0x7f1e28196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.608+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00e970 tx=0x7f1e1c00ed30 comp rx=0 tx=0).stop 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 shutdown_connections 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f1e180a1630 0x7f1e180a3af0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e28102a50 0x7f1e2819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e28100eb0 0x7f1e28196cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 --2- 172.21.15.5:0/2186815279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e28102090 0x7f1e281971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 >> 172.21.15.5:0/2186815279 conn(0x7f1e280fc470 msgr2=0x7f1e28105600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 shutdown_connections 2023-09-23T04:45:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.609+0000 7f1e2f1ff700 1 -- 172.21.15.5:0/2186815279 wait complete. 2023-09-23T04:45:19.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 -- 172.21.15.5:0/290382297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 msgr2=0x7fee281066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/290382297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281066a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009990 tx=0x7fee1c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 -- 172.21.15.5:0/290382297 shutdown_connections 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/290382297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28106be0 0x7fee28109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/290382297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28101980 0x7fee28103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/290382297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281066a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 -- 172.21.15.5:0/290382297 >> 172.21.15.5:0/290382297 conn(0x7fee280fb350 msgr2=0x7fee280fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:19.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 -- 172.21.15.5:0/290382297 shutdown_connections 2023-09-23T04:45:19.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.836+0000 7fee2d1ff700 1 -- 172.21.15.5:0/290382297 wait complete. 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 Processor -- start 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 -- start start 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 0x7fee28198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a08f0 con 0x7fee281042b0 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a0a60 con 0x7fee28101980 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.837+0000 7fee2d1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281a0c20 con 0x7fee28106be0 2023-09-23T04:45:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49822/0 (socket says 172.21.15.5:49822) 2023-09-23T04:45:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59460/0 (socket says 172.21.15.5:59460) 2023-09-23T04:45:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 -- 172.21.15.5:0/1134656515 learned_addr learned my addr 172.21.15.5:0/1134656515 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee26d9d700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 0x7fee28198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 -- 172.21.15.5:0/1134656515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 msgr2=0x7fee281a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 -- 172.21.15.5:0/1134656515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 msgr2=0x7fee28198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 0x7fee28198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 -- 172.21.15.5:0/1134656515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0095f0 con 0x7fee281042b0 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee26d9d700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 0x7fee28198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2759e700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.838+0000 7fee2659c700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee1c000c00 tx=0x7fee1c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.839+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c01d070 con 0x7fee281042b0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.839+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1c00fb90 con 0x7fee281042b0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.839+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0173f0 con 0x7fee281042b0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.839+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee281a0ea0 con 0x7fee281042b0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.839+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a1260 con 0x7fee281042b0 2023-09-23T04:45:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.840+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2810c6f0 con 0x7fee281042b0 2023-09-23T04:45:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.843+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7fee1c021410 con 0x7fee281042b0 2023-09-23T04:45:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.844+0000 7fee0ffff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 0x7fee100a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.845+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee1c0ced80 con 0x7fee281042b0 2023-09-23T04:45:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.845+0000 7fee26d9d700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 0x7fee100a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.845+0000 7fee26d9d700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 0x7fee100a3b70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee2819a280 tx=0x7fee18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:19.847+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fee1c097f30 con 0x7fee281042b0 2023-09-23T04:45:19.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:19 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3581635919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:19.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:19 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2186815279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.104+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee281a1450 con 0x7fee281042b0 2023-09-23T04:45:20.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.105+0000 7fee0ffff700 1 -- 172.21.15.5:0/1134656515 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7fee1c026090 con 0x7fee281042b0 2023-09-23T04:45:20.105 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:20.105 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 msgr2=0x7fee100a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 0x7fee100a3b70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee2819a280 tx=0x7fee18009260 comp rx=0 tx=0).stop 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 msgr2=0x7fee281992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee1c000c00 tx=0x7fee1c0057a0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 shutdown_connections 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fee100a16b0 0x7fee100a3b70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee28106be0 0x7fee281a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee28101980 0x7fee28198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 --2- 172.21.15.5:0/1134656515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee281042b0 0x7fee281992f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.107+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 >> 172.21.15.5:0/1134656515 conn(0x7fee280fb350 msgr2=0x7fee28102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:20.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.108+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 shutdown_connections 2023-09-23T04:45:20.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.108+0000 7fee2d1ff700 1 -- 172.21.15.5:0/1134656515 wait complete. 2023-09-23T04:45:20.109 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:20.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:19 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3581635919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:19 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2186815279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3581635919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:20.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2186815279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 -- 172.21.15.5:0/68407135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c102a30 msgr2=0x7f611c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 --2- 172.21.15.5:0/68407135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c102a30 0x7f611c109350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6114009f90 tx=0x7f611400b330 comp rx=0 tx=0).stop 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 -- 172.21.15.5:0/68407135 shutdown_connections 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 --2- 172.21.15.5:0/68407135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c102a30 0x7f611c109350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 --2- 172.21.15.5:0/68407135 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c100e90 0x7f611c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 --2- 172.21.15.5:0/68407135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 -- 172.21.15.5:0/68407135 >> 172.21.15.5:0/68407135 conn(0x7f611c0fc450 msgr2=0x7f611c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 -- 172.21.15.5:0/68407135 shutdown_connections 2023-09-23T04:45:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.329+0000 7f61219ff700 1 -- 172.21.15.5:0/68407135 wait complete. 2023-09-23T04:45:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.330+0000 7f61219ff700 1 Processor -- start 2023-09-23T04:45:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.330+0000 7f61219ff700 1 -- start start 2023-09-23T04:45:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 0x7f611c1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c1a0b00 con 0x7f611c100e90 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c1a0c40 con 0x7f611c102070 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f61219ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c1a0dd0 con 0x7f611c102a30 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49840/0 (socket says 172.21.15.5:49840) 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 -- 172.21.15.5:0/2722358235 learned_addr learned my addr 172.21.15.5:0/2722358235 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611b7fe700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 0x7f611c1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611a7fc700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 -- 172.21.15.5:0/2722358235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 msgr2=0x7f611c1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 0x7f611c1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 -- 172.21.15.5:0/2722358235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 msgr2=0x7f611c1994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611affd700 1 -- 172.21.15.5:0/2722358235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009c40 con 0x7f611c100e90 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611b7fe700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 0x7f611c1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.331+0000 7f611a7fc700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f611affd700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f610c00e9f0 tx=0x7f610c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c00cbd0 con 0x7f611c100e90 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f610c0044e0 con 0x7f611c100e90 2023-09-23T04:45:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c1a10b0 con 0x7f611c100e90 2023-09-23T04:45:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c003d90 con 0x7f611c100e90 2023-09-23T04:45:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.332+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c1a1610 con 0x7f611c100e90 2023-09-23T04:45:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.333+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c193540 con 0x7f611c100e90 2023-09-23T04:45:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.335+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f610c003f10 con 0x7f611c100e90 2023-09-23T04:45:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.336+0000 7f61209fd700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 0x7f61040a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.337+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f610c014070 con 0x7f611c100e90 2023-09-23T04:45:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.337+0000 7f611a7fc700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 0x7f61040a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.338+0000 7f611a7fc700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 0x7f61040a3b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f611c100bb0 tx=0x7f611000b570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:20.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.339+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f610c096a80 con 0x7f611c100e90 2023-09-23T04:45:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.552+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f611c10c750 con 0x7f611c100e90 2023-09-23T04:45:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.553+0000 7f61209fd700 1 -- 172.21.15.5:0/2722358235 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24393 (secure 0 0 0) 0x7f610c09a0a0 con 0x7f611c100e90 2023-09-23T04:45:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:20.554 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:44:24.800265+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 msgr2=0x7f61040a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 0x7f61040a3b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f611c100bb0 tx=0x7f611000b570 comp rx=0 tx=0).stop 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 msgr2=0x7f611c198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f610c00e9f0 tx=0x7f610c00edb0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 shutdown_connections 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61040a16b0 0x7f61040a3b70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f611c102a30 0x7f611c1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f611c100e90 0x7f611c198f80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 --2- 172.21.15.5:0/2722358235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f611c102070 0x7f611c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 >> 172.21.15.5:0/2722358235 conn(0x7f611c0fc450 msgr2=0x7f611c107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 shutdown_connections 2023-09-23T04:45:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.556+0000 7f61219ff700 1 -- 172.21.15.5:0/2722358235 wait complete. 2023-09-23T04:45:20.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:45:20.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 -- 172.21.15.5:0/2977928058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc101950 msgr2=0x7f25fc103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 --2- 172.21.15.5:0/2977928058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc101950 0x7f25fc103d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009990 tx=0x7f25f8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 -- 172.21.15.5:0/2977928058 shutdown_connections 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 --2- 172.21.15.5:0/2977928058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc106bb0 0x7f25fc109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 --2- 172.21.15.5:0/2977928058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc101950 0x7f25fc103d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 --2- 172.21.15.5:0/2977928058 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc104280 0x7f25fc106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 -- 172.21.15.5:0/2977928058 >> 172.21.15.5:0/2977928058 conn(0x7f25fc0fb360 msgr2=0x7f25fc0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 -- 172.21.15.5:0/2977928058 shutdown_connections 2023-09-23T04:45:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.780+0000 7f26041ff700 1 -- 172.21.15.5:0/2977928058 wait complete. 2023-09-23T04:45:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.781+0000 7f26041ff700 1 Processor -- start 2023-09-23T04:45:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.781+0000 7f26041ff700 1 -- start start 2023-09-23T04:45:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 0x7f25fc1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc106bb0 0x7f25fc0ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1005e0 con 0x7f25fc104280 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc100750 con 0x7f25fc106bb0 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f26041ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a0bf0 con 0x7f25fc101950 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49856/0 (socket says 172.21.15.5:49856) 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f2601f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 0x7f25fc1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 -- 172.21.15.5:0/917106637 learned_addr learned my addr 172.21.15.5:0/917106637 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260279c700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc106bb0 0x7f25fc0ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 -- 172.21.15.5:0/917106637 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 msgr2=0x7f25fc1012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 0x7f25fc1012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 -- 172.21.15.5:0/917106637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc106bb0 msgr2=0x7f25fc0ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc106bb0 0x7f25fc0ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f260179a700 1 -- 172.21.15.5:0/917106637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80095f0 con 0x7f25fc104280 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.782+0000 7f2601f9b700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 0x7f25fc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f260179a700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9a0 tx=0x7f25ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec009790 con 0x7f25fc104280 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec010430 con 0x7f25fc104280 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc1a0ed0 con 0x7f25fc104280 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec018680 con 0x7f25fc104280 2023-09-23T04:45:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.783+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc1a1350 con 0x7f25fc104280 2023-09-23T04:45:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.784+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25fc10c700 con 0x7f25fc104280 2023-09-23T04:45:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.786+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f25ec0105a0 con 0x7f25fc104280 2023-09-23T04:45:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.787+0000 7f25f2ffd700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 0x7f25e80a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.788+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f25ec014070 con 0x7f25fc104280 2023-09-23T04:45:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.788+0000 7f2601f9b700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 0x7f25e80a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.788+0000 7f2601f9b700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 0x7f25e80a3af0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25f800b2e0 tx=0x7f25f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:20.792+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f25ec051ec0 con 0x7f25fc104280 2023-09-23T04:45:20.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:20 smithi005 ceph-mon[97638]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:20.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1134656515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:20.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2722358235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.048+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25fc1a1570 con 0x7f25fc104280 2023-09-23T04:45:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.049+0000 7f25f2ffd700 1 -- 172.21.15.5:0/917106637 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f25ec0966d0 con 0x7f25fc104280 2023-09-23T04:45:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:21.049 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 msgr2=0x7f25e80a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 0x7f25e80a3af0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25f800b2e0 tx=0x7f25f8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 msgr2=0x7f25fc0ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9a0 tx=0x7f25ec00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 shutdown_connections 2023-09-23T04:45:21.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f25e80a1630 0x7f25e80a3af0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25fc106bb0 0x7f25fc0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f25fc101950 0x7f25fc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 --2- 172.21.15.5:0/917106637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25fc104280 0x7f25fc0ff900 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 >> 172.21.15.5:0/917106637 conn(0x7f25fc0fb360 msgr2=0x7f25fc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 shutdown_connections 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.051+0000 7f26041ff700 1 -- 172.21.15.5:0/917106637 wait complete. 2023-09-23T04:45:21.052 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:21.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-23T04:43:47.148877+0000', 'modified': '2023-09-23T04:44:20.444513+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24577, 'mds_1': 24559, 'mds_2': 24554, 'mds_3': 24535, 'mds_4': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14538': {'gid': 14538, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/1890134322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 1890134322}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 1890134322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6833/827181928', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6832', 'nonce': 827181928}, {'type': 'v1', 'addr': '172.21.15.69:6833', 'nonce': 827181928}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.118:6833/3707604246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6832', 'nonce': 3707604246}, {'type': 'v1', 'addr': '172.21.15.118:6833', 'nonce': 3707604246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6835/3966429650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3966429650}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 3966429650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6837/2958173168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2958173168}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2958173168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.118:6835/3772048959', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6834', 'nonce': 3772048959}, {'type': 'v1', 'addr': '172.21.15.118:6835', 'nonce': 3772048959}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.118:6837/704534117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6836', 'nonce': 704534117}, {'type': 'v1', 'addr': '172.21.15.118:6837', 'nonce': 704534117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.69:6839/1851157393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 1851157393}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 1851157393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.118:6839/430957039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6838', 'nonce': 430957039}, {'type': 'v1', 'addr': '172.21.15.118:6839', 'nonce': 430957039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24577': {'gid': 24577, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6839/3661239153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 3661239153}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 3661239153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T04:45:21.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-23T04:45:21.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-23T04:45:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:20 smithi118 ceph-mon[134297]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1134656515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2722358235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:20 smithi069 ceph-mon[133931]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1134656515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2722358235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 -- 172.21.15.5:0/955440459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214102060 msgr2=0x7f32141024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 --2- 172.21.15.5:0/955440459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214102060 0x7f32141024e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3208009990 tx=0x7f3208009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 -- 172.21.15.5:0/955440459 shutdown_connections 2023-09-23T04:45:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 --2- 172.21.15.5:0/955440459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102a20 0x7f3214109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 --2- 172.21.15.5:0/955440459 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214100f10 0x7f3214101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 --2- 172.21.15.5:0/955440459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214102060 0x7f32141024e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 -- 172.21.15.5:0/955440459 >> 172.21.15.5:0/955440459 conn(0x7f32140fc480 msgr2=0x7f32140fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 -- 172.21.15.5:0/955440459 shutdown_connections 2023-09-23T04:45:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.274+0000 7f321a5ff700 1 -- 172.21.15.5:0/955440459 wait complete. 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 Processor -- start 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 -- start start 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214100f10 0x7f3214074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 0x7f3214073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214073ef0 con 0x7f3214102060 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214074060 con 0x7f3214100f10 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.275+0000 7f321a5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32141a0c50 con 0x7f3214102a20 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f3218b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 0x7f3214073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53242/0 (socket says 172.21.15.5:53242) 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 -- 172.21.15.5:0/2753913539 learned_addr learned my addr 172.21.15.5:0/2753913539 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f3218b9c700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 0x7f3214073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49020/0 (socket says 172.21.15.5:49020) 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f3213fff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214100f10 0x7f3214074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 -- 172.21.15.5:0/2753913539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 msgr2=0x7f3214073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 0x7f3214073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 -- 172.21.15.5:0/2753913539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214100f10 msgr2=0x7f3214074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214100f10 0x7f3214074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 -- 172.21.15.5:0/2753913539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080095f0 con 0x7f3214102060 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.276+0000 7f32137fe700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f320800c010 tx=0x7f3208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.277+0000 7f32117fa700 1 -- 172.21.15.5:0/2753913539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320801d070 con 0x7f3214102060 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.277+0000 7f32117fa700 1 -- 172.21.15.5:0/2753913539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3208005030 con 0x7f3214102060 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.277+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32141a0ed0 con 0x7f3214102060 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.277+0000 7f32117fa700 1 -- 172.21.15.5:0/2753913539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208004b90 con 0x7f3214102060 2023-09-23T04:45:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.277+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32141a1240 con 0x7f3214102060 2023-09-23T04:45:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.280+0000 7f32117fa700 1 -- 172.21.15.5:0/2753913539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f3208004d70 con 0x7f3214102060 2023-09-23T04:45:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.281+0000 7f322de7fb80 1 Processor -- start 2023-09-23T04:45:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.281+0000 7f32117fa700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 0x7f31fc0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.281+0000 7f32117fa700 1 -- 172.21.15.5:0/2753913539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32080ce6a0 con 0x7f3214102060 2023-09-23T04:45:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 -- start start 2023-09-23T04:45:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f3218b9c700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 0x7f31fc0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb2d50 0x560ce3bb31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3b19410 con 0x560ce3bb2d50 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3b0abd0 con 0x560ce3bb1550 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f322de7fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3b084f0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f3213fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f3213fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33606/0 (socket says 172.21.15.5:33606) 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f3213fff700 1 -- 172.21.15.5:0/1252406507 learned_addr learned my addr 172.21.15.5:0/1252406507 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.282+0000 7f32137fe700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3218b9c700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 0x7f31fc0a3af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3214075230 tx=0x7f320c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3218b9c700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb2d50 0x560ce3bb31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 -- 172.21.15.5:0/1252406507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 msgr2=0x560ce3bb2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 -- 172.21.15.5:0/1252406507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb2d50 msgr2=0x560ce3bb31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb2d50 0x560ce3bb31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 -- 172.21.15.5:0/1252406507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ce3b66880 con 0x560ce3bb1550 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f32137fe700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f3213fff700 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f320400bd70 tx=0x7f32040106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2050d923367c5190 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.283+0000 7f31fa7fc700 1 -- 172.21.15.5:0/1252406507 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204010e10 con 0x560ce3bb1550 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f31fa7fc700 1 -- 172.21.15.5:0/1252406507 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3204015070 con 0x560ce3bb1550 2023-09-23T04:45:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f31fa7fc700 1 -- 172.21.15.5:0/1252406507 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204011730 con 0x560ce3bb1550 2023-09-23T04:45:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 -- 172.21.15.5:0/1252406507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 msgr2=0x560ce3bb1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f320400bd70 tx=0x7f32040106b0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 -- 172.21.15.5:0/1252406507 shutdown_connections 2023-09-23T04:45:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb2d50 0x560ce3bb31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1550 0x560ce3bb1970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1252406507 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 -- 172.21.15.5:0/1252406507 >> 172.21.15.5:0/1252406507 conn(0x560ce3b25210 msgr2=0x560ce3bac430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.284+0000 7f322de7fb80 1 -- 172.21.15.5:0/1252406507 shutdown_connections 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.285+0000 7f322de7fb80 1 -- 172.21.15.5:0/1252406507 wait complete. 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.285+0000 7f322de7fb80 1 -- start start 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.285+0000 7f322de7fb80 1 Processor -- start 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bc28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3b03de0 con 0x560ce3bb1550 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3aa06c0 con 0x560ce3bb2d50 2023-09-23T04:45:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3aa0360 con 0x560ce3bb1eb0 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f322de7fb80 10 client.0 _ll_register_callbacks cb 0x560ce38fd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3218b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bc28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f32137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53264/0 (socket says 172.21.15.5:53264) 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 -- 172.21.15.5:0/3618161886 learned_addr learned my addr 172.21.15.5:0/3618161886 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 -- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 msgr2=0x560ce3bbb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bbb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 -- 172.21.15.5:0/3618161886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 msgr2=0x560ce3bc28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bc28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f3213fff700 1 -- 172.21.15.5:0/3618161886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32141a1240 con 0x560ce3bb1550 2023-09-23T04:45:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.286+0000 7f32137fe700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bbb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f3218b9c700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bc28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f3213fff700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32040059d0 tx=0x7f3204010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fdb6a14fcde4fe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f31f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204020070 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ce3ba4200 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3204017070 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320400bec0 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f322de7fb80 10 client.15111 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.287+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f320400b9a0 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ce3ba4200 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3204004750 con 0x560ce3bb1550 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 10 client.15111 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-23T04:45:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 20 client.15111 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3bc6e30 0x560ce3bc5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 4 client.15111 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.288+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ce3afcdc0 con 0x560ce3bc6e30 2023-09-23T04:45:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.289+0000 7f32137fe700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3bc6e30 0x560ce3bc5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.289+0000 7f32137fe700 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3bc6e30 0x560ce3bc5940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d47acf29c385ca95 server_cookie=2ad365388de7db0f in_seq=0 out_seq=0 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.289+0000 7f31f37fe700 10 client.15111 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f31f37fe700 1 -- 172.21.15.5:0/3618161886 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ce3afcdc0 con 0x560ce3bc6e30 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f31f37fe700 10 client.15111 handle_command_reply: tid=0 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 client.15111 shutdown 2023-09-23T04:45:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 msgr2=0x560ce3bbb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32040059d0 tx=0x7f3204010e40 comp rx=0 tx=0).stop 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 shutdown_connections 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bc28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3bc6e30 0x560ce3bc5940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bbb320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 --2- 172.21.15.5:0/3618161886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1eb0 0x560ce3bbb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 >> 172.21.15.5:0/3618161886 conn(0x560ce3b25210 msgr2=0x560ce3bac240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f31f37fe700 0 client.15111 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 shutdown_connections 2023-09-23T04:45:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.290+0000 7f322de7fb80 1 -- 172.21.15.5:0/3618161886 wait complete. 2023-09-23T04:45:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.291+0000 7f322de7fb80 20 client.15111 trim_cache size 0 max 16384 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 Processor -- start 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 -- start start 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bb5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb2d50 0x560ce3bc2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce39848d0 con 0x560ce3bb1550 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3afcdc0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.309+0000 7f322de7fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3badb40 con 0x560ce3bb2d50 2023-09-23T04:45:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33626/0 (socket says 172.21.15.5:33626) 2023-09-23T04:45:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 -- 172.21.15.5:0/1149191358 learned_addr learned my addr 172.21.15.5:0/1149191358 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f3213fff700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bb5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f3218b9c700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb2d50 0x560ce3bc2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 -- 172.21.15.5:0/1149191358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb2d50 msgr2=0x560ce3bc2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb2d50 0x560ce3bc2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 -- 172.21.15.5:0/1149191358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 msgr2=0x560ce3bb5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bb5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 -- 172.21.15.5:0/1149191358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ce392ed90 con 0x560ce3bb1eb0 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f3213fff700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bb5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f32137fe700 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f320809b210 tx=0x7f3208023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20020f924e936ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.310+0000 7f31faffd700 1 -- 172.21.15.5:0/1149191358 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32080a3810 con 0x560ce3bb1eb0 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f31faffd700 1 -- 172.21.15.5:0/1149191358 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32080998b0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f31faffd700 1 -- 172.21.15.5:0/1149191358 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320809aa00 con 0x560ce3bb1eb0 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 -- 172.21.15.5:0/1149191358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 msgr2=0x560ce3bb61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f320809b210 tx=0x7f3208023040 comp rx=0 tx=0).stop 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 -- 172.21.15.5:0/1149191358 shutdown_connections 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb2d50 0x560ce3bc2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1550 0x560ce3bb5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 --2- 172.21.15.5:0/1149191358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb1eb0 0x560ce3bb61c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 -- 172.21.15.5:0/1149191358 >> 172.21.15.5:0/1149191358 conn(0x560ce3b25210 msgr2=0x560ce3bb9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 -- 172.21.15.5:0/1149191358 shutdown_connections 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.311+0000 7f322de7fb80 1 -- 172.21.15.5:0/1149191358 wait complete. 2023-09-23T04:45:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 -- start start 2023-09-23T04:45:21.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 Processor -- start 2023-09-23T04:45:21.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 0x560ce3bb8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bb6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3bb71f0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3bb7360 con 0x560ce3bb2d50 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ce3bb7520 con 0x560ce3bb1550 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.312+0000 7f322de7fb80 10 client.0 _ll_register_callbacks cb 0x7f320c00fdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f3213fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 0x560ce3bb8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53268/0 (socket says 172.21.15.5:53268) 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 -- 172.21.15.5:0/125674640 learned_addr learned my addr 172.21.15.5:0/125674640 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f3213fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 0x560ce3bb8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49056/0 (socket says 172.21.15.5:49056) 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f3218b9c700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bb6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 -- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 msgr2=0x560ce3bb8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 0x560ce3bb8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 -- 172.21.15.5:0/125674640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 msgr2=0x560ce3bb6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bb6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 -- 172.21.15.5:0/125674640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320400b9a0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f32137fe700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3208097f30 tx=0x7f3208097900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc775c878ede2c6b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f31f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208066070 con 0x560ce3bb1eb0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.313+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32080051a0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3204015070 con 0x560ce3bb1eb0 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320809aa00 con 0x560ce3bb1eb0 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 10 client.15117 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f32141a0ed0 con 0x560ce3bb1eb0 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3204015070 con 0x560ce3bb1eb0 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32080a1410 con 0x560ce3bb1eb0 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 10 client.15117 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 20 client.15117 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3c1efa0 0x560ce3c1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 4 client.15117 mds_command: new command op to 24529 tid=0[{"prefix": "status"}] 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.314+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ce3afcae0 con 0x560ce3c1efa0 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f3213fff700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3c1efa0 0x560ce3c1b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f3213fff700 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3c1efa0 0x560ce3c1b440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=834febd55aca7cce server_cookie=e1a9efb130ca95c6 in_seq=0 out_seq=0 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f31f37fe700 10 client.15117 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f31f37fe700 1 -- 172.21.15.5:0/125674640 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560ce3afcae0 con 0x560ce3c1efa0 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f31f37fe700 10 client.15117 handle_command_reply: tid=0 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.315+0000 7f322de7fb80 1 client.15117 shutdown 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 msgr2=0x560ce3bc2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3208097f30 tx=0x7f3208097900 comp rx=0 tx=0).stop 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 shutdown_connections 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ce3bb2d50 0x560ce3bb6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ce3bb1550 0x560ce3bb8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x560ce3c1efa0 0x560ce3c1b440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 --2- 172.21.15.5:0/125674640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ce3bb1eb0 0x560ce3bc2860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f31f37fe700 0 client.15117 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 >> 172.21.15.5:0/125674640 conn(0x560ce3b25210 msgr2=0x560ce3bb5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 shutdown_connections 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 1 -- 172.21.15.5:0/125674640 wait complete. 2023-09-23T04:45:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f322de7fb80 20 client.15117 trim_cache size 0 max 16384 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24529, 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:21.322 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 61.512103574000001, 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 116.26504197600001 2023-09-23T04:45:21.323 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 msgr2=0x7f31fc0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 0x7f31fc0a3af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3214075230 tx=0x7f320c00b040 comp rx=0 tx=0).stop 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 msgr2=0x7f3214073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.316+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f320800c010 tx=0x7f3208004730 comp rx=0 tx=0).stop 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 shutdown_connections 2023-09-23T04:45:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f31fc0a1630 0x7f31fc0a3af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3214102a20 0x7f3214073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3214100f10 0x7f3214074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 --2- 172.21.15.5:0/2753913539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3214102060 0x7f3214073210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 >> 172.21.15.5:0/2753913539 conn(0x7f32140fc480 msgr2=0x7f32141055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 shutdown_connections 2023-09-23T04:45:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.317+0000 7f321a5ff700 1 -- 172.21.15.5:0/2753913539 wait complete. 2023-09-23T04:45:21.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-23T04:45:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 -- 172.21.15.5:0/4281803896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102090 msgr2=0x7f7fe4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/4281803896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102090 0x7f7fe4102510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009990 tx=0x7f7fd8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 -- 172.21.15.5:0/4281803896 shutdown_connections 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/4281803896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102a50 0x7f7fe4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/4281803896 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 0x7f7fe41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/4281803896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102090 0x7f7fe4102510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 -- 172.21.15.5:0/4281803896 >> 172.21.15.5:0/4281803896 conn(0x7f7fe40fc470 msgr2=0x7f7fe40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 -- 172.21.15.5:0/4281803896 shutdown_connections 2023-09-23T04:45:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.539+0000 7f7fea1ff700 1 -- 172.21.15.5:0/4281803896 wait complete. 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.540+0000 7f7fea1ff700 1 Processor -- start 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.540+0000 7f7fea1ff700 1 -- start start 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 0x7f7fe4196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102090 0x7f7fe41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419e810 con 0x7f7fe4102a50 2023-09-23T04:45:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419e980 con 0x7f7fe4102090 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fea1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe419eb40 con 0x7f7fe4100eb0 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53276/0 (socket says 172.21.15.5:53276) 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 0x7f7fe4196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2317024727 learned_addr learned my addr 172.21.15.5:0/2317024727 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102090 0x7f7fe41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2317024727 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 msgr2=0x7f7fe4196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 0x7f7fe4196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.541+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2317024727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102090 msgr2=0x7f7fe41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102090 0x7f7fe41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2317024727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80095f0 con 0x7f7fe4102a50 2023-09-23T04:45:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00ef30 tx=0x7f7fdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe0ff9700 1 -- 172.21.15.5:0/2317024727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018070 con 0x7f7fe4102a50 2023-09-23T04:45:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe419edc0 con 0x7f7fe4102a50 2023-09-23T04:45:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe0ff9700 1 -- 172.21.15.5:0/2317024727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fdc00f040 con 0x7f7fe4102a50 2023-09-23T04:45:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fe0ff9700 1 -- 172.21.15.5:0/2317024727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00cd00 con 0x7f7fe4102a50 2023-09-23T04:45:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.542+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe419f240 con 0x7f7fe4102a50 2023-09-23T04:45:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.545+0000 7f7fe0ff9700 1 -- 172.21.15.5:0/2317024727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f7fdc003a00 con 0x7f7fe4102a50 2023-09-23T04:45:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.546+0000 7f7ffdbfab80 1 Processor -- start 2023-09-23T04:45:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 -- start start 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e4fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe0ff9700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 0x7f7fcc0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e50b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669db3410 con 0x564669e4f960 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669da4bd0 con 0x564669e51160 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669da24f0 con 0x564669e502c0 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe0ff9700 1 -- 172.21.15.5:0/2317024727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7fdc0cced0 con 0x7f7fe4102a50 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e4fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 0x7f7fcc0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33668/0 (socket says 172.21.15.5:33668) 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.547+0000 7f7fe3fff700 1 -- 172.21.15.5:0/3251311962 learned_addr learned my addr 172.21.15.5:0/3251311962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e50b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 -- 172.21.15.5:0/3251311962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 msgr2=0x564669e50b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e50b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 -- 172.21.15.5:0/3251311962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 msgr2=0x564669e4fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e4fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 -- 172.21.15.5:0/3251311962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564669e00880 con 0x564669e51160 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e50b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0a1b10 tx=0x7f7fdc099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28f698b0079eae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 0x7f7fcc0a3af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4197b70 tx=0x7f7fd4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fc9ffb700 1 -- 172.21.15.5:0/3251311962 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc071ba0 con 0x564669e51160 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fc9ffb700 1 -- 172.21.15.5:0/3251311962 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fdc00ce80 con 0x564669e51160 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.548+0000 7f7fc9ffb700 1 -- 172.21.15.5:0/3251311962 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc099d40 con 0x564669e51160 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/3251311962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 msgr2=0x564669e515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0a1b10 tx=0x7f7fdc099880 comp rx=0 tx=0).stop 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/3251311962 shutdown_connections 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e51160 0x564669e515e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/3251311962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e50b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/3251311962 >> 172.21.15.5:0/3251311962 conn(0x564669dbf210 msgr2=0x564669e46430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/3251311962 shutdown_connections 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.549+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/3251311962 wait complete. 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 -- start start 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 Processor -- start 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 0x564669e59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669d9dde0 con 0x564669e51160 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669d3a6c0 con 0x564669e4f960 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.550+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669d3a360 con 0x564669e502c0 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7ffdbfab80 10 client.0 _ll_register_callbacks cb 0x564669b977b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 0x564669e59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53296/0 (socket says 172.21.15.5:53296) 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2774370828 learned_addr learned my addr 172.21.15.5:0/2774370828 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 msgr2=0x564669e59c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e59c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2774370828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 msgr2=0x564669e59750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 0x564669e59750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 -- 172.21.15.5:0/2774370828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe419f240 con 0x564669e51160 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 0x564669e59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc099a40 tx=0x7f7fdc0caa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c08aa4c49a8aa7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.551+0000 7f7fc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc09de90 con 0x564669e51160 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7fdc0a2040 con 0x564669e51160 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564669e3e200 con 0x564669e51160 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc099d40 con 0x564669e51160 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 10 client.15129 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564669bc8d90 con 0x564669e51160 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564669e3e200 con 0x564669e51160 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7fdc099350 con 0x564669e51160 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 10 client.15129 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 20 client.15129 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669e64e30 0x564669e63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.552+0000 7f7ffdbfab80 4 client.15129 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.553+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564669d96dc0 con 0x564669e64e30 2023-09-23T04:45:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.553+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669e64e30 0x564669e63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.553+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669e64e30 0x564669e63c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1cf567de7317412c server_cookie=13e171af1c57bfa1 in_seq=0 out_seq=0 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.553+0000 7f7fc2ffd700 10 client.15129 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.554+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/2774370828 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564669d96dc0 con 0x564669e64e30 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.554+0000 7f7fc2ffd700 10 client.15129 handle_command_reply: tid=0 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.554+0000 7f7ffdbfab80 1 client.15129 shutdown 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.554+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 msgr2=0x564669e60cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc099a40 tx=0x7f7fdc0caa10 comp rx=0 tx=0).stop 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 shutdown_connections 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669e64e30 0x564669e63c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e51160 0x564669e60cd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e4f960 0x564669e59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7fc2ffd700 0 client.15129 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/2774370828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e502c0 0x564669e59c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 >> 172.21.15.5:0/2774370828 conn(0x564669dbf210 msgr2=0x564669e46240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 shutdown_connections 2023-09-23T04:45:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/2774370828 wait complete. 2023-09-23T04:45:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.555+0000 7f7ffdbfab80 20 client.15129 trim_cache size 0 max 16384 2023-09-23T04:45:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.573+0000 7f7ffdbfab80 1 Processor -- start 2023-09-23T04:45:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.573+0000 7f7ffdbfab80 1 -- start start 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e57680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669c1e8d0 con 0x564669e4f960 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669d96dc0 con 0x564669e502c0 2023-09-23T04:45:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669e5a940 con 0x564669e51160 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53310/0 (socket says 172.21.15.5:53310) 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 -- 172.21.15.5:0/622246185 learned_addr learned my addr 172.21.15.5:0/622246185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e57680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 -- 172.21.15.5:0/622246185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 msgr2=0x564669e60c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 -- 172.21.15.5:0/622246185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 msgr2=0x564669e57680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e57680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.574+0000 7f7fe37fe700 1 -- 172.21.15.5:0/622246185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564669bb1c10 con 0x564669e4f960 2023-09-23T04:45:21.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564669d9de70 tx=0x7f7fd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73be47ed53d1121f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7fca7fc700 1 -- 172.21.15.5:0/622246185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4024070 con 0x564669e4f960 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7fca7fc700 1 -- 172.21.15.5:0/622246185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fd4015070 con 0x564669e4f960 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7fca7fc700 1 -- 172.21.15.5:0/622246185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40119e0 con 0x564669e4f960 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/622246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 msgr2=0x564669e44eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564669d9de70 tx=0x7f7fd4013000 comp rx=0 tx=0).stop 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/622246185 shutdown_connections 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e44eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/622246185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e57680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.575+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/622246185 >> 172.21.15.5:0/622246185 conn(0x564669dbf210 msgr2=0x564669e6d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.576+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/622246185 shutdown_connections 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.576+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/622246185 wait complete. 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.576+0000 7f7ffdbfab80 1 -- start start 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.576+0000 7f7ffdbfab80 1 Processor -- start 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669e5a0d0 con 0x564669e4f960 2023-09-23T04:45:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669e60c90 con 0x564669e502c0 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564669e60e50 con 0x564669e51160 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7ffdbfab80 10 client.0 _ll_register_callbacks cb 0x7f7fdc0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7fe2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33698/0 (socket says 172.21.15.5:33698) 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7fe2ffd700 1 -- 172.21.15.5:0/1219606053 learned_addr learned my addr 172.21.15.5:0/1219606053 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.577+0000 7f7fe3fff700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 -- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 msgr2=0x564669e60750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 -- 172.21.15.5:0/1219606053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 msgr2=0x564669e5dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e5dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 -- 172.21.15.5:0/1219606053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd4010940 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fe2ffd700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4100bd0 tx=0x7f7fd8005340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b8c96b407eb222 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8004710 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7fd8004870 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fdc00ce80 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fd8004a30 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.578+0000 7f7ffdbfab80 10 client.25142 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564669bef200 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fdc00ce80 con 0x564669e502c0 2023-09-23T04:45:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7fd800fc50 con 0x564669e502c0 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 10 client.25142 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 10 client.25142 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 20 client.25142 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669ebd670 0x564669eb9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 4 client.25142 mds_command: new command op to 24535 tid=0[{"prefix": "status"}] 2023-09-23T04:45:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.579+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564669d96ae0 con 0x564669ebd670 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669ebd670 0x564669eb9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7fe37fe700 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669ebd670 0x564669eb9ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ee265f907a1f105 server_cookie=75f5642cab33bd40 in_seq=0 out_seq=0 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7fc2ffd700 10 client.25142 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7fc2ffd700 1 -- 172.21.15.5:0/1219606053 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564669d96ae0 con 0x564669ebd670 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7fc2ffd700 10 client.25142 handle_command_reply: tid=0 2023-09-23T04:45:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7ffdbfab80 1 client.25142 shutdown 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 msgr2=0x564669e60210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.580+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4100bd0 tx=0x7f7fd8005340 comp rx=0 tx=0).stop 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 shutdown_connections 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564669e51160 0x564669e60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x564669ebd670 0x564669eb9ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564669e4f960 0x564669e5dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7fc2ffd700 0 client.25142 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 --2- 172.21.15.5:0/1219606053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564669e502c0 0x564669e60210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 >> 172.21.15.5:0/1219606053 conn(0x564669dbf210 msgr2=0x564669e44eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 shutdown_connections 2023-09-23T04:45:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 1 -- 172.21.15.5:0/1219606053 wait complete. 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7ffdbfab80 20 client.25142 trim_cache size 0 max 16384 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24535, 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 62.790938173999997, 2023-09-23T04:45:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2023-09-23T04:45:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 114.366977851 2023-09-23T04:45:21.587 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 msgr2=0x7f7fcc0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 0x7f7fcc0a3af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4197b70 tx=0x7f7fd4007400 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 msgr2=0x7f7fe419e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.581+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00ef30 tx=0x7f7fdc00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 shutdown_connections 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7fcc0a1630 0x7f7fcc0a3af0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4102a50 0x7f7fe419e240 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fe4100eb0 0x7f7fe4196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 --2- 172.21.15.5:0/2317024727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fe4102090 0x7f7fe41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 >> 172.21.15.5:0/2317024727 conn(0x7f7fe40fc470 msgr2=0x7f7fe41056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 shutdown_connections 2023-09-23T04:45:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.582+0000 7f7fea1ff700 1 -- 172.21.15.5:0/2317024727 wait complete. 2023-09-23T04:45:21.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T04:45:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 -- 172.21.15.5:0/486229085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4104c90 msgr2=0x7f48e410b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/486229085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4104c90 0x7f48e410b540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b0d0 tx=0x7f48dc00b3e0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 -- 172.21.15.5:0/486229085 shutdown_connections 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/486229085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4104c90 0x7f48e410b540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/486229085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e4103100 0x7f48e4103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/486229085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e41042f0 0x7f48e4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 -- 172.21.15.5:0/486229085 >> 172.21.15.5:0/486229085 conn(0x7f48e40fe670 msgr2=0x7f48e4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 -- 172.21.15.5:0/486229085 shutdown_connections 2023-09-23T04:45:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.805+0000 7f48ea7ff700 1 -- 172.21.15.5:0/486229085 wait complete. 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.807+0000 7f48ea7ff700 1 Processor -- start 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.807+0000 7f48ea7ff700 1 -- start start 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.807+0000 7f48ea7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.807+0000 7f48ea7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e41042f0 0x7f48e4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48ea7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 0x7f48e41a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48ea7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a0a40 con 0x7f48e4103100 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48ea7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a0b80 con 0x7f48e41042f0 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48ea7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41a0d10 con 0x7f48e4104c90 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53348/0 (socket says 172.21.15.5:53348) 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 -- 172.21.15.5:0/306626191 learned_addr learned my addr 172.21.15.5:0/306626191 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e37fe700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e41042f0 0x7f48e4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 0x7f48e41a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 -- 172.21.15.5:0/306626191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 msgr2=0x7f48e41a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 0x7f48e41a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 -- 172.21.15.5:0/306626191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e41042f0 msgr2=0x7f48e4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e41042f0 0x7f48e4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 -- 172.21.15.5:0/306626191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc009d00 con 0x7f48e4103100 2023-09-23T04:45:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 0x7f48e41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.808+0000 7f48e3fff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e970 tx=0x7f48d400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.809+0000 7f48e17fa700 1 -- 172.21.15.5:0/306626191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4009790 con 0x7f48e4103100 2023-09-23T04:45:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.809+0000 7f48e17fa700 1 -- 172.21.15.5:0/306626191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d400eec0 con 0x7f48e4103100 2023-09-23T04:45:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.809+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e41a0ff0 con 0x7f48e4103100 2023-09-23T04:45:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.809+0000 7f48e17fa700 1 -- 172.21.15.5:0/306626191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003cf0 con 0x7f48e4103100 2023-09-23T04:45:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.809+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41a1420 con 0x7f48e4103100 2023-09-23T04:45:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.812+0000 7f48e17fa700 1 -- 172.21.15.5:0/306626191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f48d400ccd0 con 0x7f48e4103100 2023-09-23T04:45:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.813+0000 7f48fe1d1b80 1 Processor -- start 2023-09-23T04:45:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.813+0000 7f48e17fa700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 0x7f48cc0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e17fa700 1 -- 172.21.15.5:0/306626191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48d4014070 con 0x7f48e4103100 2023-09-23T04:45:21.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 -- start start 2023-09-23T04:45:21.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 0x7f48cc0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b312770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b3131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b279410 con 0x55797b311550 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b26abd0 con 0x55797b312d50 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b2684f0 con 0x55797b311eb0 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53364/0 (socket says 172.21.15.5:53364) 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e3fff700 1 -- 172.21.15.5:0/2704214632 learned_addr learned my addr 172.21.15.5:0/2704214632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.814+0000 7f48e37fe700 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b312770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b3131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 -- 172.21.15.5:0/2704214632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 msgr2=0x55797b312770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b312770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 -- 172.21.15.5:0/2704214632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 msgr2=0x55797b3131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b3131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 -- 172.21.15.5:0/2704214632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55797b2c6880 con 0x55797b311550 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 0x7f48cc0a3b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b770 tx=0x7f48dc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48d40a2270 tx=0x7f48d4099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cbb4ee4a7a1984 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48ca7fc700 1 -- 172.21.15.5:0/2704214632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d409a410 con 0x55797b311550 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48ca7fc700 1 -- 172.21.15.5:0/2704214632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d40b1030 con 0x55797b311550 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.815+0000 7f48ca7fc700 1 -- 172.21.15.5:0/2704214632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d409e170 con 0x55797b311550 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2704214632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 msgr2=0x55797b311970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48d40a2270 tx=0x7f48d4099df0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2704214632 shutdown_connections 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b3131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b311970 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2704214632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b312770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2704214632 >> 172.21.15.5:0/2704214632 conn(0x55797b285210 msgr2=0x55797b30c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2704214632 shutdown_connections 2023-09-23T04:45:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.816+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2704214632 wait complete. 2023-09-23T04:45:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 -- start start 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 Processor -- start 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b31b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b322920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b263de0 con 0x55797b311550 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b2006c0 con 0x55797b312d50 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b200360 con 0x55797b311eb0 2023-09-23T04:45:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.817+0000 7f48fe1d1b80 10 client.0 _ll_register_callbacks cb 0x55797b05d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53366/0 (socket says 172.21.15.5:53366) 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 -- 172.21.15.5:0/1639069652 learned_addr learned my addr 172.21.15.5:0/1639069652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b31b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b322920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 -- 172.21.15.5:0/1639069652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 msgr2=0x55797b31b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b31b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 -- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 msgr2=0x55797b322920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b322920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 -- 172.21.15.5:0/1639069652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41a1420 con 0x55797b311550 2023-09-23T04:45:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48e3fff700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d4096e80 tx=0x7f48d4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ab74bce24e3380 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d409a410 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48d409a670 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.818+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55797b304200 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d409a830 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 10 client.15147 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55797b08ed90 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55797b304200 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48d4099370 con 0x55797b311550 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T04:45:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 20 client.15147 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b326b20 0x55797b325850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 4 client.15147 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55797b25cdc0 con 0x55797b326b20 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.819+0000 7f48e37fe700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b326b20 0x55797b325850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.820+0000 7f48e37fe700 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b326b20 0x55797b325850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0f7b55a496a2c51 server_cookie=36d3b0a4ac963a3c in_seq=0 out_seq=0 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.820+0000 7f48c37fe700 10 client.15147 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48c37fe700 1 -- 172.21.15.5:0/1639069652 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55797b25cdc0 con 0x55797b326b20 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48c37fe700 10 client.15147 handle_command_reply: tid=0 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 client.15147 shutdown 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 msgr2=0x55797b31b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d4096e80 tx=0x7f48d4003680 comp rx=0 tx=0).stop 2023-09-23T04:45:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 shutdown_connections 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b326b20 0x55797b325850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b322920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b31b3a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/1639069652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b31b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 >> 172.21.15.5:0/1639069652 conn(0x55797b285210 msgr2=0x55797b30c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48c37fe700 0 client.15147 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 shutdown_connections 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.821+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/1639069652 wait complete. 2023-09-23T04:45:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.822+0000 7f48fe1d1b80 20 client.15147 trim_cache size 0 max 16384 2023-09-23T04:45:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.840+0000 7f48fe1d1b80 1 Processor -- start 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 -- start start 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b3228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b319fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b0e48d0 con 0x55797b311550 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b25cdc0 con 0x55797b312d50 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b25fed0 con 0x55797b311eb0 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53380/0 (socket says 172.21.15.5:53380) 2023-09-23T04:45:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 -- 172.21.15.5:0/2502851724 learned_addr learned my addr 172.21.15.5:0/2502851724 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e37fe700 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b3228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b319fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 -- 172.21.15.5:0/2502851724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 msgr2=0x55797b3228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b3228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 -- 172.21.15.5:0/2502851724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 msgr2=0x55797b319fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b319fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.841+0000 7f48e3fff700 1 -- 172.21.15.5:0/2502851724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55797b077c10 con 0x55797b311550 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48e3fff700 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48d409a110 tx=0x7f48d40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5e739fe67abb44 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48caffd700 1 -- 172.21.15.5:0/2502851724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40987b0 con 0x55797b311550 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48caffd700 1 -- 172.21.15.5:0/2502851724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d409a410 con 0x55797b311550 2023-09-23T04:45:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48caffd700 1 -- 172.21.15.5:0/2502851724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4099b60 con 0x55797b311550 2023-09-23T04:45:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2502851724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 msgr2=0x55797b30aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.842+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48d409a110 tx=0x7f48d40036f0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2502851724 shutdown_connections 2023-09-23T04:45:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b312d50 0x55797b319fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311550 0x55797b30aeb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/2502851724 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b311eb0 0x55797b3228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2502851724 >> 172.21.15.5:0/2502851724 conn(0x55797b285210 msgr2=0x55797b32f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2502851724 shutdown_connections 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/2502851724 wait complete. 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 -- start start 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.843+0000 7f48fe1d1b80 1 Processor -- start 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b311550 0x55797b31eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b312d50 0x55797b321e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b322390 con 0x55797b311eb0 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b322500 con 0x55797b311550 2023-09-23T04:45:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55797b3226c0 con 0x55797b312d50 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48fe1d1b80 10 client.0 _ll_register_callbacks cb 0x7f48d8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53394/0 (socket says 172.21.15.5:53394) 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 -- 172.21.15.5:0/106221684 learned_addr learned my addr 172.21.15.5:0/106221684 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e8d9c700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b312d50 0x55797b321e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e3fff700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b311550 0x55797b31eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 -- 172.21.15.5:0/106221684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b312d50 msgr2=0x55797b321e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b312d50 0x55797b321e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 -- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b311550 msgr2=0x55797b31eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b311550 0x55797b31eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 -- 172.21.15.5:0/106221684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55797b08ed90 con 0x55797b311eb0 2023-09-23T04:45:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48e37fe700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55797b327850 tx=0x7f48d8010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20392ad691730b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.844+0000 7f48c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8017040 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48d801e570 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d40b1030 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d8010e80 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48fe1d1b80 10 client.15153 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55797b0b5200 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.845+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d40b1030 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48d8003890 con 0x55797b311eb0 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T04:45:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 20 client.15153 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b37f120 0x55797b37b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 4 client.15153 mds_command: new command op to 24554 tid=0[{"prefix": "status"}] 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55797b25cae0 con 0x55797b37f120 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.846+0000 7f48e3fff700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b37f120 0x55797b37b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48e3fff700 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b37f120 0x55797b37b5a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf0de68bd70119a2 server_cookie=a437b5dd7a2da02a in_seq=0 out_seq=0 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48c37fe700 10 client.15153 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48c37fe700 1 -- 172.21.15.5:0/106221684 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55797b25cae0 con 0x55797b37f120 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48c37fe700 10 client.15153 handle_command_reply: tid=0 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 client.15153 shutdown 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 msgr2=0x55797b31f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55797b327850 tx=0x7f48d8010710 comp rx=0 tx=0).stop 2023-09-23T04:45:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 shutdown_connections 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55797b37f120 0x55797b37b5a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55797b312d50 0x55797b321e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55797b311550 0x55797b31eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 --2- 172.21.15.5:0/106221684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55797b311eb0 0x55797b31f050 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48c37fe700 0 client.15153 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 >> 172.21.15.5:0/106221684 conn(0x55797b285210 msgr2=0x55797b319170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 shutdown_connections 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 1 -- 172.21.15.5:0/106221684 wait complete. 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.847+0000 7f48fe1d1b80 20 client.15153 trim_cache size 0 max 16384 2023-09-23T04:45:21.852 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24554, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 64.063937172999999, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:21.853 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 101.396949999 2023-09-23T04:45:21.854 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 msgr2=0x7f48cc0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 0x7f48cc0a3b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b770 tx=0x7f48dc00b040 comp rx=0 tx=0).stop 2023-09-23T04:45:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 msgr2=0x7f48e4198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48d400e970 tx=0x7f48d400ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 shutdown_connections 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f48cc0a16b0 0x7f48cc0a3b70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48e4104c90 0x7f48e41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48e4103100 0x7f48e4198ec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 --2- 172.21.15.5:0/306626191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48e41042f0 0x7f48e4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 >> 172.21.15.5:0/306626191 conn(0x7f48e40fe670 msgr2=0x7f48e4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 shutdown_connections 2023-09-23T04:45:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:21.848+0000 7f48ea7ff700 1 -- 172.21.15.5:0/306626191 wait complete. 2023-09-23T04:45:21.864 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:21 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/917106637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:21.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 -- 172.21.15.5:0/1281852746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380102090 msgr2=0x7f4380102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 --2- 172.21.15.5:0/1281852746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380102090 0x7f4380102510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4374009990 tx=0x7f4374009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 -- 172.21.15.5:0/1281852746 shutdown_connections 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 --2- 172.21.15.5:0/1281852746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102a50 0x7f4380109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 --2- 172.21.15.5:0/1281852746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380100eb0 0x7f43801012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 --2- 172.21.15.5:0/1281852746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380102090 0x7f4380102510 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 -- 172.21.15.5:0/1281852746 >> 172.21.15.5:0/1281852746 conn(0x7f43800fc470 msgr2=0x7f43800fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 -- 172.21.15.5:0/1281852746 shutdown_connections 2023-09-23T04:45:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.071+0000 7f4385fff700 1 -- 172.21.15.5:0/1281852746 wait complete. 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 Processor -- start 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 -- start start 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 0x7f4380197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380102a50 0x7f438019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438019e770 con 0x7f4380100eb0 2023-09-23T04:45:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438019e8e0 con 0x7f4380102090 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f4385fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438019eaa0 con 0x7f4380102a50 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.072+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 0x7f4380197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53412/0 (socket says 172.21.15.5:53412) 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437f7fe700 1 -- 172.21.15.5:0/1271020128 learned_addr learned my addr 172.21.15.5:0/1271020128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437ffff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380102a50 0x7f438019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 -- 172.21.15.5:0/1271020128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380102a50 msgr2=0x7f438019e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380102a50 0x7f438019e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 -- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 msgr2=0x7f4380196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 -- 172.21.15.5:0/1271020128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009520 con 0x7f4380102090 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437f7fe700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437effd700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 0x7f4380197170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43740098e0 tx=0x7f4374009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.073+0000 7f437cff9700 1 -- 172.21.15.5:0/1271020128 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374021070 con 0x7f4380102090 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.074+0000 7f437cff9700 1 -- 172.21.15.5:0/1271020128 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4374026930 con 0x7f4380102090 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.074+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740095f0 con 0x7f4380102090 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.074+0000 7f437cff9700 1 -- 172.21.15.5:0/1271020128 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437400f750 con 0x7f4380102090 2023-09-23T04:45:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.074+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380073340 con 0x7f4380102090 2023-09-23T04:45:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.077+0000 7f437cff9700 1 -- 172.21.15.5:0/1271020128 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f4374026aa0 con 0x7f4380102090 2023-09-23T04:45:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 Processor -- start 2023-09-23T04:45:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 -- start start 2023-09-23T04:45:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ece4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ece5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec4c410 con 0x55f58ece5d50 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec3dbd0 con 0x55f58ece4eb0 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f4399989b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec3b4f0 con 0x55f58ece4550 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.079+0000 7f437cff9700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 0x7f43680a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ece4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53414/0 (socket says 172.21.15.5:53414) 2023-09-23T04:45:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ece4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49190/0 (socket says 172.21.15.5:49190) 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 -- 172.21.15.5:0/2172138162 learned_addr learned my addr 172.21.15.5:0/2172138162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437f7fe700 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ece5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437ffff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 0x7f43680a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437cff9700 1 -- 172.21.15.5:0/1271020128 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43740d2db0 con 0x7f4380102090 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 -- 172.21.15.5:0/2172138162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 msgr2=0x55f58ece4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ece4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 -- 172.21.15.5:0/2172138162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 msgr2=0x55f58ece5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ece5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 -- 172.21.15.5:0/2172138162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58ec99880 con 0x55f58ece5d50 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.080+0000 7f437effd700 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43740d0420 tx=0x7f437402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f9566c7f4dda85 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f437ffff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 0x7f43680a3b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4378009ec0 tx=0x7f437800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4365ffb700 1 -- 172.21.15.5:0/2172138162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374026dc0 con 0x55f58ece5d50 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4365ffb700 1 -- 172.21.15.5:0/2172138162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43740a0320 con 0x55f58ece5d50 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4365ffb700 1 -- 172.21.15.5:0/2172138162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437409f110 con 0x55f58ece5d50 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 -- 172.21.15.5:0/2172138162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 msgr2=0x55f58ece61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43740d0420 tx=0x7f437402e000 comp rx=0 tx=0).stop 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 -- 172.21.15.5:0/2172138162 shutdown_connections 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece5d50 0x55f58ece61d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ece4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 --2- 172.21.15.5:0/2172138162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ece5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.081+0000 7f4399989b80 1 -- 172.21.15.5:0/2172138162 >> 172.21.15.5:0/2172138162 conn(0x55f58ec58210 msgr2=0x55f58ecdf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.082+0000 7f4399989b80 1 -- 172.21.15.5:0/2172138162 shutdown_connections 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.082+0000 7f4399989b80 1 -- 172.21.15.5:0/2172138162 wait complete. 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.082+0000 7f4399989b80 1 -- start start 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.082+0000 7f4399989b80 1 Processor -- start 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ecf5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec36de0 con 0x55f58ece4550 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ebd36c0 con 0x55f58ece4eb0 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ebd3360 con 0x55f58ece5d50 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f4399989b80 10 client.0 _ll_register_callbacks cb 0x55f58ea307b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53422/0 (socket says 172.21.15.5:53422) 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437f7fe700 1 -- 172.21.15.5:0/1037924625 learned_addr learned my addr 172.21.15.5:0/1037924625 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33796/0 (socket says 172.21.15.5:33796) 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437ffff700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ecf5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 -- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 msgr2=0x55f58ecf5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ecf5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 -- 172.21.15.5:0/1037924625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 msgr2=0x55f58ecedfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437effd700 1 -- 172.21.15.5:0/1037924625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380073340 con 0x55f58ece4eb0 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.083+0000 7f437f7fe700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f437effd700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43740a3f30 tx=0x7f4374009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b7804d43b5d787 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f435effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374026dc0 con 0x55f58ece4eb0 2023-09-23T04:45:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f437409fc20 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58ecd7200 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43740a7ca0 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f4399989b80 10 client.25169 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f58ea61d90 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.084+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f58ecd7200 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43740a4ab0 con 0x55f58ece4eb0 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 10 client.25169 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 10 client.25169 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 20 client.25169 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ecf9660 0x55f58ecf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 4 client.25169 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f58ec2fdc0 con 0x55f58ecf9660 2023-09-23T04:45:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.085+0000 7f437f7fe700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ecf9660 0x55f58ecf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.086+0000 7f437f7fe700 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ecf9660 0x55f58ecf8490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3f176d913a13216 server_cookie=a14fd702fc1d6286 in_seq=0 out_seq=0 2023-09-23T04:45:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.086+0000 7f435effd700 10 client.25169 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f435effd700 1 -- 172.21.15.5:0/1037924625 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f58ec2fdc0 con 0x55f58ecf9660 2023-09-23T04:45:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f435effd700 10 client.25169 handle_command_reply: tid=0 2023-09-23T04:45:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 client.25169 shutdown 2023-09-23T04:45:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 msgr2=0x55f58ecee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43740a3f30 tx=0x7f4374009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 shutdown_connections 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ecf5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ecf9660 0x55f58ecf8490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 --2- 172.21.15.5:0/1037924625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecee520 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f435effd700 0 client.25169 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 >> 172.21.15.5:0/1037924625 conn(0x55f58ec58210 msgr2=0x55f58ecdf240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 shutdown_connections 2023-09-23T04:45:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 1 -- 172.21.15.5:0/1037924625 wait complete. 2023-09-23T04:45:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.087+0000 7f4399989b80 20 client.25169 trim_cache size 0 max 16384 2023-09-23T04:45:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.105+0000 7f4399989b80 1 Processor -- start 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.105+0000 7f4399989b80 1 -- start start 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ecdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece5d50 0x55f58ecf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eced9f0 con 0x55f58ece4eb0 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eab78d0 con 0x55f58ece5d50 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f4399989b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec2fdf0 con 0x55f58ece4550 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53430/0 (socket says 172.21.15.5:53430) 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 -- 172.21.15.5:0/4051949762 learned_addr learned my addr 172.21.15.5:0/4051949762 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437f7fe700 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ecdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437ffff700 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece5d50 0x55f58ecf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 -- 172.21.15.5:0/4051949762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 msgr2=0x55f58ecdf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ecdf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 -- 172.21.15.5:0/4051949762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece5d50 msgr2=0x55f58ecf8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece5d50 0x55f58ecf8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.106+0000 7f437effd700 1 -- 172.21.15.5:0/4051949762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58ea4ac10 con 0x55f58ece4eb0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f437effd700 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f437409c120 tx=0x7f43740a0200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a0c0f59736c7f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f43667fc700 1 -- 172.21.15.5:0/4051949762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374026dc0 con 0x55f58ece4eb0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f43667fc700 1 -- 172.21.15.5:0/4051949762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f437409fc20 con 0x55f58ece4eb0 2023-09-23T04:45:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f43667fc700 1 -- 172.21.15.5:0/4051949762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437409ba10 con 0x55f58ece4eb0 2023-09-23T04:45:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f4399989b80 1 -- 172.21.15.5:0/4051949762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 msgr2=0x55f58ecf5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.107+0000 7f4399989b80 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f437409c120 tx=0x7f43740a0200 comp rx=0 tx=0).stop 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 -- 172.21.15.5:0/4051949762 shutdown_connections 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece5d50 0x55f58ecf8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece4550 0x55f58ecdf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 --2- 172.21.15.5:0/4051949762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5520 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 -- 172.21.15.5:0/4051949762 >> 172.21.15.5:0/4051949762 conn(0x55f58ec58210 msgr2=0x55f58ecddeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 -- 172.21.15.5:0/4051949762 shutdown_connections 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 -- 172.21.15.5:0/4051949762 wait complete. 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 -- start start 2023-09-23T04:45:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.108+0000 7f4399989b80 1 Processor -- start 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ece79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ece7f20 con 0x55f58ece4550 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ece8090 con 0x55f58ece4eb0 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ece8250 con 0x55f58ece5d50 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f4399989b80 10 client.0 _ll_register_callbacks cb 0x7f4370009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53432/0 (socket says 172.21.15.5:53432) 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 -- 172.21.15.5:0/835920104 learned_addr learned my addr 172.21.15.5:0/835920104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437effd700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437ffff700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ece79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 -- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 msgr2=0x55f58ece79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ece79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 -- 172.21.15.5:0/835920104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 msgr2=0x55f58ecf5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.109+0000 7f437f7fe700 1 -- 172.21.15.5:0/835920104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f58ea61d90 con 0x55f58ece4550 2023-09-23T04:45:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f437f7fe700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4370009e20 tx=0x7f4370020e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487f39188eff5b8b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f435effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370016040 con 0x55f58ece4550 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43740a0320 con 0x55f58ece4550 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4370004020 con 0x55f58ece4550 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43700041e0 con 0x55f58ece4550 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f4399989b80 10 client.15171 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f58ea88200 con 0x55f58ece4550 2023-09-23T04:45:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.110+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43740a0320 con 0x55f58ece4550 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4370005020 con 0x55f58ece4550 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 10 client.15171 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 20 client.15171 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ed51bf0 0x55f58ed4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 4 client.15171 mds_command: new command op to 24559 tid=0[{"prefix": "status"}] 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f58ec2fae0 con 0x55f58ed51bf0 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.111+0000 7f437effd700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ed51bf0 0x55f58ed4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f437effd700 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ed51bf0 0x55f58ed4e050 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d62a7721a340005a server_cookie=9789449a0bcb097f in_seq=0 out_seq=0 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f435effd700 10 client.15171 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f435effd700 1 -- 172.21.15.5:0/835920104 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f58ec2fae0 con 0x55f58ed51bf0 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f435effd700 10 client.15171 handle_command_reply: tid=0 2023-09-23T04:45:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 client.15171 shutdown 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 msgr2=0x55f58ecee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4370009e20 tx=0x7f4370020e40 comp rx=0 tx=0).stop 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 shutdown_connections 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f58ece5d50 0x55f58ece79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f58ece4550 0x55f58ecee2a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55f58ed51bf0 0x55f58ed4e050 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 --2- 172.21.15.5:0/835920104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f58ece4eb0 0x55f58ecf5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 >> 172.21.15.5:0/835920104 conn(0x55f58ec58210 msgr2=0x55f58ecf8490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f435effd700 0 client.15171 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 shutdown_connections 2023-09-23T04:45:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.112+0000 7f4399989b80 1 -- 172.21.15.5:0/835920104 wait complete. 2023-09-23T04:45:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4399989b80 20 client.15171 trim_cache size 0 max 16384 2023-09-23T04:45:22.120 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24559, 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 65.341368643999999, 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2023-09-23T04:45:22.121 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:22.122 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 110.246643836 2023-09-23T04:45:22.122 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 msgr2=0x7f43680a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 0x7f43680a3b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4378009ec0 tx=0x7f437800b040 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 msgr2=0x7f4380197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 0x7f4380197170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43740098e0 tx=0x7f4374009580 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 shutdown_connections 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43680a16b0 0x7f43680a3b70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4380102a50 0x7f438019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4380100eb0 0x7f4380196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 --2- 172.21.15.5:0/1271020128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4380102090 0x7f4380197170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 >> 172.21.15.5:0/1271020128 conn(0x7f43800fc470 msgr2=0x7f43800fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 shutdown_connections 2023-09-23T04:45:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.113+0000 7f4385fff700 1 -- 172.21.15.5:0/1271020128 wait complete. 2023-09-23T04:45:22.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-09-23T04:45:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:21 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/917106637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:21 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/917106637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 -- 172.21.15.5:0/1433551447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc100eb0 msgr2=0x7f81bc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1433551447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc100eb0 0x7f81bc1012d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 -- 172.21.15.5:0/1433551447 shutdown_connections 2023-09-23T04:45:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1433551447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc102a50 0x7f81bc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1433551447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc100eb0 0x7f81bc1012d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1433551447 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102090 0x7f81bc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 -- 172.21.15.5:0/1433551447 >> 172.21.15.5:0/1433551447 conn(0x7f81bc0fc470 msgr2=0x7f81bc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 -- 172.21.15.5:0/1433551447 shutdown_connections 2023-09-23T04:45:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.334+0000 7f81c31ff700 1 -- 172.21.15.5:0/1433551447 wait complete. 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.335+0000 7f81c31ff700 1 Processor -- start 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.335+0000 7f81c31ff700 1 -- start start 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.335+0000 7f81c31ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc100eb0 0x7f81bc074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.335+0000 7f81c31ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c31ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102a50 0x7f81bc073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c31ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc073ef0 con 0x7f81bc102090 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c31ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc074060 con 0x7f81bc100eb0 2023-09-23T04:45:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c31ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc1a0c60 con 0x7f81bc102a50 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53450/0 (socket says 172.21.15.5:53450) 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 -- 172.21.15.5:0/1416289661 learned_addr learned my addr 172.21.15.5:0/1416289661 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc100eb0 0x7f81bc074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81c179c700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102a50 0x7f81bc073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 -- 172.21.15.5:0/1416289661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102a50 msgr2=0x7f81bc073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102a50 0x7f81bc073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 -- 172.21.15.5:0/1416289661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc100eb0 msgr2=0x7f81bc074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc100eb0 0x7f81bc074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.336+0000 7f81bbfff700 1 -- 172.21.15.5:0/1416289661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00095f0 con 0x7f81bc102090 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81bbfff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e970 tx=0x7f81a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81b9ffb700 1 -- 172.21.15.5:0/1416289661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8009790 con 0x7f81bc102090 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81b9ffb700 1 -- 172.21.15.5:0/1416289661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a800c7e0 con 0x7f81bc102090 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81b9ffb700 1 -- 172.21.15.5:0/1416289661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003c70 con 0x7f81bc102090 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc1a0f40 con 0x7f81bc102090 2023-09-23T04:45:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.337+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1a1370 con 0x7f81bc102090 2023-09-23T04:45:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.342+0000 7f81b9ffb700 1 -- 172.21.15.5:0/1416289661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 136242+0+0 (secure 0 0 0) 0x7f81a800cc50 con 0x7f81bc102090 2023-09-23T04:45:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 Processor -- start 2023-09-23T04:45:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 -- start start 2023-09-23T04:45:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 0x5624a09cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 0x5624a09cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a0935410 con 0x5624a09cdeb0 2023-09-23T04:45:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a0926bd0 con 0x5624a09cd550 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a09244f0 con 0x5624a09ced50 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81b9ffb700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 0x7f81ac0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81c0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.343+0000 7f81c0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53454/0 (socket says 172.21.15.5:53454) 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81c179c700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 0x5624a09cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81c0f9b700 1 -- 172.21.15.5:0/25417264 learned_addr learned my addr 172.21.15.5:0/25417264 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 0x5624a09cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81b9ffb700 1 -- 172.21.15.5:0/1416289661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81a8014070 con 0x7f81bc102090 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81c179c700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 0x7f81ac0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 -- 172.21.15.5:0/25417264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 msgr2=0x5624a09cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 0x5624a09cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 -- 172.21.15.5:0/25417264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 msgr2=0x5624a09ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 -- 172.21.15.5:0/25417264 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a0982880 con 0x5624a09ced50 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81c179c700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 0x5624a09cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81bbfff700 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 0x5624a09cf1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f81a80cabe0 tx=0x7f81a808fb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4009413d3f4d634 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.344+0000 7f81a6ffd700 1 -- 172.21.15.5:0/25417264 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a809aa80 con 0x5624a09ced50 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81a6ffd700 1 -- 172.21.15.5:0/25417264 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a809a380 con 0x5624a09ced50 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81a6ffd700 1 -- 172.21.15.5:0/25417264 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80987b0 con 0x5624a09ced50 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81c179c700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 0x7f81ac0a3af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f81bc075230 tx=0x7f81b4009130 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/25417264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 msgr2=0x5624a09cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 0x5624a09cf1d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f81a80cabe0 tx=0x7f81a808fb40 comp rx=0 tx=0).stop 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/25417264 shutdown_connections 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09ced50 0x5624a09cf1d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09cd550 0x5624a09cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/25417264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/25417264 >> 172.21.15.5:0/25417264 conn(0x5624a0941210 msgr2=0x5624a09c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/25417264 shutdown_connections 2023-09-23T04:45:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.345+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/25417264 wait complete. 2023-09-23T04:45:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.346+0000 7f81d6bf5b80 1 -- start start 2023-09-23T04:45:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.346+0000 7f81d6bf5b80 1 Processor -- start 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a091fde0 con 0x5624a09cdeb0 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a08bc6c0 con 0x5624a09ced50 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a08bc360 con 0x5624a09cd550 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81d6bf5b80 10 client.0 _ll_register_callbacks cb 0x5624a07197b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53464/0 (socket says 172.21.15.5:53464) 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 -- 172.21.15.5:0/863511575 learned_addr learned my addr 172.21.15.5:0/863511575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81c179c700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 -- 172.21.15.5:0/863511575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 msgr2=0x5624a09d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 -- 172.21.15.5:0/863511575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 msgr2=0x5624a09de7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09de7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 -- 172.21.15.5:0/863511575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc109ca0 con 0x5624a09cdeb0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81c179c700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f81bbfff700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a80a28d0 tx=0x7f81a809ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6979c609cdc0200 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.347+0000 7f819bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80cb030 con 0x5624a09cdeb0 2023-09-23T04:45:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81a809a5e0 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624a09c0200 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81a8098580 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 10 client.15183 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f81bc1a1370 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624a09c0200 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81a80a2280 con 0x5624a09cdeb0 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 10 client.15183 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 10 client.15183 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.348+0000 7f81d6bf5b80 20 client.15183 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a09e2ab0 0x5624a09e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f81d6bf5b80 4 client.15183 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624a0918dc0 con 0x5624a09e2ab0 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a09e2ab0 0x5624a09e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f819bfff700 10 client.15183 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.349+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a09e2ab0 0x5624a09e1720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6594a3066c6523d2 server_cookie=3f972920bbbe8b11 in_seq=0 out_seq=0 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f819bfff700 1 -- 172.21.15.5:0/863511575 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624a0918dc0 con 0x5624a09e2ab0 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f819bfff700 10 client.15183 handle_command_reply: tid=0 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 client.15183 shutdown 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 msgr2=0x5624a09d77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a80a28d0 tx=0x7f81a809ab00 comp rx=0 tx=0).stop 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 shutdown_connections 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a09e2ab0 0x5624a09e1720 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/863511575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09d77b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f819bfff700 0 client.15183 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 >> 172.21.15.5:0/863511575 conn(0x5624a0941210 msgr2=0x5624a09c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 shutdown_connections 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.350+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/863511575 wait complete. 2023-09-23T04:45:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.351+0000 7f81d6bf5b80 20 client.15183 trim_cache size 0 max 16384 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 Processor -- start 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 -- start start 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a07a08d0 con 0x5624a09cdeb0 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a09c75d0 con 0x5624a09ced50 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a0921070 con 0x5624a09cd550 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53472/0 (socket says 172.21.15.5:53472) 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.369+0000 7f81bbfff700 1 -- 172.21.15.5:0/3061756681 learned_addr learned my addr 172.21.15.5:0/3061756681 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81c179c700 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 -- 172.21.15.5:0/3061756681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 msgr2=0x5624a09c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 -- 172.21.15.5:0/3061756681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 msgr2=0x5624a09d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 -- 172.21.15.5:0/3061756681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a074ad90 con 0x5624a09cdeb0 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81bbfff700 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81a8096df0 tx=0x7f81a809a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fe465fef874530 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81a77fe700 1 -- 172.21.15.5:0/3061756681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8098580 con 0x5624a09cdeb0 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81a77fe700 1 -- 172.21.15.5:0/3061756681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a8003dd0 con 0x5624a09cdeb0 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.370+0000 7f81a77fe700 1 -- 172.21.15.5:0/3061756681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a809a840 con 0x5624a09cdeb0 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/3061756681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 msgr2=0x5624a09de7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81a8096df0 tx=0x7f81a809a280 comp rx=0 tx=0).stop 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/3061756681 shutdown_connections 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cd550 0x5624a09c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/3061756681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cdeb0 0x5624a09de7b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/3061756681 >> 172.21.15.5:0/3061756681 conn(0x5624a0941210 msgr2=0x5624a09eb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/3061756681 shutdown_connections 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/3061756681 wait complete. 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.371+0000 7f81d6bf5b80 1 -- start start 2023-09-23T04:45:22.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 Processor -- start 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cdeb0 0x5624a09dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a09dddc0 con 0x5624a09cd550 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a09ddf30 con 0x5624a09ced50 2023-09-23T04:45:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a09de0f0 con 0x5624a09cdeb0 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.372+0000 7f81d6bf5b80 10 client.0 _ll_register_callbacks cb 0x7f81a8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53488/0 (socket says 172.21.15.5:53488) 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 -- 172.21.15.5:0/931048658 learned_addr learned my addr 172.21.15.5:0/931048658 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81bbfff700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cdeb0 0x5624a09dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c179c700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 -- 172.21.15.5:0/931048658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cdeb0 msgr2=0x5624a09dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cdeb0 0x5624a09dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 -- 172.21.15.5:0/931048658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 msgr2=0x5624a09dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 -- 172.21.15.5:0/931048658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1a1370 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c179c700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f81c0f9b700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b000bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6537b54f43e385e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.373+0000 7f819bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b001d040 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81a809a380 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81b000be50 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81b00290b0 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 10 client.15189 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5624a0733c10 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81a809a380 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f81b0004500 con 0x5624a09cd550 2023-09-23T04:45:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 10 client.15189 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 20 client.15189 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a0a3aab0 0x5624a0a36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 4 client.15189 mds_command: new command op to 24577 tid=0[{"prefix": "status"}] 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.374+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5624a0918ae0 con 0x5624a0a3aab0 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f81bbfff700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a0a3aab0 0x5624a0a36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f819bfff700 10 client.15189 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f81bbfff700 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a0a3aab0 0x5624a0a36ef0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49f2c5889b6d122 server_cookie=ce1f54a11d603d75 in_seq=0 out_seq=0 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f819bfff700 1 -- 172.21.15.5:0/931048658 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5624a0918ae0 con 0x5624a0a3aab0 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f819bfff700 10 client.15189 handle_command_reply: tid=0 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.375+0000 7f81d6bf5b80 1 client.15189 shutdown 2023-09-23T04:45:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 msgr2=0x5624a09dcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b000bc30 comp rx=0 tx=0).stop 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 shutdown_connections 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5624a0a3aab0 0x5624a0a36ef0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5624a09ced50 0x5624a09dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624a09cd550 0x5624a09dcc30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f819bfff700 0 client.15189 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 --2- 172.21.15.5:0/931048658 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5624a09cdeb0 0x5624a09dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 >> 172.21.15.5:0/931048658 conn(0x5624a0941210 msgr2=0x5624a09c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 shutdown_connections 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 1 -- 172.21.15.5:0/931048658 wait complete. 2023-09-23T04:45:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.376+0000 7f81d6bf5b80 20 client.15189 trim_cache size 0 max 16384 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24577, 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 94.786787238000002, 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 107.321634225 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 msgr2=0x7f81ac0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 0x7f81ac0a3af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f81bc075230 tx=0x7f81b4009130 comp rx=0 tx=0).stop 2023-09-23T04:45:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 msgr2=0x7f81bc073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e970 tx=0x7f81a800ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 shutdown_connections 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f81ac0a1630 0x7f81ac0a3af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f81bc102a50 0x7f81bc073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81bc100eb0 0x7f81bc074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 --2- 172.21.15.5:0/1416289661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc102090 0x7f81bc073210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 >> 172.21.15.5:0/1416289661 conn(0x7f81bc0fc470 msgr2=0x7f81bc0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 shutdown_connections 2023-09-23T04:45:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:22.377+0000 7f81c31ff700 1 -- 172.21.15.5:0/1416289661 wait complete. 2023-09-23T04:45:22.395 INFO:tasks.cephadm:Setup complete, yielding 2023-09-23T04:45:22.395 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-23T04:45:22.405 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:45:22.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch status' 2023-09-23T04:45:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:22 smithi005 ceph-mon[97638]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:22 smithi118 ceph-mon[134297]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:22 smithi069 ceph-mon[133931]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/159446714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 msgr2=0x7f6b18104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 --2- 172.21.15.5:0/159446714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b18104740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0099c0 tx=0x7f6b0c009cd0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/159446714 shutdown_connections 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 --2- 172.21.15.5:0/159446714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 0x7f6b1810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 --2- 172.21.15.5:0/159446714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 --2- 172.21.15.5:0/159446714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b18104740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/159446714 >> 172.21.15.5:0/159446714 conn(0x7f6b180fe680 msgr2=0x7f6b18100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/159446714 shutdown_connections 2023-09-23T04:45:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.589+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/159446714 wait complete. 2023-09-23T04:45:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.590+0000 7f6b1e0cf700 1 Processor -- start 2023-09-23T04:45:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 -- start start 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 0x7f6b181a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181a08e0 con 0x7f6b181042e0 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181a0a20 con 0x7f6b18104c80 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b1e0cf700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181a0be0 con 0x7f6b181030e0 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b16ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b177fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b177fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49294/0 (socket says 172.21.15.5:49294) 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b177fe700 1 -- 172.21.15.5:0/3301376252 learned_addr learned my addr 172.21.15.5:0/3301376252 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.591+0000 7f6b17fff700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 0x7f6b181a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 -- 172.21.15.5:0/3301376252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 msgr2=0x7f6b18198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 -- 172.21.15.5:0/3301376252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 msgr2=0x7f6b181a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 0x7f6b181a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 -- 172.21.15.5:0/3301376252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0095f0 con 0x7f6b181042e0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b16ffd700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b181992d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01d070 con 0x7f6b181042e0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/3301376252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b181a0e60 con 0x7f6b181042e0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b0c005050 con 0x7f6b181042e0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.592+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004a70 con 0x7f6b181042e0 2023-09-23T04:45:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.593+0000 7f6b1e0cf700 1 -- 172.21.15.5:0/3301376252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b181a11c0 con 0x7f6b181042e0 2023-09-23T04:45:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.594+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1810e9a0 con 0x7f6b181042e0 2023-09-23T04:45:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.596+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f6b0c005450 con 0x7f6b181042e0 2023-09-23T04:45:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.597+0000 7f6b14ff9700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 0x7f6b000a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.597+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6b0c0cdf50 con 0x7f6b181042e0 2023-09-23T04:45:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.597+0000 7f6b177fe700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 0x7f6b000a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.598+0000 7f6b177fe700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 0x7f6b000a3a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b1819a270 tx=0x7f6b08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.601+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f6b0c097260 con 0x7f6b181042e0 2023-09-23T04:45:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.835+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b1810eb30 con 0x7f6b000a15a0 2023-09-23T04:45:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.836+0000 7f6b14ff9700 1 -- 172.21.15.5:0/3301376252 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6b1810eb30 con 0x7f6b000a15a0 2023-09-23T04:45:24.836 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2023-09-23T04:45:24.836 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2023-09-23T04:45:24.836 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2023-09-23T04:45:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 msgr2=0x7f6b000a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 0x7f6b000a3a60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b1819a270 tx=0x7f6b08009260 comp rx=0 tx=0).stop 2023-09-23T04:45:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 msgr2=0x7f6b181992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b181992d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c00f690 comp rx=0 tx=0).stop 2023-09-23T04:45:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 shutdown_connections 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6b000a15a0 0x7f6b000a3a60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b18104c80 0x7f6b181a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6b181030e0 0x7f6b18198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 --2- 172.21.15.5:0/3301376252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b181042e0 0x7f6b181992d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.838+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 >> 172.21.15.5:0/3301376252 conn(0x7f6b180fe680 msgr2=0x7f6b181008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.839+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 shutdown_connections 2023-09-23T04:45:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:24.839+0000 7f6afe7fc700 1 -- 172.21.15.5:0/3301376252 wait complete. 2023-09-23T04:45:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]: dispatch 2023-09-23T04:45:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-09-23T04:45:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:24 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2023-09-23T04:45:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]: dispatch 2023-09-23T04:45:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-09-23T04:45:25.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:24 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]: dispatch 2023-09-23T04:45:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]: dispatch 2023-09-23T04:45:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-09-23T04:45:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-09-23T04:45:25.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:24 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]: dispatch 2023-09-23T04:45:25.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-09-23T04:45:26.074 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T04:45:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:25 smithi005 ceph-mon[97638]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T04:45:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T04:45:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:25 smithi118 ceph-mon[134297]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T04:45:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2023-09-23T04:45:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-23T04:45:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7, 5, 6]}]': finished 2023-09-23T04:45:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-09-23T04:45:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 7, 4, 9]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 1, 2]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 10]}]': finished 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: osdmap e90: 12 total, 12 up, 12 in 2023-09-23T04:45:26.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:25 smithi069 ceph-mon[133931]: osdmap e91: 12 total, 12 up, 12 in 2023-09-23T04:45:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:26 smithi005 ceph-mon[97638]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:26 smithi005 ceph-mon[97638]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-23T04:45:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:26 smithi005 ceph-mon[97638]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T04:45:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:26 smithi118 ceph-mon[134297]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:27.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:26 smithi118 ceph-mon[134297]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-23T04:45:27.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:26 smithi118 ceph-mon[134297]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T04:45:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:26 smithi069 ceph-mon[133931]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:26 smithi069 ceph-mon[133931]: pgmap v321: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-23T04:45:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:26 smithi069 ceph-mon[133931]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 -- 172.21.15.5:0/2261001990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c100e60 msgr2=0x7fc86c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 --2- 172.21.15.5:0/2261001990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c100e60 0x7fc86c101280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009990 tx=0x7fc85c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 -- 172.21.15.5:0/2261001990 shutdown_connections 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 --2- 172.21.15.5:0/2261001990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102a10 0x7fc86c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 --2- 172.21.15.5:0/2261001990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c100e60 0x7fc86c101280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 --2- 172.21.15.5:0/2261001990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c102050 0x7fc86c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 -- 172.21.15.5:0/2261001990 >> 172.21.15.5:0/2261001990 conn(0x7fc86c0fc410 msgr2=0x7fc86c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:27.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 -- 172.21.15.5:0/2261001990 shutdown_connections 2023-09-23T04:45:27.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.600+0000 7fc8723bd700 1 -- 172.21.15.5:0/2261001990 wait complete. 2023-09-23T04:45:27.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.601+0000 7fc8723bd700 1 Processor -- start 2023-09-23T04:45:27.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.601+0000 7fc8723bd700 1 -- start start 2023-09-23T04:45:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.601+0000 7fc8723bd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.601+0000 7fc8723bd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 0x7fc86c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc8723bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc8723bd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c073ed0 con 0x7fc86c102a10 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc8723bd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c074040 con 0x7fc86c100e60 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc8723bd700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1a0c70 con 0x7fc86c102050 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 0x7fc86c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc87095a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc87095a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53532/0 (socket says 172.21.15.5:53532) 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33880/0 (socket says 172.21.15.5:33880) 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc87095a700 1 -- 172.21.15.5:0/1296928437 learned_addr learned my addr 172.21.15.5:0/1296928437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 0x7fc86c073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49320/0 (socket says 172.21.15.5:49320) 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 -- 172.21.15.5:0/1296928437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 msgr2=0x7fc86c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 0x7fc86c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 -- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 msgr2=0x7fc86c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc86bfff700 1 -- 172.21.15.5:0/1296928437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0095f0 con 0x7fc86c100e60 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.602+0000 7fc87095a700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc86bfff700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:27.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c01d070 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85c021410 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c1a0ef0 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c017650 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.603+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1a12a0 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.604+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c108550 con 0x7fc86c100e60 2023-09-23T04:45:27.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.606+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc85c021580 con 0x7fc86c100e60 2023-09-23T04:45:27.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.608+0000 7fc8697fa700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 0x7fc8540a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.608+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fc85c0cebd0 con 0x7fc86c100e60 2023-09-23T04:45:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.608+0000 7fc86b7fe700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 0x7fc8540a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:27.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.609+0000 7fc86b7fe700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 0x7fc8540a3ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc86c100b90 tx=0x7fc860009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:27.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.610+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc85c097e00 con 0x7fc86c100e60 2023-09-23T04:45:27.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.835+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc86c0008d0 con 0x7fc8540a1620 2023-09-23T04:45:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.849+0000 7fc8697fa700 1 -- 172.21.15.5:0/1296928437 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fc86c0008d0 con 0x7fc8540a1620 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (109s) - 109s 15.9M - 18.0.0-6336-g86266391 9b6b9c6d0d34 aec7687f445b 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi069 running (107s) - 107s 19.9M - 18.0.0-6336-g86266391 9b6b9c6d0d34 cf02c40952a7 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi118 running (2m) - 2m 20.0M - 18.0.0-6336-g86266391 9b6b9c6d0d34 dc0dea18b0f8 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (2m) - 2m 13.8M - 18.0.0-6336-g86266391 9b6b9c6d0d34 7ab79e94071c 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi069 running (2m) - 2m 16.8M - 18.0.0-6336-g86266391 9b6b9c6d0d34 47d826c520c6 2023-09-23T04:45:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi118 running (2m) - 2m 18.3M - 18.0.0-6336-g86266391 9b6b9c6d0d34 9dc1fcc9abff 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (113s) - 112s 30.0M - 18.0.0-6336-g86266391 9b6b9c6d0d34 62e76a10346a 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi069 running (118s) - 118s 18.1M - 18.0.0-6336-g86266391 9b6b9c6d0d34 f71571f16780 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi118 running (116s) - 115s 18.8M - 18.0.0-6336-g86266391 9b6b9c6d0d34 07775c8a2103 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) - 2m 13.9M - 18.0.0-6336-g86266391 9b6b9c6d0d34 c387557dd539 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi069 running (2m) - 2m 19.5M - 18.0.0-6336-g86266391 9b6b9c6d0d34 82e27fdc92c4 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi118 running (2m) - 2m 16.4M - 18.0.0-6336-g86266391 9b6b9c6d0d34 7aea8930cf85 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (8m) - 8m 526M - 18.0.0-6336-g86266391 9b6b9c6d0d34 7b302ee8c293 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi069 *:8443,8765 running (6m) - 6m 449M - 18.0.0-6336-g86266391 9b6b9c6d0d34 febd4f34e9f7 2023-09-23T04:45:27.850 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi118 *:8443,8765 running (7m) - 7m 450M - 18.0.0-6336-g86266391 9b6b9c6d0d34 c165a59d9a1a 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (8m) - 9m 93.6M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 a2c7341a82a4 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi069 running (7m) - 7m 65.5M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 001b0d2f62c2 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi118 running (7m) - 7m 64.4M 2048M 18.0.0-6336-g86266391 9b6b9c6d0d34 ec712ed13761 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (6m) - 6m 71.1M 4318M 18.0.0-6336-g86266391 9b6b9c6d0d34 23ac3611fd36 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi069 running (6m) - 6m 80.6M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 35198794a1ce 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi118 running (5m) - 5m 74.9M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 f670a7024c9f 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) - 5m 65.4M 4318M 18.0.0-6336-g86266391 9b6b9c6d0d34 a06d3c3c1f48 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi069 running (5m) - 5m 67.9M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 081cadf9990f 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi118 running (4m) - 4m 69.8M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 a15a84f16ac0 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) - 4m 61.9M 4318M 18.0.0-6336-g86266391 9b6b9c6d0d34 77844727c962 2023-09-23T04:45:27.851 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi069 running (4m) - 4m 69.1M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 a80c600dcd22 2023-09-23T04:45:27.852 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi118 running (3m) - 3m 64.1M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 4fcf1b5fccab 2023-09-23T04:45:27.852 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) - 3m 60.3M 4318M 18.0.0-6336-g86266391 9b6b9c6d0d34 18c381b37e26 2023-09-23T04:45:27.852 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi069 running (2m) - 2m 64.0M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 d82a89376ac6 2023-09-23T04:45:27.852 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi118 running (2m) - 2m 59.2M 4292M 18.0.0-6336-g86266391 9b6b9c6d0d34 e36f380e9f89 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 msgr2=0x7fc8540a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 0x7fc8540a3ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc86c100b90 tx=0x7fc860009260 comp rx=0 tx=0).stop 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 msgr2=0x7fc86c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c005740 comp rx=0 tx=0).stop 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 shutdown_connections 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc8540a1620 0x7fc8540a3ae0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc86c102a10 0x7fc86c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc86c100e60 0x7fc86c074bc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 --2- 172.21.15.5:0/1296928437 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc86c102050 0x7fc86c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 >> 172.21.15.5:0/1296928437 conn(0x7fc86c0fc410 msgr2=0x7fc86c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 shutdown_connections 2023-09-23T04:45:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:27.851+0000 7fc8723bd700 1 -- 172.21.15.5:0/1296928437 wait complete. 2023-09-23T04:45:28.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-09-23T04:45:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:28 smithi005 ceph-mon[97638]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 9 op/s 2023-09-23T04:45:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:28 smithi118 ceph-mon[134297]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 9 op/s 2023-09-23T04:45:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:28 smithi069 ceph-mon[133931]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 895 B/s wr, 9 op/s 2023-09-23T04:45:29.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:29 smithi005 ceph-mon[97638]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:29.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:29 smithi005 ceph-mon[97638]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-09-23T04:45:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:29 smithi118 ceph-mon[134297]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:29 smithi118 ceph-mon[134297]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-09-23T04:45:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:29 smithi069 ceph-mon[133931]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:29 smithi069 ceph-mon[133931]: pgmap v324: 129 pgs: 4 peering, 125 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-09-23T04:45:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 -- 172.21.15.5:0/1801725091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102050 msgr2=0x7f93a01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 --2- 172.21.15.5:0/1801725091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102050 0x7f93a01024d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93940098f0 tx=0x7f9394009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 -- 172.21.15.5:0/1801725091 shutdown_connections 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 --2- 172.21.15.5:0/1801725091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0102a10 0x7f93a0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 --2- 172.21.15.5:0/1801725091 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0100e60 0x7f93a0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 --2- 172.21.15.5:0/1801725091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102050 0x7f93a01024d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 -- 172.21.15.5:0/1801725091 >> 172.21.15.5:0/1801725091 conn(0x7f93a00fc410 msgr2=0x7f93a00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 -- 172.21.15.5:0/1801725091 shutdown_connections 2023-09-23T04:45:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.522+0000 7f93a6849700 1 -- 172.21.15.5:0/1801725091 wait complete. 2023-09-23T04:45:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.523+0000 7f93a6849700 1 Processor -- start 2023-09-23T04:45:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.523+0000 7f93a6849700 1 -- start start 2023-09-23T04:45:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 0x7f93a0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0102050 0x7f93a01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 0x7f93a01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a01a09f0 con 0x7f93a0100e60 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a01a0b30 con 0x7f93a0102a10 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a6849700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a01a0cc0 con 0x7f93a0102050 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f939ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 0x7f93a0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f939ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 0x7f93a0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53560/0 (socket says 172.21.15.5:53560) 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f939ffff700 1 -- 172.21.15.5:0/1246124669 learned_addr learned my addr 172.21.15.5:0/1246124669 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f93a4de6700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 0x7f93a01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.524+0000 7f939f7fe700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0102050 0x7f93a01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 -- 172.21.15.5:0/1246124669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0102050 msgr2=0x7f93a01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0102050 0x7f93a01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 -- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 msgr2=0x7f93a0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 0x7f93a0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 -- 172.21.15.5:0/1246124669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398009d00 con 0x7f93a0102a10 2023-09-23T04:45:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a4de6700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 0x7f93a01a0420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9398009cd0 tx=0x7f939800c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398017070 con 0x7f93a0102a10 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939800f040 con 0x7f93a0102a10 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9394009520 con 0x7f93a0102a10 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800e050 con 0x7f93a0102a10 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.525+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a01a1200 con 0x7f93a0102a10 2023-09-23T04:45:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.526+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0193520 con 0x7f93a0102a10 2023-09-23T04:45:30.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.528+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f939800cd10 con 0x7f93a0102a10 2023-09-23T04:45:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.529+0000 7f939d7fa700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 0x7f93880a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.530+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f93980ccc50 con 0x7f93a0102a10 2023-09-23T04:45:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.530+0000 7f939ffff700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 0x7f93880a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.530+0000 7f939ffff700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 0x7f93880a3a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9390005950 tx=0x7f9390009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:30.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.534+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9398095df0 con 0x7f93a0102a10 2023-09-23T04:45:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.752+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f93a002d230 con 0x7f93880a15a0 2023-09-23T04:45:30.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.757+0000 7f939d7fa700 1 -- 172.21.15.5:0/1246124669 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f93a002d230 con 0x7f93880a15a0 2023-09-23T04:45:30.757 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-23T04:45:30.757 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 - 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi069=b;smithi069=e;smithi069=h;smithi069=k;smithi118=c;smithi118=f;smithi118=i;smithi118=l;count:12 2023-09-23T04:45:30.757 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 - 7m smithi005=x;smithi069=y;smithi118=z;count:3 2023-09-23T04:45:30.758 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 - 7m smithi005:172.21.15.5=a;smithi069:172.21.15.69=b;smithi118:172.21.15.118=c;count:3 2023-09-23T04:45:30.758 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 - - 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 msgr2=0x7f93880a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 0x7f93880a3a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9390005950 tx=0x7f9390009260 comp rx=0 tx=0).stop 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 msgr2=0x7f93a01a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 0x7f93a01a0420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9398009cd0 tx=0x7f939800c620 comp rx=0 tx=0).stop 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 shutdown_connections 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93880a15a0 0x7f93880a3a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102a10 0x7f93a01a0420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93a0100e60 0x7f93a0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 --2- 172.21.15.5:0/1246124669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93a0102050 0x7f93a01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 >> 172.21.15.5:0/1246124669 conn(0x7f93a00fc410 msgr2=0x7f93a0107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 shutdown_connections 2023-09-23T04:45:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:30.760+0000 7f93a6849700 1 -- 172.21.15.5:0/1246124669 wait complete. 2023-09-23T04:45:31.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-09-23T04:45:31.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:31 smithi069 ceph-mon[133931]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:31 smithi005 ceph-mon[97638]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:31 smithi118 ceph-mon[134297]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:32.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:32 smithi005 ceph-mon[97638]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-09-23T04:45:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:32 smithi118 ceph-mon[134297]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-09-23T04:45:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:32 smithi069 ceph-mon[133931]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-09-23T04:45:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 -- 172.21.15.5:0/1900403184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102a10 msgr2=0x7fa034109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 --2- 172.21.15.5:0/1900403184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102a10 0x7fa034109330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00b0a0 tx=0x7fa02c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 -- 172.21.15.5:0/1900403184 shutdown_connections 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 --2- 172.21.15.5:0/1900403184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102a10 0x7fa034109330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 --2- 172.21.15.5:0/1900403184 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034100e60 0x7fa034101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 --2- 172.21.15.5:0/1900403184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034102050 0x7fa0341024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 -- 172.21.15.5:0/1900403184 >> 172.21.15.5:0/1900403184 conn(0x7fa0340fc410 msgr2=0x7fa0340fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 -- 172.21.15.5:0/1900403184 shutdown_connections 2023-09-23T04:45:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.349+0000 7fa03a594700 1 -- 172.21.15.5:0/1900403184 wait complete. 2023-09-23T04:45:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.350+0000 7fa03a594700 1 Processor -- start 2023-09-23T04:45:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.350+0000 7fa03a594700 1 -- start start 2023-09-23T04:45:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.350+0000 7fa03a594700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034100e60 0x7fa034196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa03a594700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa03a594700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034102a10 0x7fa03419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa03a594700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419e7e0 con 0x7fa034102050 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa03a594700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419e920 con 0x7fa034100e60 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa03a594700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03419eae0 con 0x7fa034102a10 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38568/0 (socket says 172.21.15.5:38568) 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa033fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034100e60 0x7fa034196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 -- 172.21.15.5:0/1364889867 learned_addr learned my addr 172.21.15.5:0/1364889867 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa038b31700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034102a10 0x7fa03419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 -- 172.21.15.5:0/1364889867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034102a10 msgr2=0x7fa03419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034102a10 0x7fa03419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 -- 172.21.15.5:0/1364889867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034100e60 msgr2=0x7fa034196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034100e60 0x7fa034196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.351+0000 7fa0337fe700 1 -- 172.21.15.5:0/1364889867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c009d00 con 0x7fa034102050 2023-09-23T04:45:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa0337fe700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa02800d980 tx=0x7fa02800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800f890 con 0x7fa034102050 2023-09-23T04:45:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03419edc0 con 0x7fa034102050 2023-09-23T04:45:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa028010460 con 0x7fa034102050 2023-09-23T04:45:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280195c0 con 0x7fa034102050 2023-09-23T04:45:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.352+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03419f260 con 0x7fa034102050 2023-09-23T04:45:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.353+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa034107e20 con 0x7fa034102050 2023-09-23T04:45:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.355+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa0280105d0 con 0x7fa034102050 2023-09-23T04:45:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.356+0000 7fa0317fa700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 0x7fa01c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.357+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fa0280ccfe0 con 0x7fa034102050 2023-09-23T04:45:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.357+0000 7fa033fff700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 0x7fa01c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.357+0000 7fa033fff700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 0x7fa01c0a3a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa034197b40 tx=0x7fa024009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.361+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa0280961c0 con 0x7fa034102050 2023-09-23T04:45:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.577+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa03402d290 con 0x7fa01c0a15a0 2023-09-23T04:45:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.578+0000 7fa0317fa700 1 -- 172.21.15.5:0/1364889867 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fa03402d290 con 0x7fa01c0a15a0 2023-09-23T04:45:33.578 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2023-09-23T04:45:33.578 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2023-09-23T04:45:33.579 INFO:teuthology.orchestra.run.smithi005.stdout:smithi069 172.21.15.69 2023-09-23T04:45:33.579 INFO:teuthology.orchestra.run.smithi005.stdout:smithi118 172.21.15.118 2023-09-23T04:45:33.579 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2023-09-23T04:45:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.580+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 msgr2=0x7fa01c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.580+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 0x7fa01c0a3a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa034197b40 tx=0x7fa024009260 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.580+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 msgr2=0x7fa0341971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.580+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa02800d980 tx=0x7fa02800dc90 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 shutdown_connections 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa01c0a15a0 0x7fa01c0a3a60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa034102a10 0x7fa03419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa034100e60 0x7fa034196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 --2- 172.21.15.5:0/1364889867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa034102050 0x7fa0341971a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 >> 172.21.15.5:0/1364889867 conn(0x7fa0340fc410 msgr2=0x7fa0340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 shutdown_connections 2023-09-23T04:45:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:33.581+0000 7fa03a594700 1 -- 172.21.15.5:0/1364889867 wait complete. 2023-09-23T04:45:33.754 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:33 smithi005 ceph-mon[97638]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-23T04:45:33.754 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:33 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T04:45:33.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:33 smithi118 ceph-mon[134297]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-23T04:45:33.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:33 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T04:45:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:33 smithi069 ceph-mon[133931]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-09-23T04:45:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:33 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T04:45:34.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-09-23T04:45:34.771 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:34.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:34 smithi005 ceph-mon[97638]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:34.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:34 smithi005 ceph-mon[97638]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 346 B/s wr, 9 op/s 2023-09-23T04:45:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:34 smithi118 ceph-mon[134297]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:34 smithi118 ceph-mon[134297]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 346 B/s wr, 9 op/s 2023-09-23T04:45:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:34 smithi069 ceph-mon[133931]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:34 smithi069 ceph-mon[133931]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 346 B/s wr, 9 op/s 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 -- 172.21.15.5:0/1002086728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec102050 msgr2=0x7f5cec1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/1002086728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec102050 0x7f5cec1024d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009990 tx=0x7f5ce0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 -- 172.21.15.5:0/1002086728 shutdown_connections 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/1002086728 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/1002086728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec100e60 0x7f5cec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/1002086728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec102050 0x7f5cec1024d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 -- 172.21.15.5:0/1002086728 >> 172.21.15.5:0/1002086728 conn(0x7f5cec0fc410 msgr2=0x7f5cec0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 -- 172.21.15.5:0/1002086728 shutdown_connections 2023-09-23T04:45:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.274+0000 7f5cf1f01700 1 -- 172.21.15.5:0/1002086728 wait complete. 2023-09-23T04:45:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.275+0000 7f5cf1f01700 1 Processor -- start 2023-09-23T04:45:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.275+0000 7f5cf1f01700 1 -- start start 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 0x7f5cec199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a0950 con 0x7f5cec100e60 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a0a90 con 0x7f5cec102050 2023-09-23T04:45:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cf1f01700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a0c20 con 0x7f5cec102a10 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38576/0 (socket says 172.21.15.5:38576) 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/3053192193 learned_addr learned my addr 172.21.15.5:0/3053192193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceaffd700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 0x7f5cec199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cebfff700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/3053192193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 msgr2=0x7f5cec1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/3053192193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 msgr2=0x7f5cec199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 0x7f5cec199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/3053192193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce00095f0 con 0x7f5cec100e60 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5ceaffd700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 0x7f5cec199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.276+0000 7f5cebfff700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5ceb7fe700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00e9a0 tx=0x7f5cdc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc009790 con 0x7f5cec100e60 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cdc00c810 con 0x7f5cec100e60 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc003c70 con 0x7f5cec100e60 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1a0f00 con 0x7f5cec100e60 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.277+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1a1310 con 0x7f5cec100e60 2023-09-23T04:45:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.278+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec10c6e0 con 0x7f5cec100e60 2023-09-23T04:45:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.280+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f5cdc00cc80 con 0x7f5cec100e60 2023-09-23T04:45:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.282+0000 7f5ce8ff9700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 0x7f5cd40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.282+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f5cdc014070 con 0x7f5cec100e60 2023-09-23T04:45:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.282+0000 7f5ceaffd700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 0x7f5cd40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.282+0000 7f5ceaffd700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 0x7f5cd40a3ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b2e0 tx=0x7f5ce0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.286+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5cdc0965a0 con 0x7f5cec100e60 2023-09-23T04:45:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.507+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cec108a80 con 0x7f5cd40a1620 2023-09-23T04:45:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.509+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3053192193 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f5cec108a80 con 0x7f5cd40a1620 2023-09-23T04:45:36.509 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-23T04:45:36.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-09-23T04:45:36.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-23T04:45:36.509 INFO:teuthology.orchestra.run.smithi005.stdout:smithi118 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500JS480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-09-23T04:45:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 msgr2=0x7f5cd40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 0x7f5cd40a3ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b2e0 tx=0x7f5ce0005960 comp rx=0 tx=0).stop 2023-09-23T04:45:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 msgr2=0x7f5cec198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00e9a0 tx=0x7f5cdc00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 shutdown_connections 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5cd40a1620 0x7f5cd40a3ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5cec102a10 0x7f5cec1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cec100e60 0x7f5cec198dc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 --2- 172.21.15.5:0/3053192193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5cec102050 0x7f5cec199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.511+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 >> 172.21.15.5:0/3053192193 conn(0x7f5cec0fc410 msgr2=0x7f5cec0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.512+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 shutdown_connections 2023-09-23T04:45:36.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:36.512+0000 7f5cf1f01700 1 -- 172.21.15.5:0/3053192193 wait complete. 2023-09-23T04:45:36.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:36 smithi069 ceph-mon[133931]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T04:45:36.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:36 smithi005 ceph-mon[97638]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T04:45:36.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:36 smithi118 ceph-mon[134297]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T04:45:36.976 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-23T04:45:36.990 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:45:36.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-09-23T04:45:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:37.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:37 smithi069 ceph-mon[133931]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:37.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:37 smithi005 ceph-mon[97638]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:37.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:37 smithi118 ceph-mon[134297]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:45:38.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:38 smithi005 ceph-mon[97638]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 172 B/s wr, 9 op/s 2023-09-23T04:45:38.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:38 smithi118 ceph-mon[134297]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 172 B/s wr, 9 op/s 2023-09-23T04:45:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:38 smithi069 ceph-mon[133931]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 172 B/s wr, 9 op/s 2023-09-23T04:45:39.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/1151176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 msgr2=0x7fa6a0106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/1151176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a0106660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa690009990 tx=0x7fa690009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/1151176071 shutdown_connections 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/1151176071 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0106ba0 0x7fa6a0109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/1151176071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0101940 0x7fa6a0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/1151176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a0106660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/1151176071 >> 172.21.15.5:0/1151176071 conn(0x7fa6a00fb330 msgr2=0x7fa6a00fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/1151176071 shutdown_connections 2023-09-23T04:45:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.245+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/1151176071 wait complete. 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.246+0000 7fa6a7a3a700 1 Processor -- start 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.246+0000 7fa6a7a3a700 1 -- start start 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0101940 0x7fa6a0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0106ba0 0x7fa6a01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01a08f0 con 0x7fa6a0104270 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01a0a30 con 0x7fa6a0106ba0 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a7a3a700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01a0bf0 con 0x7fa6a0101940 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a4fd5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a4fd5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38598/0 (socket says 172.21.15.5:38598) 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a4fd5700 1 -- 172.21.15.5:0/3692617259 learned_addr learned my addr 172.21.15.5:0/3692617259 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:39.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a5fd7700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0106ba0 0x7fa6a01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.247+0000 7fa6a57d6700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0101940 0x7fa6a0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 -- 172.21.15.5:0/3692617259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0101940 msgr2=0x7fa6a0198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0101940 0x7fa6a0198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 -- 172.21.15.5:0/3692617259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0106ba0 msgr2=0x7fa6a01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0106ba0 0x7fa6a01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 -- 172.21.15.5:0/3692617259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900095f0 con 0x7fa6a0104270 2023-09-23T04:45:39.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a4fd5700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa690005910 tx=0x7fa69000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69001d070 con 0x7fa6a0104270 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa690022470 con 0x7fa6a0104270 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a01a0e70 con 0x7fa6a0104270 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.248+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690017650 con 0x7fa6a0104270 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.249+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01a1220 con 0x7fa6a0104270 2023-09-23T04:45:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.249+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a0108770 con 0x7fa6a0104270 2023-09-23T04:45:39.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.251+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fa69000fad0 con 0x7fa6a0104270 2023-09-23T04:45:39.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.253+0000 7fa6967fc700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 0x7fa68c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:39.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.253+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fa6900ce600 con 0x7fa6a0104270 2023-09-23T04:45:39.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.253+0000 7fa6a57d6700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 0x7fa68c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:39.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.254+0000 7fa6a57d6700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 0x7fa68c0a3ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6a019a280 tx=0x7fa69c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:39.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.256+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa690097830 con 0x7fa6a0104270 2023-09-23T04:45:39.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.513+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6a010c720 con 0x7fa6a0104270 2023-09-23T04:45:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.513+0000 7fa6967fc700 1 -- 172.21.15.5:0/3692617259 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2961 (secure 0 0 0) 0x7fa69009ae50 con 0x7fa6a0104270 2023-09-23T04:45:39.514 INFO:teuthology.orchestra.run.smithi005.stdout:e27 2023-09-23T04:45:39.514 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-23T04:45:39.514 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 27 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-09-23T04:43:47.148877+0000 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2023-09-23T04:44:20.444513+0000 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2023-09-23T04:45:39.515 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24577,1=24559,2=24554,3=24535,4=24529} 2023-09-23T04:45:39.516 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3,4] 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{0:24577} state up:active seq 5 addr [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{0:24506} state up:standby-replay seq 1 addr [v2:172.21.15.69:6836/2958173168,v1:172.21.15.69:6837/2958173168] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{1:24559} state up:active seq 13 addr [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.517 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{1:24482} state up:standby-replay seq 1 addr [v2:172.21.15.69:6834/3966429650,v1:172.21.15.69:6835/3966429650] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{2:24554} state up:active seq 11 addr [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{2:24469} state up:standby-replay seq 1 addr [v2:172.21.15.118:6832/3707604246,v1:172.21.15.118:6833/3707604246] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{3:24535} state up:active seq 14 addr [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{3:24443} state up:standby-replay seq 1 addr [v2:172.21.15.69:6832/827181928,v1:172.21.15.69:6833/827181928] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{4:24529} state up:active seq 15 addr [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{4:14538} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/1890134322,v1:172.21.15.5:6841/1890134322] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:39.518 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14457} state up:standby seq 1 addr [v2:172.21.15.5:6834/3475689025,v1:172.21.15.5:6835/3475689025] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.519 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14469} state up:standby seq 1 addr [v2:172.21.15.5:6836/3951415099,v1:172.21.15.5:6837/3951415099] compat {c=[1],r=[1],i=[fff]}] 2023-09-23T04:45:39.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 msgr2=0x7fa68c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:39.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 0x7fa68c0a3ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6a019a280 tx=0x7fa69c00b410 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 msgr2=0x7fa6a01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa690005910 tx=0x7fa69000f690 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 shutdown_connections 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fa68c0a1620 0x7fa68c0a3ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6a0106ba0 0x7fa6a01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa6a0101940 0x7fa6a0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 --2- 172.21.15.5:0/3692617259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a0104270 0x7fa6a01992e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 >> 172.21.15.5:0/3692617259 conn(0x7fa6a00fb330 msgr2=0x7fa6a00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 shutdown_connections 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:39.516+0000 7fa6a7a3a700 1 -- 172.21.15.5:0/3692617259 wait complete. 2023-09-23T04:45:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:39.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:39 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3692617259' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T04:45:39.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:39 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3692617259' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T04:45:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:39 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3692617259' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-23T04:45:40.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-09-23T04:45:40.761 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:40 smithi005 ceph-mon[97638]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2023-09-23T04:45:40.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:40 smithi118 ceph-mon[134297]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2023-09-23T04:45:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:40 smithi069 ceph-mon[133931]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 11 op/s 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 -- 172.21.15.5:0/4072354076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94074df0 msgr2=0x7fca94073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 --2- 172.21.15.5:0/4072354076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94074df0 0x7fca94073250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca90009990 tx=0x7fca90009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 -- 172.21.15.5:0/4072354076 shutdown_connections 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 --2- 172.21.15.5:0/4072354076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108e00 0x7fca9410b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 --2- 172.21.15.5:0/4072354076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94074df0 0x7fca94073250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 --2- 172.21.15.5:0/4072354076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94073820 0x7fca94073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 -- 172.21.15.5:0/4072354076 >> 172.21.15.5:0/4072354076 conn(0x7fca940fc4a0 msgr2=0x7fca940fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 -- 172.21.15.5:0/4072354076 shutdown_connections 2023-09-23T04:45:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.303+0000 7fca9c571700 1 -- 172.21.15.5:0/4072354076 wait complete. 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.304+0000 7fca9c571700 1 Processor -- start 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 -- start start 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94074df0 0x7fca94199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94108e00 0x7fca941a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca941a09e0 con 0x7fca94073820 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca941a0b20 con 0x7fca94074df0 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9c571700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca941a0cb0 con 0x7fca94108e00 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9a30d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9a30d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50496/0 (socket says 172.21.15.5:50496) 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9a30d700 1 -- 172.21.15.5:0/3004328711 learned_addr learned my addr 172.21.15.5:0/3004328711 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca99b0c700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94074df0 0x7fca94199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.305+0000 7fca9ab0e700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94108e00 0x7fca941a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 -- 172.21.15.5:0/3004328711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94108e00 msgr2=0x7fca941a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94108e00 0x7fca941a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 -- 172.21.15.5:0/3004328711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94074df0 msgr2=0x7fca94199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94074df0 0x7fca94199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 -- 172.21.15.5:0/3004328711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900095f0 con 0x7fca94073820 2023-09-23T04:45:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9a30d700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c010 tx=0x7fca9000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9001d070 con 0x7fca94073820 2023-09-23T04:45:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca941a0f30 con 0x7fca94073820 2023-09-23T04:45:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca90022470 con 0x7fca94073820 2023-09-23T04:45:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.306+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90017650 con 0x7fca94073820 2023-09-23T04:45:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.307+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca941a12e0 con 0x7fca94073820 2023-09-23T04:45:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.308+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca94107500 con 0x7fca94073820 2023-09-23T04:45:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.309+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fca900225e0 con 0x7fca94073820 2023-09-23T04:45:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.311+0000 7fca8b7fe700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 0x7fca800a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.311+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fca900ce500 con 0x7fca94073820 2023-09-23T04:45:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.311+0000 7fca99b0c700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 0x7fca800a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.312+0000 7fca99b0c700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 0x7fca800a3ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca84005950 tx=0x7fca8400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.315+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fca90097730 con 0x7fca94073820 2023-09-23T04:45:42.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.527+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fca9410e950 con 0x7fca94073820 2023-09-23T04:45:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.528+0000 7fca8b7fe700 1 -- 172.21.15.5:0/3004328711 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5755 (secure 0 0 0) 0x7fca9009ad50 con 0x7fca94073820 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 91 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-09-23T04:36:25.203666+0000 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2023-09-23T04:45:25.785139+0000 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 27 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2023-09-23T04:45:42.529 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release reef 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/2285726296,v1:172.21.15.5:6803/2285726296] [v2:172.21.15.5:6804/2285726296,v1:172.21.15.5:6805/2285726296] exists,up 35700daf-4df6-4103-ade1-12999750ad33 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6800/286933964,v1:172.21.15.69:6801/286933964] [v2:172.21.15.69:6802/286933964,v1:172.21.15.69:6803/286933964] exists,up d0d9aada-e5c2-4198-8a8e-a9db453fd897 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6800/275897405,v1:172.21.15.118:6801/275897405] [v2:172.21.15.118:6802/275897405,v1:172.21.15.118:6803/275897405] exists,up 101be3f4-34d5-43eb-a131-2144d5a0f309 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/4155254124,v1:172.21.15.5:6811/4155254124] [v2:172.21.15.5:6812/4155254124,v1:172.21.15.5:6813/4155254124] exists,up b08d036a-cfc0-424d-963b-fc5fbeb8af79 2023-09-23T04:45:42.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6808/3327136642,v1:172.21.15.69:6809/3327136642] [v2:172.21.15.69:6810/3327136642,v1:172.21.15.69:6811/3327136642] exists,up 03c71626-c985-43df-92a1-aa005836c3cf 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6808/2643723046,v1:172.21.15.118:6809/2643723046] [v2:172.21.15.118:6810/2643723046,v1:172.21.15.118:6811/2643723046] exists,up 35af4018-9f6b-4950-85d4-4af32e09ef2a 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/1080291272,v1:172.21.15.5:6819/1080291272] [v2:172.21.15.5:6820/1080291272,v1:172.21.15.5:6821/1080291272] exists,up 3a9d6647-95c6-4cf0-ac70-52cb2639c1b4 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6816/1999230066,v1:172.21.15.69:6817/1999230066] [v2:172.21.15.69:6818/1999230066,v1:172.21.15.69:6819/1999230066] exists,up 496c61c7-6762-44b0-8361-bc62e51fb6cb 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 58 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6816/1078105960,v1:172.21.15.118:6817/1078105960] [v2:172.21.15.118:6818/1078105960,v1:172.21.15.118:6819/1078105960] exists,up 3aea8c15-9de3-4519-9f86-93b891d108e7 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 66 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/2457675866,v1:172.21.15.5:6827/2457675866] [v2:172.21.15.5:6828/2457675866,v1:172.21.15.5:6829/2457675866] exists,up 463a1b28-e1f2-485e-a11c-16f788e17414 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 71 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6824/4220981201,v1:172.21.15.69:6825/4220981201] [v2:172.21.15.69:6826/4220981201,v1:172.21.15.69:6827/4220981201] exists,up 33ab3d09-beb2-4736-99b1-554ad9d15218 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 78 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.118:6824/1838374502,v1:172.21.15.118:6825/1838374502] [v2:172.21.15.118:6826/1838374502,v1:172.21.15.118:6827/1838374502] exists,up 6354fc3d-d3ae-491d-8592-1b9bb4336d15 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.2 [7,11] 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.b [2,1] 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.f [8,11] 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.10 [2,4] 2023-09-23T04:45:42.531 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1a [8,4] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.0 [4,8] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.10 [5,9] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,7,5,6] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.18 [3,9] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1b [3,7,4,9] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1d [4,8,1,2] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.27 [4,6] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.28 [5,8] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.b [9,8] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.e [5,8] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.14 [9,8] 2023-09-23T04:45:42.532 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.1c [9,10] 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3461224331 expires 2023-09-24T04:37:23.553088+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/1973270664 expires 2023-09-24T04:37:23.553088+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/1973270664 expires 2023-09-24T04:37:23.553088+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1525019998 expires 2023-09-24T04:37:23.553088+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/774964976 expires 2023-09-24T04:36:56.816377+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2022747688 expires 2023-09-24T04:36:56.816377+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2627772518 expires 2023-09-24T04:36:56.816377+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3431143820 expires 2023-09-24T04:36:56.816377+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/774964976 expires 2023-09-24T04:36:56.816377+0000 2023-09-23T04:45:42.533 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1934869339 expires 2023-09-24T04:37:23.553088+0000 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 msgr2=0x7fca800a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 0x7fca800a3ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca84005950 tx=0x7fca8400b410 comp rx=0 tx=0).stop 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 msgr2=0x7fca94198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c010 tx=0x7fca9000f690 comp rx=0 tx=0).stop 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 shutdown_connections 2023-09-23T04:45:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fca800a1620 0x7fca800a3ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94108e00 0x7fca941a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94073820 0x7fca94198e50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 --2- 172.21.15.5:0/3004328711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94074df0 0x7fca94199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.530+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 >> 172.21.15.5:0/3004328711 conn(0x7fca940fc4a0 msgr2=0x7fca940fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.531+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 shutdown_connections 2023-09-23T04:45:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:42.531+0000 7fca9c571700 1 -- 172.21.15.5:0/3004328711 wait complete. 2023-09-23T04:45:42.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:42 smithi005 ceph-mon[97638]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:42 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3004328711' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T04:45:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:42 smithi069 ceph-mon[133931]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:42 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3004328711' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T04:45:43.059 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-23T04:45:43.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:42 smithi118 ceph-mon[134297]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:42 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3004328711' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-23T04:45:43.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 -- 172.21.15.5:0/3203722035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102080 msgr2=0x7fb654102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/3203722035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102080 0x7fb654102500 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb644009990 tx=0x7fb644009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 -- 172.21.15.5:0/3203722035 shutdown_connections 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/3203722035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654102a40 0x7fb654109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/3203722035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654100ea0 0x7fb6541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/3203722035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102080 0x7fb654102500 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 -- 172.21.15.5:0/3203722035 >> 172.21.15.5:0/3203722035 conn(0x7fb6540fc480 msgr2=0x7fb6540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 -- 172.21.15.5:0/3203722035 shutdown_connections 2023-09-23T04:45:43.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.301+0000 7fb65b9ff700 1 -- 172.21.15.5:0/3203722035 wait complete. 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.302+0000 7fb65b9ff700 1 Processor -- start 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.302+0000 7fb65b9ff700 1 -- start start 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.302+0000 7fb65b9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654102080 0x7fb654199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a0990 con 0x7fb654102a40 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a0ad0 con 0x7fb654102080 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65b9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541a0c60 con 0x7fb654100ea0 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb659f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb659f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50522/0 (socket says 172.21.15.5:50522) 2023-09-23T04:45:43.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb659f9c700 1 -- 172.21.15.5:0/360442189 learned_addr learned my addr 172.21.15.5:0/360442189 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55088/0 (socket says 172.21.15.5:55088) 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb658f9a700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654102080 0x7fb654199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 -- 172.21.15.5:0/360442189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654102080 msgr2=0x7fb654199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654102080 0x7fb654199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 -- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 msgr2=0x7fb6541a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb65979b700 1 -- 172.21.15.5:0/360442189 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440095f0 con 0x7fb654100ea0 2023-09-23T04:45:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.303+0000 7fb659f9c700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb65979b700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d720 tx=0x7fb65000da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000f690 con 0x7fb654100ea0 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6541a0f40 con 0x7fb654100ea0 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb65000fcb0 con 0x7fb654100ea0 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650010e60 con 0x7fb654100ea0 2023-09-23T04:45:43.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.304+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541a1370 con 0x7fb654100ea0 2023-09-23T04:45:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.305+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65410c760 con 0x7fb654100ea0 2023-09-23T04:45:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.307+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fb65000f7f0 con 0x7fb654100ea0 2023-09-23T04:45:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.309+0000 7fb64a7fc700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 0x7fb6400a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.309+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fb6500cce60 con 0x7fb654100ea0 2023-09-23T04:45:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.309+0000 7fb658f9a700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 0x7fb6400a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.309+0000 7fb658f9a700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 0x7fb6400a3ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb64400b2e0 tx=0x7fb644005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.311+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb650096000 con 0x7fb654100ea0 2023-09-23T04:45:43.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.523+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb65410c970 con 0x7fb654100ea0 2023-09-23T04:45:43.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.526+0000 7fb64a7fc700 1 -- 172.21.15.5:0/360442189 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7fb650014090 con 0x7fb654100ea0 2023-09-23T04:45:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:43.527 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:43.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 msgr2=0x7fb6400a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 0x7fb6400a3ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb64400b2e0 tx=0x7fb644005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 msgr2=0x7fb654198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb65000d720 tx=0x7fb65000da30 comp rx=0 tx=0).stop 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 shutdown_connections 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb6400a1620 0x7fb6400a3ae0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb654102a40 0x7fb6541a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb654100ea0 0x7fb654198e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 --2- 172.21.15.5:0/360442189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb654102080 0x7fb654199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 >> 172.21.15.5:0/360442189 conn(0x7fb6540fc480 msgr2=0x7fb6540fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 shutdown_connections 2023-09-23T04:45:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.528+0000 7fb65b9ff700 1 -- 172.21.15.5:0/360442189 wait complete. 2023-09-23T04:45:43.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:43.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:43.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 -- 172.21.15.5:0/817511672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102060 msgr2=0x7f943c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 --2- 172.21.15.5:0/817511672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102060 0x7f943c1024e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f942c009990 tx=0x7f942c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 -- 172.21.15.5:0/817511672 shutdown_connections 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 --2- 172.21.15.5:0/817511672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c102a20 0x7f943c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 --2- 172.21.15.5:0/817511672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c100f10 0x7f943c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 --2- 172.21.15.5:0/817511672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102060 0x7f943c1024e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.759+0000 7f94439ff700 1 -- 172.21.15.5:0/817511672 >> 172.21.15.5:0/817511672 conn(0x7f943c0fc480 msgr2=0x7f943c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.760+0000 7f94439ff700 1 -- 172.21.15.5:0/817511672 shutdown_connections 2023-09-23T04:45:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.760+0000 7f94439ff700 1 -- 172.21.15.5:0/817511672 wait complete. 2023-09-23T04:45:43.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 Processor -- start 2023-09-23T04:45:43.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 -- start start 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c100f10 0x7f943c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c1a0950 con 0x7f943c102a20 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c1a0a90 con 0x7f943c102060 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f94439ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c1a0c20 con 0x7f943c100f10 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f9441f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f9440f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f9440f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:51968/0 (socket says 172.21.15.5:51968) 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f9441f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50544/0 (socket says 172.21.15.5:50544) 2023-09-23T04:45:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f9440f9a700 1 -- 172.21.15.5:0/432671228 learned_addr learned my addr 172.21.15.5:0/432671228 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.761+0000 7f944179b700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c100f10 0x7f943c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 -- 172.21.15.5:0/432671228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c100f10 msgr2=0x7f943c198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c100f10 0x7f943c198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 -- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 msgr2=0x7f943c1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 -- 172.21.15.5:0/432671228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0095f0 con 0x7f943c102060 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9441f9c700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f9440f9a700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f942c004760 tx=0x7f942c004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c01d070 con 0x7f943c102060 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f942c004830 con 0x7f943c102060 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c1a0ea0 con 0x7f943c102060 2023-09-23T04:45:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.762+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c005350 con 0x7f943c102060 2023-09-23T04:45:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.763+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c1a1210 con 0x7f943c102060 2023-09-23T04:45:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.763+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c10c760 con 0x7f943c102060 2023-09-23T04:45:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.766+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f942c0054d0 con 0x7f943c102060 2023-09-23T04:45:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.767+0000 7f94327fc700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 0x7f94280a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.767+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f942c0ce950 con 0x7f943c102060 2023-09-23T04:45:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.767+0000 7f944179b700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 0x7f94280a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.768+0000 7f944179b700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 0x7f94280a3a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9438009c60 tx=0x7f9438009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.769+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f942c097b80 con 0x7f943c102060 2023-09-23T04:45:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:43 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/360442189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:43.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.980+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f943c1a1460 con 0x7f943c102060 2023-09-23T04:45:43.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.981+0000 7f94327fc700 1 -- 172.21.15.5:0/432671228 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f942c09b1a0 con 0x7f943c102060 2023-09-23T04:45:43.982 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 msgr2=0x7f94280a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 0x7f94280a3a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9438009c60 tx=0x7f9438009260 comp rx=0 tx=0).stop 2023-09-23T04:45:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 msgr2=0x7f943c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f942c004760 tx=0x7f942c004790 comp rx=0 tx=0).stop 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 shutdown_connections 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f94280a15a0 0x7f94280a3a60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f943c102a20 0x7f943c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f943c100f10 0x7f943c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 --2- 172.21.15.5:0/432671228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943c102060 0x7f943c199310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 >> 172.21.15.5:0/432671228 conn(0x7f943c0fc480 msgr2=0x7f943c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 shutdown_connections 2023-09-23T04:45:43.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:43.983+0000 7f94439ff700 1 -- 172.21.15.5:0/432671228 wait complete. 2023-09-23T04:45:43.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:43 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/360442189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:44.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:43 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/360442189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 -- 172.21.15.5:0/1006886138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc074e60 msgr2=0x7ff2cc0732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/1006886138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc074e60 0x7ff2cc0732b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009990 tx=0x7ff2c8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 -- 172.21.15.5:0/1006886138 shutdown_connections 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/1006886138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc10b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/1006886138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc074e60 0x7ff2cc0732b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/1006886138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc073880 0x7ff2cc073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 -- 172.21.15.5:0/1006886138 >> 172.21.15.5:0/1006886138 conn(0x7ff2cc0fc470 msgr2=0x7ff2cc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 -- 172.21.15.5:0/1006886138 shutdown_connections 2023-09-23T04:45:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.209+0000 7ff2d3bff700 1 -- 172.21.15.5:0/1006886138 wait complete. 2023-09-23T04:45:44.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 Processor -- start 2023-09-23T04:45:44.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 -- start start 2023-09-23T04:45:44.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc073880 0x7ff2cc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc074e60 0x7ff2cc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.210+0000 7ff2d3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a09b0 con 0x7ff2cc108dc0 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d3bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a0af0 con 0x7ff2cc073880 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d3bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a0c80 con 0x7ff2cc074e60 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50550/0 (socket says 172.21.15.5:50550) 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 -- 172.21.15.5:0/299272518 learned_addr learned my addr 172.21.15.5:0/299272518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:44.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d119a700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc074e60 0x7ff2cc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d199b700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc073880 0x7ff2cc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 -- 172.21.15.5:0/299272518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc074e60 msgr2=0x7ff2cc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc074e60 0x7ff2cc199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 -- 172.21.15.5:0/299272518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc073880 msgr2=0x7ff2cc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc073880 0x7ff2cc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 -- 172.21.15.5:0/299272518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80095f0 con 0x7ff2cc108dc0 2023-09-23T04:45:44.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.211+0000 7ff2d219c700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ef30 tx=0x7ff2c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.212+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4018070 con 0x7ff2cc108dc0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.212+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c400f040 con 0x7ff2cc108dc0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.212+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc1a0f00 con 0x7ff2cc108dc0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.212+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400cd00 con 0x7ff2cc108dc0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.212+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc1a1380 con 0x7ff2cc108dc0 2023-09-23T04:45:44.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.213+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc107470 con 0x7ff2cc108dc0 2023-09-23T04:45:44.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.215+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7ff2c4003a00 con 0x7ff2cc108dc0 2023-09-23T04:45:44.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.216+0000 7ff2c2ffd700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 0x7ff2b80a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.216+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7ff2c40ccdf0 con 0x7ff2cc108dc0 2023-09-23T04:45:44.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.217+0000 7ff2d199b700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 0x7ff2b80a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.217+0000 7ff2d199b700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 0x7ff2b80a3ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b2e0 tx=0x7ff2c8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:44.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.220+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff2c4096020 con 0x7ff2cc108dc0 2023-09-23T04:45:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.431+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2cc02d1a0 con 0x7ff2cc108dc0 2023-09-23T04:45:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.431+0000 7ff2c2ffd700 1 -- 172.21.15.5:0/299272518 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2c4099640 con 0x7ff2cc108dc0 2023-09-23T04:45:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:44.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 msgr2=0x7ff2b80a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 0x7ff2b80a3ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b2e0 tx=0x7ff2c8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 msgr2=0x7ff2cc1a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ef30 tx=0x7ff2c400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 shutdown_connections 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7ff2b80a1620 0x7ff2b80a3ae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2cc108dc0 0x7ff2cc1a03e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2cc073880 0x7ff2cc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 --2- 172.21.15.5:0/299272518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff2cc074e60 0x7ff2cc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.433+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 >> 172.21.15.5:0/299272518 conn(0x7ff2cc0fc470 msgr2=0x7ff2cc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.434+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 shutdown_connections 2023-09-23T04:45:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.434+0000 7ff2d3bff700 1 -- 172.21.15.5:0/299272518 wait complete. 2023-09-23T04:45:44.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.658+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2062754402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024100eb0 msgr2=0x7fe0241012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.658+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2062754402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024100eb0 0x7fe0241012d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe020009990 tx=0x7fe020009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2062754402 shutdown_connections 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2062754402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102a50 0x7fe024109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2062754402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024100eb0 0x7fe0241012d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2062754402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024102090 0x7fe024102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2062754402 >> 172.21.15.5:0/2062754402 conn(0x7fe0240fc470 msgr2=0x7fe0240fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2062754402 shutdown_connections 2023-09-23T04:45:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.659+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2062754402 wait complete. 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 Processor -- start 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 -- start start 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 0x7fe024074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102090 0x7fe024073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02419aad0 con 0x7fe024102a50 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe024073c90 con 0x7fe024100eb0 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02c3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe024073e50 con 0x7fe024102090 2023-09-23T04:45:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02a99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02a99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50572/0 (socket says 172.21.15.5:50572) 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02a99c700 1 -- 172.21.15.5:0/2048610456 learned_addr learned my addr 172.21.15.5:0/2048610456 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02a19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 0x7fe024074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.660+0000 7fe02999a700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102090 0x7fe024073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 -- 172.21.15.5:0/2048610456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102090 msgr2=0x7fe024073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102090 0x7fe024073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 -- 172.21.15.5:0/2048610456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 msgr2=0x7fe024074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 0x7fe024074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 -- 172.21.15.5:0/2048610456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200095f0 con 0x7fe024102a50 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a19b700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 0x7fe024074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02a99c700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00ef30 tx=0x7fe01c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:44.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c018070 con 0x7fe024102a50 2023-09-23T04:45:44.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01c00f040 con 0x7fe024102a50 2023-09-23T04:45:44.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0241a5480 con 0x7fe024102a50 2023-09-23T04:45:44.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00cd00 con 0x7fe024102a50 2023-09-23T04:45:44.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.661+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0241a57f0 con 0x7fe024102a50 2023-09-23T04:45:44.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.662+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe024067ca0 con 0x7fe024102a50 2023-09-23T04:45:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.664+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fe01c003a00 con 0x7fe024102a50 2023-09-23T04:45:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.666+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 0x7fe0100a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.666+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fe01c0cced0 con 0x7fe024102a50 2023-09-23T04:45:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.667+0000 7fe02a19b700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 0x7fe0100a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.667+0000 7fe02a19b700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 0x7fe0100a3a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b2e0 tx=0x7fe020005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:44.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.670+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe01c096100 con 0x7fe024102a50 2023-09-23T04:45:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.883+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0241a5a10 con 0x7fe024102a50 2023-09-23T04:45:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.883+0000 7fe01b7fe700 1 -- 172.21.15.5:0/2048610456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe01c099720 con 0x7fe024102a50 2023-09-23T04:45:44.884 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:45:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 msgr2=0x7fe0100a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 0x7fe0100a3a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b2e0 tx=0x7fe020005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 msgr2=0x7fe024073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:44.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00ef30 tx=0x7fe01c00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 shutdown_connections 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe0100a15a0 0x7fe0100a3a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe024102a50 0x7fe024073750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe024100eb0 0x7fe024074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 --2- 172.21.15.5:0/2048610456 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe024102090 0x7fe024073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 >> 172.21.15.5:0/2048610456 conn(0x7fe0240fc470 msgr2=0x7fe0240fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 shutdown_connections 2023-09-23T04:45:44.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:44.886+0000 7fe02c3ff700 1 -- 172.21.15.5:0/2048610456 wait complete. 2023-09-23T04:45:44.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:45:44.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:44 smithi069 ceph-mon[133931]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:45.203 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:44 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/432671228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.203 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:44 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/299272518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.204 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:44 smithi118 ceph-mon[134297]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:45.204 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:44 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/432671228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.204 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:44 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/299272518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/2452138608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54106be0 msgr2=0x7f5d54109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/2452138608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54106be0 0x7f5d54109050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b0a0 tx=0x7f5d4c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/2452138608 shutdown_connections 2023-09-23T04:45:45.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/2452138608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54106be0 0x7f5d54109050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/2452138608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/2452138608 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d541042b0 0x7f5d541066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/2452138608 >> 172.21.15.5:0/2452138608 conn(0x7f5d540fb350 msgr2=0x7f5d540fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/2452138608 shutdown_connections 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.110+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/2452138608 wait complete. 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.111+0000 7f5d5a7ff700 1 Processor -- start 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.111+0000 7f5d5a7ff700 1 -- start start 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d54106be0 0x7f5d5419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419e830 con 0x7f5d541042b0 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419e9a0 con 0x7f5d54101980 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d5a7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5419eb60 con 0x7f5d54106be0 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50578/0 (socket says 172.21.15.5:50578) 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d537fe700 1 -- 172.21.15.5:0/1498738119 learned_addr learned my addr 172.21.15.5:0/1498738119 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d53fff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.112+0000 7f5d58d9c700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d54106be0 0x7f5d5419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 -- 172.21.15.5:0/1498738119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d54106be0 msgr2=0x7f5d5419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d54106be0 0x7f5d5419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 -- 172.21.15.5:0/1498738119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 msgr2=0x7f5d54196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 -- 172.21.15.5:0/1498738119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009d00 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d53fff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d537fe700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e9a0 tx=0x7f5d4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48009790 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d48010430 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48018680 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5419ee40 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.113+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5419f260 con 0x7f5d541042b0 2023-09-23T04:45:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.114+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540fced0 con 0x7f5d541042b0 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.116+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f5d480105a0 con 0x7f5d541042b0 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.118+0000 7f5d517fa700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 0x7f5d3c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.118+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f5d48014070 con 0x7f5d541042b0 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.118+0000 7f5d53fff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 0x7f5d3c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.119+0000 7f5d53fff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 0x7f5d3c0a3ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d541981c0 tx=0x7f5d44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.119+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5d48096880 con 0x7f5d541042b0 2023-09-23T04:45:45.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:44 smithi005 ceph-mon[97638]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:45.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:44 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/432671228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:44 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/299272518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.331+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5d5410c8c0 con 0x7f5d541042b0 2023-09-23T04:45:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.332+0000 7f5d517fa700 1 -- 172.21.15.5:0/1498738119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5d48026020 con 0x7f5d541042b0 2023-09-23T04:45:45.333 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 msgr2=0x7f5d3c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 0x7f5d3c0a3ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d541981c0 tx=0x7f5d44009260 comp rx=0 tx=0).stop 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 msgr2=0x7f5d54197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e9a0 tx=0x7f5d4800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 shutdown_connections 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5d3c0a1620 0x7f5d3c0a3ae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.334+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d54106be0 0x7f5d5419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.335+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d54101980 0x7f5d54196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.335+0000 7f5d5a7ff700 1 --2- 172.21.15.5:0/1498738119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541042b0 0x7f5d54197230 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.335+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 >> 172.21.15.5:0/1498738119 conn(0x7f5d540fb350 msgr2=0x7f5d54102690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.335+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 shutdown_connections 2023-09-23T04:45:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.335+0000 7f5d5a7ff700 1 -- 172.21.15.5:0/1498738119 wait complete. 2023-09-23T04:45:45.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.559+0000 7f66fe3ff700 1 -- 172.21.15.5:0/877151103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8102a30 msgr2=0x7f66f8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.559+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/877151103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8102a30 0x7f66f8109350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b0a0 tx=0x7f66f000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 -- 172.21.15.5:0/877151103 shutdown_connections 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/877151103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8102a30 0x7f66f8109350 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/877151103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8100e90 0x7f66f81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/877151103 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102070 0x7f66f81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 -- 172.21.15.5:0/877151103 >> 172.21.15.5:0/877151103 conn(0x7f66f80fc450 msgr2=0x7f66f80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 -- 172.21.15.5:0/877151103 shutdown_connections 2023-09-23T04:45:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.560+0000 7f66fe3ff700 1 -- 172.21.15.5:0/877151103 wait complete. 2023-09-23T04:45:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.561+0000 7f66fe3ff700 1 Processor -- start 2023-09-23T04:45:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 -- start start 2023-09-23T04:45:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8102070 0x7f66f8073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102a30 0x7f66f8073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8073ef0 con 0x7f66f8100e90 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8074060 con 0x7f66f8102070 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fe3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81a0d10 con 0x7f66f8102a30 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50598/0 (socket says 172.21.15.5:50598) 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66f7fff700 1 -- 172.21.15.5:0/2138025878 learned_addr learned my addr 172.21.15.5:0/2138025878 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66f77fe700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8102070 0x7f66f8073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.562+0000 7f66fc99c700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102a30 0x7f66f8073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 -- 172.21.15.5:0/2138025878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102a30 msgr2=0x7f66f8073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102a30 0x7f66f8073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 -- 172.21.15.5:0/2138025878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8102070 msgr2=0x7f66f8073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8102070 0x7f66f8073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 -- 172.21.15.5:0/2138025878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec009520 con 0x7f66f8100e90 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f7fff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e970 tx=0x7f66e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8009790 con 0x7f66f8100e90 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e800eec0 con 0x7f66f8100e90 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0009d00 con 0x7f66f8100e90 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8003cf0 con 0x7f66f8100e90 2023-09-23T04:45:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.563+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f81a12e0 con 0x7f66f8100e90 2023-09-23T04:45:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.564+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f81084a0 con 0x7f66f8100e90 2023-09-23T04:45:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.566+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f66e800ccd0 con 0x7f66f8100e90 2023-09-23T04:45:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.568+0000 7f66f57fa700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 0x7f66e00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.569+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f66e8014070 con 0x7f66f8100e90 2023-09-23T04:45:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.569+0000 7f66f77fe700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 0x7f66e00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.569+0000 7f66f77fe700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 0x7f66e00a3ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f66f8100bb0 tx=0x7f66ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:45.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.573+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f66e80966e0 con 0x7f66f8100e90 2023-09-23T04:45:45.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.832+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66f810c700 con 0x7f66f8100e90 2023-09-23T04:45:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.833+0000 7f66f57fa700 1 -- 172.21.15.5:0/2138025878 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f66e8099d00 con 0x7f66f8100e90 2023-09-23T04:45:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:45.835 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:45:45.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 msgr2=0x7f66e00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 0x7f66e00a3ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f66f8100bb0 tx=0x7f66ec009260 comp rx=0 tx=0).stop 2023-09-23T04:45:45.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 msgr2=0x7f66f8074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:45.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e970 tx=0x7f66e800ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 shutdown_connections 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f66e00a1620 0x7f66e00a3ae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f66f8102a30 0x7f66f8073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f8100e90 0x7f66f8074bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 --2- 172.21.15.5:0/2138025878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f66f8102070 0x7f66f8073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 >> 172.21.15.5:0/2138025878 conn(0x7f66f80fc450 msgr2=0x7f66f81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 shutdown_connections 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:45.836+0000 7f66fe3ff700 1 -- 172.21.15.5:0/2138025878 wait complete. 2023-09-23T04:45:45.838 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:45.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:45.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:45 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2048610456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:45.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:45 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1498738119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 -- 172.21.15.5:0/804832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 msgr2=0x7efda01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 --2- 172.21.15.5:0/804832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda01012c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd90009990 tx=0x7efd90009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 -- 172.21.15.5:0/804832238 shutdown_connections 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 --2- 172.21.15.5:0/804832238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 0x7efda0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 --2- 172.21.15.5:0/804832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda01012c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 --2- 172.21.15.5:0/804832238 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 0x7efda0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 -- 172.21.15.5:0/804832238 >> 172.21.15.5:0/804832238 conn(0x7efda00fc480 msgr2=0x7efda00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 -- 172.21.15.5:0/804832238 shutdown_connections 2023-09-23T04:45:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.059+0000 7efda67ff700 1 -- 172.21.15.5:0/804832238 wait complete. 2023-09-23T04:45:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.061+0000 7efda67ff700 1 Processor -- start 2023-09-23T04:45:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.061+0000 7efda67ff700 1 -- start start 2023-09-23T04:45:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.061+0000 7efda67ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.061+0000 7efda67ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 0x7efda0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efda67ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 0x7efda01a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efda67ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01a0970 con 0x7efda0100ea0 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efda67ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01a0ab0 con 0x7efda0102a40 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efda67ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01a0c40 con 0x7efda0102080 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50610/0 (socket says 172.21.15.5:50610) 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 -- 172.21.15.5:0/1993863750 learned_addr learned my addr 172.21.15.5:0/1993863750 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 0x7efda0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efda4d9c700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 0x7efda01a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 -- 172.21.15.5:0/1993863750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 msgr2=0x7efda0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 0x7efda0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 -- 172.21.15.5:0/1993863750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 msgr2=0x7efda01a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 0x7efda01a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 -- 172.21.15.5:0/1993863750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009c40 con 0x7efda0100ea0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.062+0000 7efd9ffff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd90004710 tx=0x7efd90009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.063+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9001d070 con 0x7efda0100ea0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.063+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd900095f0 con 0x7efda0100ea0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.063+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd90021410 con 0x7efda0100ea0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.063+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90017650 con 0x7efda0100ea0 2023-09-23T04:45:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.063+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda01a11b0 con 0x7efda0100ea0 2023-09-23T04:45:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.064+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda0193520 con 0x7efda0100ea0 2023-09-23T04:45:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.066+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7efd90021580 con 0x7efda0100ea0 2023-09-23T04:45:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.067+0000 7efd9d7fa700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 0x7efd880a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.067+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7efd900cea70 con 0x7efda0100ea0 2023-09-23T04:45:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.068+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 0x7efd880a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.068+0000 7efd9f7fe700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 0x7efd880a3a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda0100bc0 tx=0x7efd94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.069+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efd90097ca0 con 0x7efda0100ea0 2023-09-23T04:45:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:45 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2048610456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:45 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1498738119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:45 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2048610456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:45 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1498738119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:46.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.285+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda010c980 con 0x7efda0100ea0 2023-09-23T04:45:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.287+0000 7efd9d7fa700 1 -- 172.21.15.5:0/1993863750 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7efd9009b2c0 con 0x7efda0100ea0 2023-09-23T04:45:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:46.288 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 msgr2=0x7efd880a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 0x7efd880a3a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda0100bc0 tx=0x7efd94009260 comp rx=0 tx=0).stop 2023-09-23T04:45:46.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 msgr2=0x7efda0198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd90004710 tx=0x7efd90009580 comp rx=0 tx=0).stop 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 shutdown_connections 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efd880a15a0 0x7efd880a3a60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efda0102a40 0x7efda01a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda0100ea0 0x7efda0198df0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 --2- 172.21.15.5:0/1993863750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efda0102080 0x7efda0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 >> 172.21.15.5:0/1993863750 conn(0x7efda00fc480 msgr2=0x7efda0107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.289+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 shutdown_connections 2023-09-23T04:45:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.290+0000 7efda67ff700 1 -- 172.21.15.5:0/1993863750 wait complete. 2023-09-23T04:45:46.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:46.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 -- 172.21.15.5:0/1982962943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48041019d0 msgr2=0x7f4804103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 --2- 172.21.15.5:0/1982962943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48041019d0 0x7f4804103dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009990 tx=0x7f47f4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 -- 172.21.15.5:0/1982962943 shutdown_connections 2023-09-23T04:45:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 --2- 172.21.15.5:0/1982962943 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 0x7f4804109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 --2- 172.21.15.5:0/1982962943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48041019d0 0x7f4804103dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 --2- 172.21.15.5:0/1982962943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4804104300 0x7f48041066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 -- 172.21.15.5:0/1982962943 >> 172.21.15.5:0/1982962943 conn(0x7f48040fb3a0 msgr2=0x7f48040fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 -- 172.21.15.5:0/1982962943 shutdown_connections 2023-09-23T04:45:46.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.517+0000 7f4809bff700 1 -- 172.21.15.5:0/1982962943 wait complete. 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.518+0000 7f4809bff700 1 Processor -- start 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.518+0000 7f4809bff700 1 -- start start 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.518+0000 7f4809bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041019d0 0x7f4804198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4809bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4809bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 0x7f48041a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4809bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041a0980 con 0x7f4804104300 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4809bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041a0ac0 con 0x7f48041019d0 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4809bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041a0c50 con 0x7f4804106c30 2023-09-23T04:45:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50620/0 (socket says 172.21.15.5:50620) 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 -- 172.21.15.5:0/4059330557 learned_addr learned my addr 172.21.15.5:0/4059330557 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f48037fe700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041019d0 0x7f4804198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4803fff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 0x7f48041a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 -- 172.21.15.5:0/4059330557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 msgr2=0x7f48041a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 0x7f48041a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 -- 172.21.15.5:0/4059330557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041019d0 msgr2=0x7f4804198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041019d0 0x7f4804198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.519+0000 7f4802ffd700 1 -- 172.21.15.5:0/4059330557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40095f0 con 0x7f4804104300 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4802ffd700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9a0 tx=0x7f47f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8009790 con 0x7f4804104300 2023-09-23T04:45:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47f8010430 con 0x7f4804104300 2023-09-23T04:45:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48041a0f30 con 0x7f4804104300 2023-09-23T04:45:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8018680 con 0x7f4804104300 2023-09-23T04:45:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.520+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48041a13b0 con 0x7f4804104300 2023-09-23T04:45:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.521+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48041087b0 con 0x7f4804104300 2023-09-23T04:45:46.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.523+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f47f80105a0 con 0x7f4804104300 2023-09-23T04:45:46.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.524+0000 7f4800ff9700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 0x7f47ec0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.525+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f47f8014070 con 0x7f4804104300 2023-09-23T04:45:46.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.525+0000 7f48037fe700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 0x7f47ec0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.525+0000 7f48037fe700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 0x7f47ec0a3a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b2e0 tx=0x7f47f4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.528+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f47f8051ec0 con 0x7f4804104300 2023-09-23T04:45:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.743+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f480402d180 con 0x7f4804104300 2023-09-23T04:45:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.744+0000 7f4800ff9700 1 -- 172.21.15.5:0/4059330557 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47f8096670 con 0x7f4804104300 2023-09-23T04:45:46.745 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 msgr2=0x7f47ec0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 0x7f47ec0a3a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47f400b2e0 tx=0x7f47f4005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 msgr2=0x7f4804199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9a0 tx=0x7f47f800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 shutdown_connections 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f47ec0a15a0 0x7f47ec0a3a60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4804106c30 0x7f48041a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48041019d0 0x7f4804198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 --2- 172.21.15.5:0/4059330557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4804104300 0x7f4804199340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.746+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 >> 172.21.15.5:0/4059330557 conn(0x7f48040fb3a0 msgr2=0x7f4804102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.747+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 shutdown_connections 2023-09-23T04:45:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.747+0000 7f4809bff700 1 -- 172.21.15.5:0/4059330557 wait complete. 2023-09-23T04:45:46.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 -- 172.21.15.5:0/1744606902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 msgr2=0x7f83a8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1744606902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a8109370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f83a000b0a0 tx=0x7f83a000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 -- 172.21.15.5:0/1744606902 shutdown_connections 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1744606902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a8109370 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1744606902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8100eb0 0x7f83a81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1744606902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8102090 0x7f83a8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 -- 172.21.15.5:0/1744606902 >> 172.21.15.5:0/1744606902 conn(0x7f83a80fc470 msgr2=0x7f83a80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 -- 172.21.15.5:0/1744606902 shutdown_connections 2023-09-23T04:45:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.972+0000 7f83b03ff700 1 -- 172.21.15.5:0/1744606902 wait complete. 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.973+0000 7f83b03ff700 1 Processor -- start 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 -- start start 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8100eb0 0x7f83a8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8102090 0x7f83a8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81a0a70 con 0x7f83a8102a50 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81a0bb0 con 0x7f83a8102090 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83b03ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a81a0d40 con 0x7f83a8100eb0 2023-09-23T04:45:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83ae99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83ae99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50630/0 (socket says 172.21.15.5:50630) 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83ae99c700 1 -- 172.21.15.5:0/1526676731 learned_addr learned my addr 172.21.15.5:0/1526676731 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83ae19b700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8100eb0 0x7f83a8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.974+0000 7f83ad99a700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8102090 0x7f83a8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 -- 172.21.15.5:0/1526676731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8100eb0 msgr2=0x7f83a8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8100eb0 0x7f83a8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 -- 172.21.15.5:0/1526676731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8102090 msgr2=0x7f83a8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8102090 0x7f83a8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 -- 172.21.15.5:0/1526676731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009d00 con 0x7f83a8102a50 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83ae99c700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000e040 con 0x7f83a8102a50 2023-09-23T04:45:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a00094a0 con 0x7f83a8102a50 2023-09-23T04:45:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a001b6c0 con 0x7f83a8102a50 2023-09-23T04:45:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a81a0fc0 con 0x7f83a8102a50 2023-09-23T04:45:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.975+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a81a1380 con 0x7f83a8102a50 2023-09-23T04:45:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.977+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a810c770 con 0x7f83a8102a50 2023-09-23T04:45:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.978+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f83a001d470 con 0x7f83a8102a50 2023-09-23T04:45:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.980+0000 7f839f7fe700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 0x7f83940a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:46.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.980+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f83a00ce260 con 0x7f83a8102a50 2023-09-23T04:45:46.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.980+0000 7f83ae19b700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 0x7f83940a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:46.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.981+0000 7f83ae19b700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 0x7f83940a3ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f83a8101c00 tx=0x7f83a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:46.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:46.984+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f83a0097490 con 0x7f83a8102a50 2023-09-23T04:45:46.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:46 smithi069 ceph-mon[133931]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:46.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:46 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2138025878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:46 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1993863750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:46 smithi005 ceph-mon[97638]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2138025878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1993863750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:46 smithi118 ceph-mon[134297]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:46 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2138025878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:46 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1993863750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.194+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83a81a1570 con 0x7f83a8102a50 2023-09-23T04:45:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.194+0000 7f839f7fe700 1 -- 172.21.15.5:0/1526676731 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83a009aab0 con 0x7f83a8102a50 2023-09-23T04:45:47.195 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 msgr2=0x7f83940a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 0x7f83940a3ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f83a8101c00 tx=0x7f83a4009260 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 msgr2=0x7f83a81a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009fd0 tx=0x7f83a000bce0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 shutdown_connections 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83940a1620 0x7f83940a3ae0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83a8102a50 0x7f83a81a04a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83a8100eb0 0x7f83a8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 --2- 172.21.15.5:0/1526676731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83a8102090 0x7f83a8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 >> 172.21.15.5:0/1526676731 conn(0x7f83a80fc470 msgr2=0x7f83a81079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 shutdown_connections 2023-09-23T04:45:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.197+0000 7f83b03ff700 1 -- 172.21.15.5:0/1526676731 wait complete. 2023-09-23T04:45:47.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 -- 172.21.15.5:0/313608372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 msgr2=0x7f1608101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 --2- 172.21.15.5:0/313608372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608101330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009990 tx=0x7f15f8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 -- 172.21.15.5:0/313608372 shutdown_connections 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 --2- 172.21.15.5:0/313608372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102a20 0x7f1608109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 --2- 172.21.15.5:0/313608372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608101330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 --2- 172.21.15.5:0/313608372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102060 0x7f16081024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 -- 172.21.15.5:0/313608372 >> 172.21.15.5:0/313608372 conn(0x7f16080fc480 msgr2=0x7f16080fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:47.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 -- 172.21.15.5:0/313608372 shutdown_connections 2023-09-23T04:45:47.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.421+0000 7f160e7ff700 1 -- 172.21.15.5:0/313608372 wait complete. 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 Processor -- start 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 -- start start 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102a20 0x7f16081a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a0960 con 0x7f1608102060 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a0aa0 con 0x7f1608100f10 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.422+0000 7f160e7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081a0c30 con 0x7f1608102a20 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50650/0 (socket says 172.21.15.5:50650) 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f1607fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 -- 172.21.15.5:0/3547848101 learned_addr learned my addr 172.21.15.5:0/3547848101 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f160cd9c700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102a20 0x7f16081a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 -- 172.21.15.5:0/3547848101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102a20 msgr2=0x7f16081a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102a20 0x7f16081a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 -- 172.21.15.5:0/3547848101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 msgr2=0x7f1608198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 -- 172.21.15.5:0/3547848101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80095f0 con 0x7f1608102060 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.423+0000 7f16077fe700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00e9a0 tx=0x7f15fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.424+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc009790 con 0x7f1608102060 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.424+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc010430 con 0x7f1608102060 2023-09-23T04:45:47.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.424+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16081a0f10 con 0x7f1608102060 2023-09-23T04:45:47.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.424+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018680 con 0x7f1608102060 2023-09-23T04:45:47.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.424+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16081a1390 con 0x7f1608102060 2023-09-23T04:45:47.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.425+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608193550 con 0x7f1608102060 2023-09-23T04:45:47.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.427+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f15fc0105a0 con 0x7f1608102060 2023-09-23T04:45:47.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.428+0000 7f16057fa700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 0x7f15f00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.428+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f15fc014070 con 0x7f1608102060 2023-09-23T04:45:47.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.428+0000 7f1607fff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 0x7f15f00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.429+0000 7f1607fff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 0x7f15f00a3a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b2e0 tx=0x7f15f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:47.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.432+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f15fc051ec0 con 0x7f1608102060 2023-09-23T04:45:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.647+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f160802d180 con 0x7f1608102060 2023-09-23T04:45:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.648+0000 7f16057fa700 1 -- 172.21.15.5:0/3547848101 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15fc096670 con 0x7f1608102060 2023-09-23T04:45:47.649 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:45:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 msgr2=0x7f15f00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 0x7f15f00a3a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f15f800b2e0 tx=0x7f15f8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 msgr2=0x7f1608199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00e9a0 tx=0x7f15fc00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 shutdown_connections 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f15f00a15a0 0x7f15f00a3a60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1608102a20 0x7f16081a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1608100f10 0x7f1608198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 --2- 172.21.15.5:0/3547848101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1608102060 0x7f1608199320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 >> 172.21.15.5:0/3547848101 conn(0x7f16080fc480 msgr2=0x7f16081079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 shutdown_connections 2023-09-23T04:45:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.650+0000 7f160e7ff700 1 -- 172.21.15.5:0/3547848101 wait complete. 2023-09-23T04:45:47.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:45:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 -- 172.21.15.5:0/704430004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334102a50 msgr2=0x7f9334109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 --2- 172.21.15.5:0/704430004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334102a50 0x7f9334109370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b0a0 tx=0x7f932c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 -- 172.21.15.5:0/704430004 shutdown_connections 2023-09-23T04:45:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 --2- 172.21.15.5:0/704430004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334102a50 0x7f9334109370 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 --2- 172.21.15.5:0/704430004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334100eb0 0x7f93341012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 --2- 172.21.15.5:0/704430004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 0x7f9334102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 -- 172.21.15.5:0/704430004 >> 172.21.15.5:0/704430004 conn(0x7f93340fc470 msgr2=0x7f93340fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 -- 172.21.15.5:0/704430004 shutdown_connections 2023-09-23T04:45:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.878+0000 7f933c1ff700 1 -- 172.21.15.5:0/704430004 wait complete. 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.879+0000 7f933c1ff700 1 Processor -- start 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.879+0000 7f933c1ff700 1 -- start start 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 0x7f9334073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334102a50 0x7f9334073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334073ef0 con 0x7f9334100eb0 2023-09-23T04:45:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334074060 con 0x7f9334102a50 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933c1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341a0d30 con 0x7f9334102090 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50666/0 (socket says 172.21.15.5:50666) 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 -- 172.21.15.5:0/2916802349 learned_addr learned my addr 172.21.15.5:0/2916802349 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933979a700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 0x7f9334073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f933a79c700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334102a50 0x7f9334073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 -- 172.21.15.5:0/2916802349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 msgr2=0x7f9334073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 0x7f9334073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 -- 172.21.15.5:0/2916802349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334102a50 msgr2=0x7f9334073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334102a50 0x7f9334073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.880+0000 7f9339f9b700 1 -- 172.21.15.5:0/2916802349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009d00 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f9339f9b700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f933000e970 tx=0x7f933000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330009790 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933000c7e0 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93341a0fb0 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330003c70 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.881+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93341a13e0 con 0x7f9334100eb0 2023-09-23T04:45:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.882+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9334107e10 con 0x7f9334100eb0 2023-09-23T04:45:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.884+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f933000cc50 con 0x7f9334100eb0 2023-09-23T04:45:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.886+0000 7f932affd700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 0x7f93200a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.886+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f9330014070 con 0x7f9334100eb0 2023-09-23T04:45:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.886+0000 7f933979a700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 0x7f93200a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.887+0000 7f933979a700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 0x7f93200a3a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9334100bd0 tx=0x7f9324009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:47.889+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9330096680 con 0x7f9334100eb0 2023-09-23T04:45:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:47 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4059330557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:47 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1526676731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:47 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3547848101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.100+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f933402d230 con 0x7f9334100eb0 2023-09-23T04:45:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.101+0000 7f932affd700 1 -- 172.21.15.5:0/2916802349 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9330003dd0 con 0x7f9334100eb0 2023-09-23T04:45:48.102 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 msgr2=0x7f93200a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 0x7f93200a3a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9334100bd0 tx=0x7f9324009260 comp rx=0 tx=0).stop 2023-09-23T04:45:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 msgr2=0x7f9334074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f933000e970 tx=0x7f933000ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 shutdown_connections 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93200a15a0 0x7f93200a3a60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9334102a50 0x7f9334073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9334100eb0 0x7f9334074bd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 --2- 172.21.15.5:0/2916802349 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9334102090 0x7f9334073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 >> 172.21.15.5:0/2916802349 conn(0x7f93340fc470 msgr2=0x7f93340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 shutdown_connections 2023-09-23T04:45:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.103+0000 7f933c1ff700 1 -- 172.21.15.5:0/2916802349 wait complete. 2023-09-23T04:45:48.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:47 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4059330557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:47 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1526676731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:47 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3547848101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:45:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:47 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4059330557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:47 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1526676731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:47 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3547848101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 -- 172.21.15.5:0/2601483063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c100ea0 msgr2=0x7fc70c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 --2- 172.21.15.5:0/2601483063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c100ea0 0x7fc70c1012c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc7080098f0 tx=0x7fc708009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 -- 172.21.15.5:0/2601483063 shutdown_connections 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 --2- 172.21.15.5:0/2601483063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c102a40 0x7fc70c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 --2- 172.21.15.5:0/2601483063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c100ea0 0x7fc70c1012c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 --2- 172.21.15.5:0/2601483063 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 -- 172.21.15.5:0/2601483063 >> 172.21.15.5:0/2601483063 conn(0x7fc70c0fc480 msgr2=0x7fc70c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:48.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 -- 172.21.15.5:0/2601483063 shutdown_connections 2023-09-23T04:45:48.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.325+0000 7fc7141ff700 1 -- 172.21.15.5:0/2601483063 wait complete. 2023-09-23T04:45:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.326+0000 7fc7141ff700 1 Processor -- start 2023-09-23T04:45:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.326+0000 7fc7141ff700 1 -- start start 2023-09-23T04:45:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 0x7fc70c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a09c0 con 0x7fc70c102a40 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a0b00 con 0x7fc70c100ea0 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc7141ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1a0c90 con 0x7fc70c102080 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc71279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc711f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 0x7fc70c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc71279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50678/0 (socket says 172.21.15.5:50678) 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc71279c700 1 -- 172.21.15.5:0/3107373848 learned_addr learned my addr 172.21.15.5:0/3107373848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.327+0000 7fc71179a700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 -- 172.21.15.5:0/3107373848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 msgr2=0x7fc70c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 -- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 msgr2=0x7fc70c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 -- 172.21.15.5:0/3107373848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc009520 con 0x7fc70c100ea0 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc71179a700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc71279c700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc711f9b700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 0x7fc70c198e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc708000c00 tx=0x7fc70800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70801c070 con 0x7fc70c100ea0 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704009d00 con 0x7fc70c100ea0 2023-09-23T04:45:48.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70800f7d0 con 0x7fc70c100ea0 2023-09-23T04:45:48.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708003ac0 con 0x7fc70c100ea0 2023-09-23T04:45:48.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.328+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708009520 con 0x7fc70c100ea0 2023-09-23T04:45:48.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.329+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c193550 con 0x7fc70c100ea0 2023-09-23T04:45:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.332+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc708020030 con 0x7fc70c100ea0 2023-09-23T04:45:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.333+0000 7fc702ffd700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 0x7fc6f80a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.333+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fc7080ce590 con 0x7fc70c100ea0 2023-09-23T04:45:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.333+0000 7fc71179a700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 0x7fc6f80a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.334+0000 7fc71179a700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 0x7fc6f80a3ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc70c100bc0 tx=0x7fc6fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.335+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc708097870 con 0x7fc70c100ea0 2023-09-23T04:45:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.590+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc70c02d180 con 0x7fc70c100ea0 2023-09-23T04:45:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.592+0000 7fc702ffd700 1 -- 172.21.15.5:0/3107373848 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7fc70809ae90 con 0x7fc70c100ea0 2023-09-23T04:45:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:48.592 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:45:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 msgr2=0x7fc6f80a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 0x7fc6f80a3ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc70c100bc0 tx=0x7fc6fc009260 comp rx=0 tx=0).stop 2023-09-23T04:45:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 msgr2=0x7fc70c198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 0x7fc70c198e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc708000c00 tx=0x7fc70800b790 comp rx=0 tx=0).stop 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 shutdown_connections 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc6f80a1620 0x7fc6f80a3ae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc70c102a40 0x7fc70c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc70c100ea0 0x7fc70c198e40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 --2- 172.21.15.5:0/3107373848 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc70c102080 0x7fc70c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.593+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 >> 172.21.15.5:0/3107373848 conn(0x7fc70c0fc480 msgr2=0x7fc70c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.594+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 shutdown_connections 2023-09-23T04:45:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.594+0000 7fc7141ff700 1 -- 172.21.15.5:0/3107373848 wait complete. 2023-09-23T04:45:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:45:48.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:48.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 -- 172.21.15.5:0/2333344258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 msgr2=0x7f5e04106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 --2- 172.21.15.5:0/2333344258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04106670 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5df80098f0 tx=0x7f5df8009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 -- 172.21.15.5:0/2333344258 shutdown_connections 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 --2- 172.21.15.5:0/2333344258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 0x7f5e04109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 --2- 172.21.15.5:0/2333344258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 --2- 172.21.15.5:0/2333344258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04106670 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 -- 172.21.15.5:0/2333344258 >> 172.21.15.5:0/2333344258 conn(0x7f5e040fb360 msgr2=0x7f5e040fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:48.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 -- 172.21.15.5:0/2333344258 shutdown_connections 2023-09-23T04:45:48.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.817+0000 7f5e097ff700 1 -- 172.21.15.5:0/2333344258 wait complete. 2023-09-23T04:45:48.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.818+0000 7f5e097ff700 1 Processor -- start 2023-09-23T04:45:48.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.818+0000 7f5e097ff700 1 -- start start 2023-09-23T04:45:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 0x7f5e041a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a0900 con 0x7f5e04101950 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a0a70 con 0x7f5e04104280 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e097ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041a0c30 con 0x7f5e04106bb0 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50694/0 (socket says 172.21.15.5:50694) 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e02ffd700 1 -- 172.21.15.5:0/1269602583 learned_addr learned my addr 172.21.15.5:0/1269602583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e027fc700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.819+0000 7f5e037fe700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 0x7f5e041a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 -- 172.21.15.5:0/1269602583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 msgr2=0x7f5e041a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 0x7f5e041a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 -- 172.21.15.5:0/1269602583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 msgr2=0x7f5e04199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 -- 172.21.15.5:0/1269602583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40095f0 con 0x7f5e04101950 2023-09-23T04:45:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e02ffd700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5df400ec10 tx=0x7f5df400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400cc90 con 0x7f5e04101950 2023-09-23T04:45:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df40044e0 con 0x7f5e04101950 2023-09-23T04:45:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df8009520 con 0x7f5e04101950 2023-09-23T04:45:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4010430 con 0x7f5e04101950 2023-09-23T04:45:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.820+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e041a11d0 con 0x7f5e04101950 2023-09-23T04:45:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.821+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e04108780 con 0x7f5e04101950 2023-09-23T04:45:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.823+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f5df4003fb0 con 0x7f5e04101950 2023-09-23T04:45:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.825+0000 7f5debfff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 0x7f5dec0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.825+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f5df4014070 con 0x7f5e04101950 2023-09-23T04:45:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.825+0000 7f5e027fc700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 0x7f5dec0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.826+0000 7f5e027fc700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 0x7f5dec0a3a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5df800fa40 tx=0x7f5df8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:48.829+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5df4096a40 con 0x7f5e04101950 2023-09-23T04:45:48.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:48 smithi005 ceph-mon[97638]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:48.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:48 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2916802349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:48 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3107373848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:48 smithi069 ceph-mon[133931]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:48 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2916802349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:48 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3107373848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:49.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.042+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e0410c6c0 con 0x7f5e04101950 2023-09-23T04:45:49.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.044+0000 7f5debfff700 1 -- 172.21.15.5:0/1269602583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7f5df409a060 con 0x7f5e04101950 2023-09-23T04:45:49.045 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:49.045 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 msgr2=0x7f5dec0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 0x7f5dec0a3a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5df800fa40 tx=0x7f5df8019040 comp rx=0 tx=0).stop 2023-09-23T04:45:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 msgr2=0x7f5e04198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5df400ec10 tx=0x7f5df400ef20 comp rx=0 tx=0).stop 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 shutdown_connections 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f5dec0a15a0 0x7f5dec0a3a60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5e04106bb0 0x7f5e041a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e04101950 0x7f5e04198dc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 --2- 172.21.15.5:0/1269602583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e04104280 0x7f5e04199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 >> 172.21.15.5:0/1269602583 conn(0x7f5e040fb360 msgr2=0x7f5e040ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.046+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 shutdown_connections 2023-09-23T04:45:49.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.047+0000 7f5e097ff700 1 -- 172.21.15.5:0/1269602583 wait complete. 2023-09-23T04:45:49.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-23T04:43:47.148877+0000', 'modified': '2023-09-23T04:44:20.444513+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24577, 'mds_1': 24559, 'mds_2': 24554, 'mds_3': 24535, 'mds_4': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14538': {'gid': 14538, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/1890134322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 1890134322}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 1890134322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6833/827181928', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6832', 'nonce': 827181928}, {'type': 'v1', 'addr': '172.21.15.69:6833', 'nonce': 827181928}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.118:6833/3707604246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6832', 'nonce': 3707604246}, {'type': 'v1', 'addr': '172.21.15.118:6833', 'nonce': 3707604246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6835/3966429650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6834', 'nonce': 3966429650}, {'type': 'v1', 'addr': '172.21.15.69:6835', 'nonce': 3966429650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6837/2958173168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6836', 'nonce': 2958173168}, {'type': 'v1', 'addr': '172.21.15.69:6837', 'nonce': 2958173168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24529': {'gid': 24529, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.118:6835/3772048959', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6834', 'nonce': 3772048959}, {'type': 'v1', 'addr': '172.21.15.118:6835', 'nonce': 3772048959}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.118:6837/704534117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6836', 'nonce': 704534117}, {'type': 'v1', 'addr': '172.21.15.118:6837', 'nonce': 704534117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.69:6839/1851157393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 1851157393}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 1851157393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.118:6839/430957039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.118:6838', 'nonce': 430957039}, {'type': 'v1', 'addr': '172.21.15.118:6839', 'nonce': 430957039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24577': {'gid': 24577, 'name': 'g', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.5:6839/3661239153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 3661239153}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 3661239153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T04:45:49.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-23T04:45:49.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-23T04:45:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:48 smithi118 ceph-mon[134297]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:48 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2916802349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:48 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3107373848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 -- 172.21.15.5:0/2764064690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24100f10 msgr2=0x7fcf24101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 --2- 172.21.15.5:0/2764064690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24100f10 0x7fcf24101330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009990 tx=0x7fcf14009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 -- 172.21.15.5:0/2764064690 shutdown_connections 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 --2- 172.21.15.5:0/2764064690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102a20 0x7fcf24109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 --2- 172.21.15.5:0/2764064690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24100f10 0x7fcf24101330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 --2- 172.21.15.5:0/2764064690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24102060 0x7fcf241024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 -- 172.21.15.5:0/2764064690 >> 172.21.15.5:0/2764064690 conn(0x7fcf240fc480 msgr2=0x7fcf240fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.271+0000 7fcf29dff700 1 -- 172.21.15.5:0/2764064690 shutdown_connections 2023-09-23T04:45:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.272+0000 7fcf29dff700 1 -- 172.21.15.5:0/2764064690 wait complete. 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.273+0000 7fcf29dff700 1 Processor -- start 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.273+0000 7fcf29dff700 1 -- start start 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102060 0x7fcf241970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24102a20 0x7fcf2419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419e6f0 con 0x7fcf24100f10 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419e830 con 0x7fcf24102060 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf29dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419e9c0 con 0x7fcf24102a20 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50706/0 (socket says 172.21.15.5:50706) 2023-09-23T04:45:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 -- 172.21.15.5:0/3458569789 learned_addr learned my addr 172.21.15.5:0/3458569789 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf23fff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24102a20 0x7fcf2419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102060 0x7fcf241970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 -- 172.21.15.5:0/3458569789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24102a20 msgr2=0x7fcf2419e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24102a20 0x7fcf2419e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 -- 172.21.15.5:0/3458569789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102060 msgr2=0x7fcf241970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102060 0x7fcf241970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.274+0000 7fcf237fe700 1 -- 172.21.15.5:0/3458569789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140095f0 con 0x7fcf24100f10 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf237fe700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b680 tx=0x7fcf140057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf20ff9700 1 -- 172.21.15.5:0/3458569789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1401d070 con 0x7fcf24100f10 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf20ff9700 1 -- 172.21.15.5:0/3458569789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1400fbc0 con 0x7fcf24100f10 2023-09-23T04:45:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2419ec40 con 0x7fcf24100f10 2023-09-23T04:45:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf20ff9700 1 -- 172.21.15.5:0/3458569789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14017650 con 0x7fcf24100f10 2023-09-23T04:45:49.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.275+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2419f0c0 con 0x7fcf24100f10 2023-09-23T04:45:49.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.278+0000 7fcf20ff9700 1 -- 172.21.15.5:0/3458569789 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fcf14022470 con 0x7fcf24100f10 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.279+0000 7fcf3d7fcb80 1 Processor -- start 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf20ff9700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 0x7fcf0c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 -- start start 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf20ff9700 1 -- 172.21.15.5:0/3458569789 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fcf140cea50 con 0x7fcf24100f10 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf23fff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 0x7fcf0c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096609770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096609d50 0x55b09660a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b096570410 con 0x55b096608eb0 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b096561bd0 con 0x55b096608550 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09655f4f0 con 0x55b096609d50 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52168/0 (socket says 172.21.15.5:52168) 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf237fe700 1 -- 172.21.15.5:0/1616504339 learned_addr learned my addr 172.21.15.5:0/1616504339 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.280+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096609770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf23fff700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096609d50 0x55b09660a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf23fff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 0x7fcf0c0a3a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c003f20 tx=0x7fcf1c009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 -- 172.21.15.5:0/1616504339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096609d50 msgr2=0x55b09660a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096609d50 0x55b09660a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 -- 172.21.15.5:0/1616504339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 msgr2=0x55b096609770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096609770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 -- 172.21.15.5:0/1616504339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0965bd880 con 0x55b096608550 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096609770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf140a3710 tx=0x7fcf14009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b1c42c4ac5cd24 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf09ffb700 1 -- 172.21.15.5:0/1616504339 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1409b940 con 0x55b096608550 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf09ffb700 1 -- 172.21.15.5:0/1616504339 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1409fa90 con 0x55b096608550 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.281+0000 7fcf09ffb700 1 -- 172.21.15.5:0/1616504339 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1409ae30 con 0x55b096608550 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1616504339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 msgr2=0x55b096608970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf140a3710 tx=0x7fcf14009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1616504339 shutdown_connections 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096609d50 0x55b09660a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096608550 0x55b096608970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1616504339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096609770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1616504339 >> 172.21.15.5:0/1616504339 conn(0x55b09657c210 msgr2=0x55b096603430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1616504339 shutdown_connections 2023-09-23T04:45:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.282+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1616504339 wait complete. 2023-09-23T04:45:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.283+0000 7fcf3d7fcb80 1 -- start start 2023-09-23T04:45:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.283+0000 7fcf3d7fcb80 1 Processor -- start 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.283+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b096616420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.283+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09655ade0 con 0x55b096608eb0 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0964f76c0 con 0x55b096609d50 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0964f7360 con 0x55b096608550 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf3d7fcb80 10 client.0 _ll_register_callbacks cb 0x55b0963547b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50716/0 (socket says 172.21.15.5:50716) 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 -- 172.21.15.5:0/1214797976 learned_addr learned my addr 172.21.15.5:0/1214797976 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b096616420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf23fff700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 -- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 msgr2=0x55b096616420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b096616420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 -- 172.21.15.5:0/1214797976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 msgr2=0x55b09661d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 -- 172.21.15.5:0/1214797976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18009830 con 0x55b096608eb0 2023-09-23T04:45:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcf180128d0 tx=0x7fcf18012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10bb32c2a34e49 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.284+0000 7fcf02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18017070 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0965fb200 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fcf180118d0 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf18011a90 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf3d7fcb80 10 client.15321 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fcf2419f0c0 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0965fb200 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf1801c420 con 0x55b096608eb0 2023-09-23T04:45:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.285+0000 7fcf3d7fcb80 10 client.15321 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf3d7fcb80 10 client.15321 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf3d7fcb80 20 client.15321 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b0966220d0 0x55b096620b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf3d7fcb80 4 client.15321 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b096553dc0 con 0x55b0966220d0 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.286+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b0966220d0 0x55b096620b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.287+0000 7fcf237fe700 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b0966220d0 0x55b096620b50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74a55cec565ba1c2 server_cookie=6c13952942fecff4 in_seq=0 out_seq=0 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.287+0000 7fcf02ffd700 10 client.15321 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.287+0000 7fcf02ffd700 1 -- 172.21.15.5:0/1214797976 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b096553dc0 con 0x55b0966220d0 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.287+0000 7fcf02ffd700 10 client.15321 handle_command_reply: tid=0 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.287+0000 7fcf3d7fcb80 1 client.15321 shutdown 2023-09-23T04:45:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 msgr2=0x55b096616960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcf180128d0 tx=0x7fcf18012be0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 shutdown_connections 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b096616420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b0966220d0 0x55b096620b50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/1214797976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b096616960 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 >> 172.21.15.5:0/1214797976 conn(0x55b09657c210 msgr2=0x55b096603240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf02ffd700 0 client.15321 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 shutdown_connections 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/1214797976 wait complete. 2023-09-23T04:45:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.288+0000 7fcf3d7fcb80 20 client.15321 trim_cache size 0 max 16384 2023-09-23T04:45:49.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.306+0000 7fcf3d7fcb80 1 Processor -- start 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.306+0000 7fcf3d7fcb80 1 -- start start 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b09661e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b0966150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0963db8d0 con 0x55b096608eb0 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b096556ed0 con 0x55b096609d50 2023-09-23T04:45:49.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0965014d0 con 0x55b096608550 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50718/0 (socket says 172.21.15.5:50718) 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 -- 172.21.15.5:0/3175870052 learned_addr learned my addr 172.21.15.5:0/3175870052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf23fff700 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b0966150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf237fe700 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b09661e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 -- 172.21.15.5:0/3175870052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 msgr2=0x55b09661e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b09661e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 -- 172.21.15.5:0/3175870052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 msgr2=0x55b0966150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b0966150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.307+0000 7fcf22ffd700 1 -- 172.21.15.5:0/3175870052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b096385d90 con 0x55b096608eb0 2023-09-23T04:45:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.308+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf18001eb0 tx=0x7fcf18005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35599de84811e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.308+0000 7fcf0a7fc700 1 -- 172.21.15.5:0/3175870052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18017040 con 0x55b096608eb0 2023-09-23T04:45:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.308+0000 7fcf0a7fc700 1 -- 172.21.15.5:0/3175870052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf18011d90 con 0x55b096608eb0 2023-09-23T04:45:49.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.308+0000 7fcf0a7fc700 1 -- 172.21.15.5:0/3175870052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005590 con 0x55b096608eb0 2023-09-23T04:45:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/3175870052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 msgr2=0x55b09661d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf18001eb0 tx=0x7fcf18005980 comp rx=0 tx=0).stop 2023-09-23T04:45:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/3175870052 shutdown_connections 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b0966150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608550 0x55b09661e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/3175870052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608eb0 0x55b09661d960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/3175870052 >> 172.21.15.5:0/3175870052 conn(0x55b09657c210 msgr2=0x55b096601eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/3175870052 shutdown_connections 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/3175870052 wait complete. 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 -- start start 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.309+0000 7fcf3d7fcb80 1 Processor -- start 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608eb0 0x55b09661c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09661d420 con 0x55b096608550 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09661d590 con 0x55b096609d50 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09661d750 con 0x55b096608eb0 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf3d7fcb80 10 client.0 _ll_register_callbacks cb 0x7fcf18009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50724/0 (socket says 172.21.15.5:50724) 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 -- 172.21.15.5:0/2818663388 learned_addr learned my addr 172.21.15.5:0/2818663388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608eb0 0x55b09661c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf23fff700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 -- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608eb0 msgr2=0x55b09661c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608eb0 0x55b09661c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 -- 172.21.15.5:0/2818663388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 msgr2=0x55b09661cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf237fe700 1 -- 172.21.15.5:0/2818663388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2419f0c0 con 0x55b096608550 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.310+0000 7fcf23fff700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf237fe700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf14097e40 tx=0x7fcf14009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b5709f7d40618 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1409b2a0 con 0x55b096608550 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf1409fa90 con 0x55b096608550 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fcf1400fd30 con 0x55b096608550 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf140671a0 con 0x55b096608550 2023-09-23T04:45:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf3d7fcb80 10 client.15327 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b09636ec10 con 0x55b096608550 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1409fa90 con 0x55b096608550 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.311+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf1409ae30 con 0x55b096608550 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 10 client.15327 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 20 client.15327 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b09667a230 0x55b096676690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 4 client.15327 mds_command: new command op to 24529 tid=0[{"prefix": "status"}] 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b096553ae0 con 0x55b09667a230 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.312+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b09667a230 0x55b096676690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf22ffd700 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b09667a230 0x55b096676690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bad26b322db0a0ec server_cookie=c894f1ab32bbbaaa in_seq=0 out_seq=0 2023-09-23T04:45:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf02ffd700 10 client.15327 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf02ffd700 1 -- 172.21.15.5:0/2818663388 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b096553ae0 con 0x55b09667a230 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf02ffd700 10 client.15327 handle_command_reply: tid=0 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 client.15327 shutdown 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 msgr2=0x55b096616750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf14097e40 tx=0x7fcf14009310 comp rx=0 tx=0).stop 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 shutdown_connections 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b096609d50 0x55b09661cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55b09667a230 0x55b096676690 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b096608550 0x55b096616750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 --2- 172.21.15.5:0/2818663388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b096608eb0 0x55b09661c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf02ffd700 0 client.15327 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:45:49.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 >> 172.21.15.5:0/2818663388 conn(0x55b09657c210 msgr2=0x55b096603200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 shutdown_connections 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 1 -- 172.21.15.5:0/2818663388 wait complete. 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.313+0000 7fcf3d7fcb80 20 client.15327 trim_cache size 0 max 16384 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24529, 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:49.321 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 89.510525225999999, 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 144.263463903 2023-09-23T04:45:49.322 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 msgr2=0x7fcf0c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 0x7fcf0c0a3a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c003f20 tx=0x7fcf1c009ba0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 msgr2=0x7fcf24196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b680 tx=0x7fcf140057b0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 shutdown_connections 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcf0c0a15a0 0x7fcf0c0a3a60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf24102a20 0x7fcf2419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf24100f10 0x7fcf24196b70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 --2- 172.21.15.5:0/3458569789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf24102060 0x7fcf241970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 >> 172.21.15.5:0/3458569789 conn(0x7fcf240fc480 msgr2=0x7fcf241055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 shutdown_connections 2023-09-23T04:45:49.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.314+0000 7fcf29dff700 1 -- 172.21.15.5:0/3458569789 wait complete. 2023-09-23T04:45:49.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-23T04:45:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 -- 172.21.15.5:0/639134866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8102a50 msgr2=0x7fced8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/639134866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8102a50 0x7fced8109370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fced000b0a0 tx=0x7fced000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 -- 172.21.15.5:0/639134866 shutdown_connections 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/639134866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8102a50 0x7fced8109370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/639134866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8100eb0 0x7fced81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/639134866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 -- 172.21.15.5:0/639134866 >> 172.21.15.5:0/639134866 conn(0x7fced80fc470 msgr2=0x7fced80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 -- 172.21.15.5:0/639134866 shutdown_connections 2023-09-23T04:45:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.536+0000 7fcedd5ff700 1 -- 172.21.15.5:0/639134866 wait complete. 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.537+0000 7fcedd5ff700 1 Processor -- start 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 -- start start 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8102a50 0x7fced81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81a0a20 con 0x7fced8100eb0 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81a0b60 con 0x7fced8102a50 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcedd5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81a0cf0 con 0x7fced8102090 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50730/0 (socket says 172.21.15.5:50730) 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fced6ffd700 1 -- 172.21.15.5:0/1753157236 learned_addr learned my addr 172.21.15.5:0/1753157236 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fced77fe700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8102a50 0x7fced81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.538+0000 7fcecffff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 -- 172.21.15.5:0/1753157236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 msgr2=0x7fced81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 -- 172.21.15.5:0/1753157236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8102a50 msgr2=0x7fced81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8102a50 0x7fced81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 -- 172.21.15.5:0/1753157236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009520 con 0x7fced8100eb0 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fcecffff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced6ffd700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcec800ea50 tx=0x7fcec800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced4ff9700 1 -- 172.21.15.5:0/1753157236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec800cbd0 con 0x7fced8100eb0 2023-09-23T04:45:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced4ff9700 1 -- 172.21.15.5:0/1753157236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcec800cd30 con 0x7fced8100eb0 2023-09-23T04:45:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced0009d00 con 0x7fced8100eb0 2023-09-23T04:45:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fced4ff9700 1 -- 172.21.15.5:0/1753157236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80186c0 con 0x7fced8100eb0 2023-09-23T04:45:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.539+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced81a1270 con 0x7fced8100eb0 2023-09-23T04:45:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.544+0000 7fced4ff9700 1 -- 172.21.15.5:0/1753157236 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fcec8018920 con 0x7fced8100eb0 2023-09-23T04:45:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.545+0000 7fcef0ef7b80 1 Processor -- start 2023-09-23T04:45:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.545+0000 7fcef0ef7b80 1 -- start start 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.545+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77de1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77de21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77d48410 con 0x556c77de0550 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77d39bd0 con 0x556c77de1d50 2023-09-23T04:45:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77d374f0 con 0x556c77de0eb0 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50746/0 (socket says 172.21.15.5:50746) 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 -- 172.21.15.5:0/764613481 learned_addr learned my addr 172.21.15.5:0/764613481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced6ffd700 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77de1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced4ff9700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 0x7fcec40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fcecffff700 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77de21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced4ff9700 1 -- 172.21.15.5:0/1753157236 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fcec8014070 con 0x7fced8100eb0 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 -- 172.21.15.5:0/764613481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 msgr2=0x556c77de1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77de1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 -- 172.21.15.5:0/764613481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 msgr2=0x556c77de21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77de21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.546+0000 7fced77fe700 1 -- 172.21.15.5:0/764613481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c77d95880 con 0x556c77de0550 2023-09-23T04:45:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.547+0000 7fced77fe700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 0x7fcec40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.547+0000 7fced77fe700 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fced0009360 tx=0x7fced0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf532b4edd49bd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.547+0000 7fcecd7fa700 1 -- 172.21.15.5:0/764613481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000e040 con 0x556c77de0550 2023-09-23T04:45:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.547+0000 7fcecd7fa700 1 -- 172.21.15.5:0/764613481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced000be90 con 0x556c77de0550 2023-09-23T04:45:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.547+0000 7fcecd7fa700 1 -- 172.21.15.5:0/764613481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced001eda0 con 0x556c77de0550 2023-09-23T04:45:49.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/764613481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 msgr2=0x556c77de0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fced0009360 tx=0x7fced0009390 comp rx=0 tx=0).stop 2023-09-23T04:45:49.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fced77fe700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 0x7fcec40a3ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fced002a000 tx=0x7fced0027000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/764613481 shutdown_connections 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77de21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77de0970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/764613481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77de1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/764613481 >> 172.21.15.5:0/764613481 conn(0x556c77d54210 msgr2=0x556c77ddb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/764613481 shutdown_connections 2023-09-23T04:45:49.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.548+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/764613481 wait complete. 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 -- start start 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 Processor -- start 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0550 0x556c77dea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77df1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77d32de0 con 0x556c77de0eb0 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77ccf6c0 con 0x556c77de1d50 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77ccf360 con 0x556c77de0550 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcef0ef7b80 10 client.0 _ll_register_callbacks cb 0x556c77b2c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.549+0000 7fcecffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50754/0 (socket says 172.21.15.5:50754) 2023-09-23T04:45:49.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 -- 172.21.15.5:0/2431035220 learned_addr learned my addr 172.21.15.5:0/2431035220 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fced77fe700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77df1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0550 0x556c77dea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 -- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0550 msgr2=0x556c77dea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0550 0x556c77dea2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 -- 172.21.15.5:0/2431035220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 msgr2=0x556c77df1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77df1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 -- 172.21.15.5:0/2431035220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced8109ca0 con 0x556c77de0eb0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcecffff700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcec00126e0 tx=0x7fcec00129f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f732c5466fa3e994 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.550+0000 7fcebe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0017070 con 0x556c77de0eb0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fcec0004320 con 0x556c77de0eb0 2023-09-23T04:45:49.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c77dd3200 con 0x556c77de0eb0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcec00044e0 con 0x556c77de0eb0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcef0ef7b80 10 client.15339 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c77b5dd90 con 0x556c77de0eb0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c77dd3200 con 0x556c77de0eb0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.551+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcec000f8b0 con 0x556c77de0eb0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 10 client.15339 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 20 client.15339 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77df5f90 0x556c77df4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 4 client.15339 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c77d2bdc0 con 0x556c77df5f90 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77df5f90 0x556c77df4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77df5f90 0x556c77df4a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5591bf61d111d6b3 server_cookie=e367a57fc5a9d0c0 in_seq=0 out_seq=0 2023-09-23T04:45:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.552+0000 7fcebe7fc700 10 client.15339 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.553+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2431035220 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c77d2bdc0 con 0x556c77df5f90 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.553+0000 7fcebe7fc700 10 client.15339 handle_command_reply: tid=0 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.553+0000 7fcef0ef7b80 1 client.15339 shutdown 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.553+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 msgr2=0x556c77dea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.553+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcec00126e0 tx=0x7fcec00129f0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 shutdown_connections 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de1d50 0x556c77df1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0550 0x556c77dea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77df5f90 0x556c77df4a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2431035220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0eb0 0x556c77dea830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcebe7fc700 0 client.15339 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:49.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 >> 172.21.15.5:0/2431035220 conn(0x556c77d54210 msgr2=0x556c77ddb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 shutdown_connections 2023-09-23T04:45:49.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2431035220 wait complete. 2023-09-23T04:45:49.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.554+0000 7fcef0ef7b80 20 client.15339 trim_cache size 0 max 16384 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 Processor -- start 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 -- start start 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de1d50 0x556c77de8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77de9ec0 con 0x556c77de0550 2023-09-23T04:45:49.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77bb38d0 con 0x556c77de0eb0 2023-09-23T04:45:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.572+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77cd94d0 con 0x556c77de1d50 2023-09-23T04:45:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de1d50 0x556c77de8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50760/0 (socket says 172.21.15.5:50760) 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 -- 172.21.15.5:0/2619020530 learned_addr learned my addr 172.21.15.5:0/2619020530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fcecffff700 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 -- 172.21.15.5:0/2619020530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de1d50 msgr2=0x556c77de8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de1d50 0x556c77de8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 -- 172.21.15.5:0/2619020530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 msgr2=0x556c77df1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 -- 172.21.15.5:0/2619020530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c77b46c10 con 0x556c77de0550 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fcecffff700 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.573+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcec8097110 tx=0x7fcec8085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ed251370866cc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcecdffb700 1 -- 172.21.15.5:0/2619020530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec809a000 con 0x556c77de0550 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcecdffb700 1 -- 172.21.15.5:0/2619020530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcec80ba0b0 con 0x556c77de0550 2023-09-23T04:45:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcecdffb700 1 -- 172.21.15.5:0/2619020530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80850b0 con 0x556c77de0550 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2619020530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 msgr2=0x556c77df2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcec8097110 tx=0x7fcec8085040 comp rx=0 tx=0).stop 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2619020530 shutdown_connections 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de1d50 0x556c77de8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de0550 0x556c77df2040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2619020530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2619020530 >> 172.21.15.5:0/2619020530 conn(0x556c77d54210 msgr2=0x556c77dd9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2619020530 shutdown_connections 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.574+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2619020530 wait complete. 2023-09-23T04:45:49.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 -- start start 2023-09-23T04:45:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 Processor -- start 2023-09-23T04:45:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 0x556c77de6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77de7730 con 0x556c77de1d50 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77de78a0 con 0x556c77de0550 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.575+0000 7fcef0ef7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c77de7a60 con 0x556c77de0eb0 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fcef0ef7b80 10 client.0 _ll_register_callbacks cb 0x7fcec0009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50772/0 (socket says 172.21.15.5:50772) 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 -- 172.21.15.5:0/2446500817 learned_addr learned my addr 172.21.15.5:0/2446500817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 0x556c77de6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fcecffff700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 -- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 msgr2=0x556c77df1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 -- 172.21.15.5:0/2446500817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 msgr2=0x556c77de6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 0x556c77de6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced77fe700 1 -- 172.21.15.5:0/2446500817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced81a1270 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.576+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 0x556c77de6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fced77fe700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fd0 tx=0x7fced0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d664e23998f03d86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0036040 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fced000be90 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fced0006000 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fced0028710 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcef0ef7b80 10 client.15345 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556c77b84200 con 0x556c77de1d50 2023-09-23T04:45:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced000be90 con 0x556c77de1d50 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.577+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fced00055a0 con 0x556c77de1d50 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 10 client.15345 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 20 client.15345 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77e4e6c0 0x556c77e4ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 4 client.15345 mds_command: new command op to 24535 tid=0[{"prefix": "status"}] 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c77d2bae0 con 0x556c77e4e6c0 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77e4e6c0 0x556c77e4ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fced6ffd700 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77e4e6c0 0x556c77e4ab80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b22306696a87953 server_cookie=83f7b72168532572 in_seq=0 out_seq=0 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.578+0000 7fcebe7fc700 10 client.15345 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcebe7fc700 1 -- 172.21.15.5:0/2446500817 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556c77d2bae0 con 0x556c77e4e6c0 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcebe7fc700 10 client.15345 handle_command_reply: tid=0 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 client.15345 shutdown 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 msgr2=0x556c77de7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fced0009fd0 tx=0x7fced0003c10 comp rx=0 tx=0).stop 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 shutdown_connections 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x556c77e4e6c0 0x556c77e4ab80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c77de1d50 0x556c77de7160 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556c77de0550 0x556c77de6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 --2- 172.21.15.5:0/2446500817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556c77de0eb0 0x556c77df1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 >> 172.21.15.5:0/2446500817 conn(0x556c77d54210 msgr2=0x556c77de8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcebe7fc700 0 client.15345 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:45:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 shutdown_connections 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 1 -- 172.21.15.5:0/2446500817 wait complete. 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.579+0000 7fcef0ef7b80 20 client.15345 trim_cache size 0 max 16384 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24535, 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:49.585 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 90.790334909999999, 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 142.366374914 2023-09-23T04:45:49.586 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 msgr2=0x7fcec40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 0x7fcec40a3ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fced002a000 tx=0x7fced0027000 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 msgr2=0x7fced8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcec800ea50 tx=0x7fcec800ed60 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 shutdown_connections 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fcec40a1620 0x7fcec40a3ae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fced8102a50 0x7fced81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced8100eb0 0x7fced8198ea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 --2- 172.21.15.5:0/1753157236 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fced8102090 0x7fced81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 >> 172.21.15.5:0/1753157236 conn(0x7fced80fc470 msgr2=0x7fced81079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 shutdown_connections 2023-09-23T04:45:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.580+0000 7fcedd5ff700 1 -- 172.21.15.5:0/1753157236 wait complete. 2023-09-23T04:45:49.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T04:45:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 -- 172.21.15.5:0/2735399499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 msgr2=0x7fe180102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2735399499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe180102290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098f0 tx=0x7fe174009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 -- 172.21.15.5:0/2735399499 shutdown_connections 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2735399499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe180109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2735399499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2735399499 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe180102290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 -- 172.21.15.5:0/2735399499 >> 172.21.15.5:0/2735399499 conn(0x7fe1800fc190 msgr2=0x7fe1800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 -- 172.21.15.5:0/2735399499 shutdown_connections 2023-09-23T04:45:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.807+0000 7fe184d9d700 1 -- 172.21.15.5:0/2735399499 wait complete. 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.808+0000 7fe184d9d700 1 Processor -- start 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 -- start start 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe1801991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe1801a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a07c0 con 0x7fe1801027d0 2023-09-23T04:45:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a0930 con 0x7fe180100c40 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe184d9d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a0af0 con 0x7fe180101e30 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe1801a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52262/0 (socket says 172.21.15.5:52262) 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe17f7fe700 1 -- 172.21.15.5:0/2711878493 learned_addr learned my addr 172.21.15.5:0/2711878493 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.809+0000 7fe17effd700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe1801991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 -- 172.21.15.5:0/2711878493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 msgr2=0x7fe1801991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe1801991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 -- 172.21.15.5:0/2711878493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 msgr2=0x7fe180198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 -- 172.21.15.5:0/2711878493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009520 con 0x7fe1801027d0 2023-09-23T04:45:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe1801a01f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe17800efc0 tx=0x7fe17800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17cff9700 1 -- 172.21.15.5:0/2711878493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178018070 con 0x7fe1801027d0 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1801a0d70 con 0x7fe1801027d0 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17cff9700 1 -- 172.21.15.5:0/2711878493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe17800f040 con 0x7fe1801027d0 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe17cff9700 1 -- 172.21.15.5:0/2711878493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17800ccb0 con 0x7fe1801027d0 2023-09-23T04:45:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.810+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1801a11d0 con 0x7fe1801027d0 2023-09-23T04:45:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.813+0000 7fe17cff9700 1 -- 172.21.15.5:0/2711878493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fe178003a00 con 0x7fe1801027d0 2023-09-23T04:45:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.815+0000 7fe1998d4b80 1 Processor -- start 2023-09-23T04:45:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.815+0000 7fe17cff9700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 0x7fe1680a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.815+0000 7fe17cff9700 1 -- 172.21.15.5:0/2711878493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fe1780ccd00 con 0x7fe1801027d0 2023-09-23T04:45:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.815+0000 7fe1998d4b80 1 -- start start 2023-09-23T04:45:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.815+0000 7fe17effd700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 0x7fe1680a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 0x5579edf89aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89fe0 0x5579edf8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edef1410 con 0x5579edf8aed0 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edee2bd0 con 0x5579edf89680 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edee04f0 con 0x5579edf89fe0 2023-09-23T04:45:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17effd700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 0x7fe1680a3a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe174005e90 tx=0x7fe174019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17ffff700 1 -- 172.21.15.5:0/754966705 learned_addr learned my addr 172.21.15.5:0/754966705 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 0x5579edf89aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.816+0000 7fe17effd700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89fe0 0x5579edf8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 -- 172.21.15.5:0/754966705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89fe0 msgr2=0x5579edf8a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89fe0 0x5579edf8a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 -- 172.21.15.5:0/754966705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 msgr2=0x5579edf89aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 0x5579edf89aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 -- 172.21.15.5:0/754966705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579edf3e880 con 0x5579edf8aed0 2023-09-23T04:45:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 0x5579edf89aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe17ffff700 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1780090a0 tx=0x7fe178099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef593374552144d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe165ffb700 1 -- 172.21.15.5:0/754966705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17809a2f0 con 0x5579edf8aed0 2023-09-23T04:45:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe165ffb700 1 -- 172.21.15.5:0/754966705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe178099bf0 con 0x5579edf8aed0 2023-09-23T04:45:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.817+0000 7fe165ffb700 1 -- 172.21.15.5:0/754966705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178004ad0 con 0x5579edf8aed0 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 -- 172.21.15.5:0/754966705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 msgr2=0x5579edf8b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1780090a0 tx=0x7fe178099800 comp rx=0 tx=0).stop 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 -- 172.21.15.5:0/754966705 shutdown_connections 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8b350 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89680 0x5579edf89aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/754966705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89fe0 0x5579edf8a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 -- 172.21.15.5:0/754966705 >> 172.21.15.5:0/754966705 conn(0x5579edefd210 msgr2=0x5579edf845b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 -- 172.21.15.5:0/754966705 shutdown_connections 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.818+0000 7fe1998d4b80 1 -- 172.21.15.5:0/754966705 wait complete. 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 -- start start 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 Processor -- start 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf93a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf8aed0 0x5579edf9aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579ededbde0 con 0x5579edf89680 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579ede786c0 con 0x5579edf89fe0 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579ede78360 con 0x5579edf8aed0 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe1998d4b80 10 client.0 _ll_register_callbacks cb 0x5579edcd57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.819+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50806/0 (socket says 172.21.15.5:50806) 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf8aed0 0x5579edf9aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 -- 172.21.15.5:0/3960852493 learned_addr learned my addr 172.21.15.5:0/3960852493 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17effd700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf93a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 -- 172.21.15.5:0/3960852493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf8aed0 msgr2=0x5579edf9aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf8aed0 0x5579edf9aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 -- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 msgr2=0x5579edf93a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf93a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 -- 172.21.15.5:0/3960852493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe170009d50 con 0x5579edf89680 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe170012df0 tx=0x7fe17000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b01377dadc0443f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe15effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.820+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fe1700122e0 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579edf7c450 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17000fb80 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 10 client.15357 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe1801a11d0 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579edf7c450 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe170003aa0 con 0x5579edf89680 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 10 client.15357 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 20 client.15357 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edf9f160 0x5579edf9dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 4 client.15357 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.821+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579eded4dc0 con 0x5579edf9f160 2023-09-23T04:45:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.822+0000 7fe17effd700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edf9f160 0x5579edf9dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.822+0000 7fe17effd700 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edf9f160 0x5579edf9dc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c29d8554dce10e server_cookie=57f2f170bdc7fe7a in_seq=0 out_seq=0 2023-09-23T04:45:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.822+0000 7fe15effd700 10 client.15357 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe15effd700 1 -- 172.21.15.5:0/3960852493 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579eded4dc0 con 0x5579edf9f160 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe15effd700 10 client.15357 handle_command_reply: tid=0 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 client.15357 shutdown 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 msgr2=0x5579edf93540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe170012df0 tx=0x7fe17000f6a0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 shutdown_connections 2023-09-23T04:45:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edf9f160 0x5579edf9dc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf8aed0 0x5579edf9aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf89680 0x5579edf93540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/3960852493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf93a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe15effd700 0 client.15357 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 >> 172.21.15.5:0/3960852493 conn(0x5579edefd210 msgr2=0x5579edf83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.823+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 shutdown_connections 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.824+0000 7fe1998d4b80 1 -- 172.21.15.5:0/3960852493 wait complete. 2023-09-23T04:45:49.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.824+0000 7fe1998d4b80 20 client.15357 trim_cache size 0 max 16384 2023-09-23T04:45:49.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.842+0000 7fe1998d4b80 1 Processor -- start 2023-09-23T04:45:49.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.842+0000 7fe1998d4b80 1 -- start start 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.842+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf8de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.842+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edd5c8d0 con 0x5579edf8aed0 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579eded4dc0 con 0x5579edf89fe0 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edf91a00 con 0x5579edf89680 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf8de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50810/0 (socket says 172.21.15.5:50810) 2023-09-23T04:45:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 -- 172.21.15.5:0/2400552368 learned_addr learned my addr 172.21.15.5:0/2400552368 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17effd700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 -- 172.21.15.5:0/2400552368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 msgr2=0x5579edf8de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf8de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 -- 172.21.15.5:0/2400552368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 msgr2=0x5579edf9aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 -- 172.21.15.5:0/2400552368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579edd06d90 con 0x5579edf8aed0 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17effd700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf8de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.843+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe178096660 tx=0x7fe1780093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30893ce441e2d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1667fc700 1 -- 172.21.15.5:0/2400552368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17809a2f0 con 0x5579edf8aed0 2023-09-23T04:45:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1667fc700 1 -- 172.21.15.5:0/2400552368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1780b2280 con 0x5579edf8aed0 2023-09-23T04:45:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1667fc700 1 -- 172.21.15.5:0/2400552368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178098090 con 0x5579edf8aed0 2023-09-23T04:45:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2400552368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 msgr2=0x5579edf8e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe178096660 tx=0x7fe1780093e0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2400552368 shutdown_connections 2023-09-23T04:45:49.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf8e3d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf8de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2400552368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2400552368 >> 172.21.15.5:0/2400552368 conn(0x5579edefd210 msgr2=0x5579edf920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2400552368 shutdown_connections 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.844+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2400552368 wait complete. 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.845+0000 7fe1998d4b80 1 -- start start 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.845+0000 7fe1998d4b80 1 Processor -- start 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.845+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edf972b0 con 0x5579edf8aed0 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edf97420 con 0x5579edf89fe0 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579edf975e0 con 0x5579edf89680 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe1998d4b80 10 client.0 _ll_register_callbacks cb 0x7fe1700128d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50824/0 (socket says 172.21.15.5:50824) 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17ffff700 1 -- 172.21.15.5:0/2873000454 learned_addr learned my addr 172.21.15.5:0/2873000454 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55416/0 (socket says 172.21.15.5:55416) 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17effd700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 -- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 msgr2=0x5579edf9a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 -- 172.21.15.5:0/2873000454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 msgr2=0x5579edf96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17f7fe700 1 -- 172.21.15.5:0/2873000454 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe170021040 con 0x5579edf89680 2023-09-23T04:45:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.846+0000 7fe17ffff700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe17f7fe700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe170003fd0 tx=0x7fe170009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a23c29bfaedca36 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe15effd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170017040 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe178099bf0 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fe17000fb80 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe17000fd40 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe1998d4b80 10 client.25408 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5579edf3e880 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.847+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe178099bf0 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe170010780 con 0x5579edf89680 2023-09-23T04:45:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 10 client.25408 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 20 client.25408 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edff78c0 0x5579edff3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 4 client.25408 mds_command: new command op to 24554 tid=0[{"prefix": "status"}] 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579eded4ae0 con 0x5579edff78c0 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.848+0000 7fe17effd700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edff78c0 0x5579edff3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe17effd700 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edff78c0 0x5579edff3d40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d702bc1d8d10e610 server_cookie=c2d157a4705f99ba in_seq=0 out_seq=0 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe15effd700 10 client.25408 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe15effd700 1 -- 172.21.15.5:0/2873000454 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5579eded4ae0 con 0x5579edff78c0 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe15effd700 10 client.25408 handle_command_reply: tid=0 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 client.25408 shutdown 2023-09-23T04:45:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 msgr2=0x5579edf9a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe170003fd0 tx=0x7fe170009fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 shutdown_connections 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5579edff78c0 0x5579edff3d40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579edf8aed0 0x5579edf96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579edf89680 0x5579edf9a2d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 --2- 172.21.15.5:0/2873000454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579edf89fe0 0x5579edf9a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe15effd700 0 client.25408 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 >> 172.21.15.5:0/2873000454 conn(0x5579edefd210 msgr2=0x5579edf9dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 shutdown_connections 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 1 -- 172.21.15.5:0/2873000454 wait complete. 2023-09-23T04:45:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.849+0000 7fe1998d4b80 20 client.25408 trim_cache size 0 max 16384 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24554, 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 92.066971221000003, 2023-09-23T04:45:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 129.39998375799999 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 msgr2=0x7fe1680a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 0x7fe1680a3a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe174005e90 tx=0x7fe174019040 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 msgr2=0x7fe1801a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe1801a01f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe17800efc0 tx=0x7fe17800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 shutdown_connections 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe1680a15a0 0x7fe1680a3a60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801027d0 0x7fe1801a01f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe180100c40 0x7fe180198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 --2- 172.21.15.5:0/2711878493 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe180101e30 0x7fe1801991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 >> 172.21.15.5:0/2711878493 conn(0x7fe1800fc190 msgr2=0x7fe180107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 shutdown_connections 2023-09-23T04:45:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:49.850+0000 7fe184d9d700 1 -- 172.21.15.5:0/2711878493 wait complete. 2023-09-23T04:45:49.866 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:49 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1269602583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:49.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-23T04:45:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:49 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1269602583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 -- 172.21.15.5:0/3666818237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940074e60 msgr2=0x7fb9400732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 --2- 172.21.15.5:0/3666818237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940074e60 0x7fb9400732b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb930009990 tx=0x7fb930009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 -- 172.21.15.5:0/3666818237 shutdown_connections 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 --2- 172.21.15.5:0/3666818237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940108dc0 0x7fb94010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 --2- 172.21.15.5:0/3666818237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940074e60 0x7fb9400732b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 --2- 172.21.15.5:0/3666818237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 0x7fb940073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 -- 172.21.15.5:0/3666818237 >> 172.21.15.5:0/3666818237 conn(0x7fb9400fc470 msgr2=0x7fb9400fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 -- 172.21.15.5:0/3666818237 shutdown_connections 2023-09-23T04:45:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.072+0000 7fb9465ff700 1 -- 172.21.15.5:0/3666818237 wait complete. 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.073+0000 7fb9465ff700 1 Processor -- start 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.073+0000 7fb9465ff700 1 -- start start 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 0x7fb940198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940074e60 0x7fb940199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9401a0960 con 0x7fb940108dc0 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9401a0aa0 con 0x7fb940073880 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb9465ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9401a0c30 con 0x7fb940074e60 2023-09-23T04:45:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50838/0 (socket says 172.21.15.5:50838) 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb944b9c700 1 -- 172.21.15.5:0/2383216050 learned_addr learned my addr 172.21.15.5:0/2383216050 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940074e60 0x7fb940199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.074+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 0x7fb940198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 -- 172.21.15.5:0/2383216050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940074e60 msgr2=0x7fb940199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940074e60 0x7fb940199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 -- 172.21.15.5:0/2383216050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 msgr2=0x7fb940198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 0x7fb940198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 -- 172.21.15.5:0/2383216050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300095f0 con 0x7fb940108dc0 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ef30 tx=0x7fb93800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb93d7fa700 1 -- 172.21.15.5:0/2383216050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938018070 con 0x7fb940108dc0 2023-09-23T04:45:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb93d7fa700 1 -- 172.21.15.5:0/2383216050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93800f040 con 0x7fb940108dc0 2023-09-23T04:45:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9401a0eb0 con 0x7fb940108dc0 2023-09-23T04:45:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb93d7fa700 1 -- 172.21.15.5:0/2383216050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800cd00 con 0x7fb940108dc0 2023-09-23T04:45:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.075+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9401a1330 con 0x7fb940108dc0 2023-09-23T04:45:50.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.078+0000 7fb93d7fa700 1 -- 172.21.15.5:0/2383216050 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fb938003a00 con 0x7fb940108dc0 2023-09-23T04:45:50.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb93d7fa700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 0x7fb9280a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 Processor -- start 2023-09-23T04:45:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb93d7fa700 1 -- 172.21.15.5:0/2383216050 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fb9380ccdf0 con 0x7fb940108dc0 2023-09-23T04:45:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 -- start start 2023-09-23T04:45:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 0x7fb9280a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0550 0x558fbc8d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc838410 con 0x558fbc8d1d50 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc829bd0 con 0x558fbc8d0eb0 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.080+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc8274f0 con 0x558fbc8d0550 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0550 0x558fbc8d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52316/0 (socket says 172.21.15.5:52316) 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 -- 172.21.15.5:0/1572360204 learned_addr learned my addr 172.21.15.5:0/1572360204 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb944b9c700 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 0x7fb9280a3ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b2e0 tx=0x7fb930005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 -- 172.21.15.5:0/1572360204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0550 msgr2=0x558fbc8d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0550 0x558fbc8d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 -- 172.21.15.5:0/1572360204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 msgr2=0x558fbc8d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 -- 172.21.15.5:0/1572360204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fbc885880 con 0x558fbc8d0eb0 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.081+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb934009d30 tx=0x7fb934011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8acce61278f852c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb9267fc700 1 -- 172.21.15.5:0/1572360204 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934004220 con 0x558fbc8d0eb0 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb9267fc700 1 -- 172.21.15.5:0/1572360204 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb934004380 con 0x558fbc8d0eb0 2023-09-23T04:45:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb9267fc700 1 -- 172.21.15.5:0/1572360204 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340198d0 con 0x558fbc8d0eb0 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 -- 172.21.15.5:0/1572360204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 msgr2=0x558fbc8d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb934009d30 tx=0x7fb934011720 comp rx=0 tx=0).stop 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 -- 172.21.15.5:0/1572360204 shutdown_connections 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0550 0x558fbc8d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/1572360204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8d1770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.082+0000 7fb959ef2b80 1 -- 172.21.15.5:0/1572360204 >> 172.21.15.5:0/1572360204 conn(0x558fbc844210 msgr2=0x558fbc8cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.083+0000 7fb959ef2b80 1 -- 172.21.15.5:0/1572360204 shutdown_connections 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.083+0000 7fb959ef2b80 1 -- 172.21.15.5:0/1572360204 wait complete. 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.083+0000 7fb959ef2b80 1 -- start start 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.083+0000 7fb959ef2b80 1 Processor -- start 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 0x558fbc8da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc822de0 con 0x558fbc8d1d50 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc7bf6c0 con 0x558fbc8d0550 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc7bf360 con 0x558fbc8d0eb0 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb959ef2b80 10 client.0 _ll_register_callbacks cb 0x558fbc61c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50846/0 (socket says 172.21.15.5:50846) 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb944b9c700 1 -- 172.21.15.5:0/2755763130 learned_addr learned my addr 172.21.15.5:0/2755763130 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.084+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 0x558fbc8da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 -- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 msgr2=0x558fbc8da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 0x558fbc8da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 -- 172.21.15.5:0/2755763130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 msgr2=0x558fbc8da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 -- 172.21.15.5:0/2755763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb934009960 con 0x558fbc8d1d50 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 0x558fbc8da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb944b9c700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ca4a0 tx=0x7fb9380996c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb29c99c1c427bfa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb91f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93809a140 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fb93809a3e0 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fbc8c3200 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb93800ce60 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb959ef2b80 10 client.15375 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.085+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fbc64dd90 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fbc8c3200 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb9380b2120 con 0x558fbc8d1d50 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 10 client.15375 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T04:45:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 20 client.15375 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc8e5e30 0x558fbc8e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 4 client.15375 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fbc81bdc0 con 0x558fbc8e5e30 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.086+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc8e5e30 0x558fbc8e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.087+0000 7fb93ffff700 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc8e5e30 0x558fbc8e4900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63d520e1eb709454 server_cookie=89bf2877a2f931e1 in_seq=0 out_seq=0 2023-09-23T04:45:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.087+0000 7fb91f7fe700 10 client.15375 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb91f7fe700 1 -- 172.21.15.5:0/2755763130 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fbc81bdc0 con 0x558fbc8e5e30 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb91f7fe700 10 client.15375 handle_command_reply: tid=0 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 client.15375 shutdown 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 msgr2=0x558fbc8e1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ca4a0 tx=0x7fb9380996c0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 shutdown_connections 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d1d50 0x558fbc8e1850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc8e5e30 0x558fbc8e4900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/2755763130 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d0eb0 0x558fbc8da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 >> 172.21.15.5:0/2755763130 conn(0x558fbc844210 msgr2=0x558fbc8cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb91f7fe700 0 client.15375 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 shutdown_connections 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 1 -- 172.21.15.5:0/2755763130 wait complete. 2023-09-23T04:45:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.088+0000 7fb959ef2b80 20 client.15375 trim_cache size 0 max 16384 2023-09-23T04:45:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 Processor -- start 2023-09-23T04:45:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 -- start start 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc6a38d0 con 0x558fbc8d0550 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc81eed0 con 0x558fbc8d0eb0 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc6afde0 con 0x558fbc8d1d50 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50848/0 (socket says 172.21.15.5:50848) 2023-09-23T04:45:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb93ffff700 1 -- 172.21.15.5:0/484542311 learned_addr learned my addr 172.21.15.5:0/484542311 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.107+0000 7fb944b9c700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 -- 172.21.15.5:0/484542311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 msgr2=0x558fbc8cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 -- 172.21.15.5:0/484542311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 msgr2=0x558fbc8e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 -- 172.21.15.5:0/484542311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fbc636c10 con 0x558fbc8d0550 2023-09-23T04:45:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:50.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb93ffff700 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b960 tx=0x7fb930025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b7d1ee4b91cf8d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb926ffd700 1 -- 172.21.15.5:0/484542311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930035ab0 con 0x558fbc8d0550 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb926ffd700 1 -- 172.21.15.5:0/484542311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb93001e040 con 0x558fbc8d0550 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.108+0000 7fb926ffd700 1 -- 172.21.15.5:0/484542311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930025470 con 0x558fbc8d0550 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- 172.21.15.5:0/484542311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 msgr2=0x558fbc8c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b960 tx=0x7fb930025000 comp rx=0 tx=0).stop 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- 172.21.15.5:0/484542311 shutdown_connections 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0550 0x558fbc8c9eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/484542311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- 172.21.15.5:0/484542311 >> 172.21.15.5:0/484542311 conn(0x558fbc844210 msgr2=0x558fbc8d8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- 172.21.15.5:0/484542311 shutdown_connections 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- 172.21.15.5:0/484542311 wait complete. 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.109+0000 7fb959ef2b80 1 -- start start 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 Processor -- start 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc8e12d0 con 0x558fbc8d0eb0 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc8e1440 con 0x558fbc8d0550 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc8e1600 con 0x558fbc8d1d50 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb959ef2b80 10 client.0 _ll_register_callbacks cb 0x7fb938099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb93f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50862/0 (socket says 172.21.15.5:50862) 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb93f7fe700 1 -- 172.21.15.5:0/3534485353 learned_addr learned my addr 172.21.15.5:0/3534485353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.110+0000 7fb944b9c700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb93ffff700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 -- 172.21.15.5:0/3534485353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 msgr2=0x558fbc8da650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 -- 172.21.15.5:0/3534485353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 msgr2=0x558fbc8e0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 -- 172.21.15.5:0/3534485353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94010be70 con 0x558fbc8d1d50 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb93f7fe700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb944b9c700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8e0d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ef00 tx=0x7fb9380b2330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc963f3965d1eb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb91f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9380993f0 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb934004380 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.111+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fb93809a3e0 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9380991d0 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 10 client.25426 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fbc674200 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb934004380 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb93809eb70 con 0x558fbc8d1d50 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 10 client.25426 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 10 client.25426 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-23T04:45:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 20 client.25426 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc93e000 0x558fbc93a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 4 client.25426 mds_command: new command op to 24559 tid=0[{"prefix": "status"}] 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558fbc81bae0 con 0x558fbc93e000 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.112+0000 7fb93ffff700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc93e000 0x558fbc93a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb93ffff700 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc93e000 0x558fbc93a4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3c42629763d9dcb server_cookie=6a3dfbeaedd5c827 in_seq=0 out_seq=0 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb91f7fe700 10 client.25426 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb91f7fe700 1 -- 172.21.15.5:0/3534485353 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558fbc81bae0 con 0x558fbc93e000 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb91f7fe700 10 client.25426 handle_command_reply: tid=0 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 client.25426 shutdown 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 msgr2=0x558fbc8e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8e0d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ef00 tx=0x7fb9380b2330 comp rx=0 tx=0).stop 2023-09-23T04:45:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 shutdown_connections 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558fbc8d1d50 0x558fbc8e0d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558fbc93e000 0x558fbc93a4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558fbc8d0550 0x558fbc8da650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 --2- 172.21.15.5:0/3534485353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558fbc8d0eb0 0x558fbc8e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb91f7fe700 0 client.25426 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.113+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 >> 172.21.15.5:0/3534485353 conn(0x558fbc844210 msgr2=0x558fbc8c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 shutdown_connections 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb959ef2b80 1 -- 172.21.15.5:0/3534485353 wait complete. 2023-09-23T04:45:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb959ef2b80 20 client.25426 trim_cache size 0 max 16384 2023-09-23T04:45:50.123 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2023-09-23T04:45:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24559, 2023-09-23T04:45:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 93.343687607999996, 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 138.248963076 2023-09-23T04:45:50.124 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 msgr2=0x7fb9280a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 0x7fb9280a3ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b2e0 tx=0x7fb930005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 msgr2=0x7fb9401a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb93800ef30 tx=0x7fb93800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 shutdown_connections 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb9280a1620 0x7fb9280a3ae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb940108dc0 0x7fb9401a0390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb940073880 0x7fb940198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 --2- 172.21.15.5:0/2383216050 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb940074e60 0x7fb940199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.114+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 >> 172.21.15.5:0/2383216050 conn(0x7fb9400fc470 msgr2=0x7fb9400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.115+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 shutdown_connections 2023-09-23T04:45:50.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.115+0000 7fb9465ff700 1 -- 172.21.15.5:0/2383216050 wait complete. 2023-09-23T04:45:50.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-09-23T04:45:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:49 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1269602583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/3032166494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4100eb0 msgr2=0x7f6ff41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/3032166494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4100eb0 0x7f6ff41012d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40098f0 tx=0x7f6fe4009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/3032166494 shutdown_connections 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/3032166494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102a50 0x7f6ff4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/3032166494 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4100eb0 0x7f6ff41012d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/3032166494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4102090 0x7f6ff4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/3032166494 >> 172.21.15.5:0/3032166494 conn(0x7f6ff40fc470 msgr2=0x7f6ff40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/3032166494 shutdown_connections 2023-09-23T04:45:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.338+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/3032166494 wait complete. 2023-09-23T04:45:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.339+0000 7f6ffa3ff700 1 Processor -- start 2023-09-23T04:45:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.339+0000 7f6ffa3ff700 1 -- start start 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102090 0x7f6ff41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 0x7f6ff419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419e810 con 0x7f6ff4100eb0 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419e980 con 0x7f6ff4102090 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ffa3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff419eb40 con 0x7f6ff4102a50 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50878/0 (socket says 172.21.15.5:50878) 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ff3fff700 1 -- 172.21.15.5:0/2917607855 learned_addr learned my addr 172.21.15.5:0/2917607855 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102090 0x7f6ff41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.340+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 0x7f6ff419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 -- 172.21.15.5:0/2917607855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 msgr2=0x7f6ff419e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 0x7f6ff419e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 -- 172.21.15.5:0/2917607855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102090 msgr2=0x7f6ff41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102090 0x7f6ff41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 -- 172.21.15.5:0/2917607855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009520 con 0x7f6ff4100eb0 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 0x7f6ff419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009fd0 tx=0x7f6fe400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff17fa700 1 -- 172.21.15.5:0/2917607855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe401c070 con 0x7f6ff4100eb0 2023-09-23T04:45:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff419edc0 con 0x7f6ff4100eb0 2023-09-23T04:45:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff17fa700 1 -- 172.21.15.5:0/2917607855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe400f7d0 con 0x7f6ff4100eb0 2023-09-23T04:45:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ff17fa700 1 -- 172.21.15.5:0/2917607855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4003a80 con 0x7f6ff4100eb0 2023-09-23T04:45:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.341+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff419f190 con 0x7f6ff4100eb0 2023-09-23T04:45:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.344+0000 7f6ff17fa700 1 -- 172.21.15.5:0/2917607855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f6fe4020030 con 0x7f6ff4100eb0 2023-09-23T04:45:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 Processor -- start 2023-09-23T04:45:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 -- start start 2023-09-23T04:45:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f6ff17fa700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 0x7f6fdc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d61970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f6ff17fa700 1 -- 172.21.15.5:0/2917607855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f6fe40ce370 con 0x7f6ff4100eb0 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4cc9410 con 0x55e5f4d61eb0 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4cbabd0 con 0x55e5f4d62d50 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4cb84f0 con 0x55e5f4d61550 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 0x7f6fdc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.346+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50880/0 (socket says 172.21.15.5:50880) 2023-09-23T04:45:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 -- 172.21.15.5:0/1601093981 learned_addr learned my addr 172.21.15.5:0/1601093981 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d61970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff899c700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 -- 172.21.15.5:0/1601093981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 msgr2=0x55e5f4d61970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d61970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 -- 172.21.15.5:0/1601093981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 msgr2=0x55e5f4d631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 -- 172.21.15.5:0/1601093981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f4d16880 con 0x55e5f4d61eb0 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff899c700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 0x7f6fdc0a3a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0093c0 tx=0x7f6fec009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.347+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009cf0 tx=0x7f6fe8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164d7492150ac0bb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f6fda7fc700 1 -- 172.21.15.5:0/1601093981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8011d80 con 0x55e5f4d61eb0 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f6fda7fc700 1 -- 172.21.15.5:0/1601093981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe80044e0 con 0x55e5f4d61eb0 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f6fda7fc700 1 -- 172.21.15.5:0/1601093981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8019430 con 0x55e5f4d61eb0 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1601093981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 msgr2=0x55e5f4d62770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009cf0 tx=0x7f6fe8011440 comp rx=0 tx=0).stop 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1601093981 shutdown_connections 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d61970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1601093981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d62770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.348+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1601093981 >> 172.21.15.5:0/1601093981 conn(0x55e5f4cd5210 msgr2=0x55e5f4d5c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.349+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1601093981 shutdown_connections 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.349+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1601093981 wait complete. 2023-09-23T04:45:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.349+0000 7f700ddd9b80 1 -- start start 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.349+0000 7f700ddd9b80 1 Processor -- start 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 0x55e5f4d6f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 0x55e5f4d76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4cb3de0 con 0x55e5f4d62d50 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4c506c0 con 0x55e5f4d61550 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4c50360 con 0x55e5f4d61eb0 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f700ddd9b80 10 client.0 _ll_register_callbacks cb 0x55e5f4aad7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 0x55e5f4d6f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff899c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 0x55e5f4d76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55512/0 (socket says 172.21.15.5:55512) 2023-09-23T04:45:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff37fe700 1 -- 172.21.15.5:0/1717039659 learned_addr learned my addr 172.21.15.5:0/1717039659 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1717039659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 msgr2=0x55e5f4d6fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 msgr2=0x55e5f4d76c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 0x55e5f4d76c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1717039659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe8009920 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.350+0000 7f6ff899c700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 0x55e5f4d76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 0x55e5f4d6f6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40cba20 tx=0x7f6fe409ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db09335d05f7f4e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe409b960 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5f4d54200 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f6fe409b260 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fe409f3b0 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f700ddd9b80 10 client.25388 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e5f4aded90 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.351+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5f4d54200 con 0x55e5f4d61550 2023-09-23T04:45:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6fe40b9330 con 0x55e5f4d61550 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 10 client.25388 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 10 client.25388 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 20 client.25388 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4d7ae60 0x55e5f4d79b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 4 client.25388 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5f4cacdc0 con 0x55e5f4d7ae60 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.352+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4d7ae60 0x55e5f4d79b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4d7ae60 0x55e5f4d79b10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5695a04998475128 server_cookie=f4a6d158155da0d in_seq=0 out_seq=0 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f6fd37fe700 10 client.25388 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1717039659 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e5f4cacdc0 con 0x55e5f4d7ae60 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f6fd37fe700 10 client.25388 handle_command_reply: tid=0 2023-09-23T04:45:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 client.25388 shutdown 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 msgr2=0x55e5f4d6f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 0x55e5f4d6f6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40cba20 tx=0x7f6fe409ada0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 shutdown_connections 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4d7ae60 0x55e5f4d79b10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d62d50 0x55e5f4d76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d61550 0x55e5f4d6f6f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f6fd37fe700 0 client.25388 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1717039659 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.353+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 >> 172.21.15.5:0/1717039659 conn(0x55e5f4cd5210 msgr2=0x55e5f4d5c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.354+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 shutdown_connections 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.354+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1717039659 wait complete. 2023-09-23T04:45:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.354+0000 7f700ddd9b80 20 client.25388 trim_cache size 0 max 16384 2023-09-23T04:45:50.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.372+0000 7f700ddd9b80 1 Processor -- start 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.372+0000 7f700ddd9b80 1 -- start start 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4b348d0 con 0x55e5f4d61eb0 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4d5d310 con 0x55e5f4d62d50 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4cacdf0 con 0x55e5f4d61550 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50890/0 (socket says 172.21.15.5:50890) 2023-09-23T04:45:50.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 -- 172.21.15.5:0/2682495573 learned_addr learned my addr 172.21.15.5:0/2682495573 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 -- 172.21.15.5:0/2682495573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 msgr2=0x55e5f4d77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 -- 172.21.15.5:0/2682495573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 msgr2=0x55e5f4d690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff37fe700 1 -- 172.21.15.5:0/2682495573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f4ac7c10 con 0x55e5f4d61eb0 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.373+0000 7f6ff899c700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:45:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801b040 tx=0x7f6fe8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661fe40b56eb3d2d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f6fdaffd700 1 -- 172.21.15.5:0/2682495573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8029040 con 0x55e5f4d61eb0 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f6fdaffd700 1 -- 172.21.15.5:0/2682495573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe8005600 con 0x55e5f4d61eb0 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f6fdaffd700 1 -- 172.21.15.5:0/2682495573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe802bc30 con 0x55e5f4d61eb0 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f700ddd9b80 1 -- 172.21.15.5:0/2682495573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 msgr2=0x55e5f4d76c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801b040 tx=0x7f6fe8009310 comp rx=0 tx=0).stop 2023-09-23T04:45:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f700ddd9b80 1 -- 172.21.15.5:0/2682495573 shutdown_connections 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.374+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61550 0x55e5f4d77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/2682495573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d76c30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 -- 172.21.15.5:0/2682495573 >> 172.21.15.5:0/2682495573 conn(0x55e5f4cd5210 msgr2=0x55e5f4d5aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 -- 172.21.15.5:0/2682495573 shutdown_connections 2023-09-23T04:45:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 -- 172.21.15.5:0/2682495573 wait complete. 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 -- start start 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.375+0000 7f700ddd9b80 1 Processor -- start 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d75c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4d76800 con 0x55e5f4d61550 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4d76970 con 0x55e5f4d62d50 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f4d76b30 con 0x55e5f4d61eb0 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f700ddd9b80 10 client.0 _ll_register_callbacks cb 0x7f6fe4096e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50896/0 (socket says 172.21.15.5:50896) 2023-09-23T04:45:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1290595769 learned_addr learned my addr 172.21.15.5:0/1290595769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d75c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff899c700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1290595769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 msgr2=0x55e5f4d75c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d75c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1290595769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 msgr2=0x55e5f4d761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 -- 172.21.15.5:0/1290595769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5f4d16880 con 0x55e5f4d61550 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6ff3fff700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6fe409af60 tx=0x7f6fe409aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5545adeece26f227 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.376+0000 7f6fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40b94a0 con 0x55e5f4d61550 2023-09-23T04:45:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fe80044e0 con 0x55e5f4d61550 2023-09-23T04:45:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f6fe409b960 con 0x55e5f4d61550 2023-09-23T04:45:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fe409b260 con 0x55e5f4d61550 2023-09-23T04:45:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 10 client.15399 fetch_fsmap learned FSMap version 27 2023-09-23T04:45:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e5f4b05200 con 0x55e5f4d61550 2023-09-23T04:45:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe80044e0 con 0x55e5f4d61550 2023-09-23T04:45:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6fe409a750 con 0x55e5f4d61550 2023-09-23T04:45:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:45:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 10 client.15399 resolve_mds: resolved name 'g' to daemon mds.g 2023-09-23T04:45:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 20 client.15399 populate_metadata read hostname 'smithi005' 2023-09-23T04:45:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4dd2d10 0x55e5f4dcf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 4 client.15399 mds_command: new command op to 24577 tid=0[{"prefix": "status"}] 2023-09-23T04:45:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.377+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5f4cacae0 con 0x55e5f4dd2d10 2023-09-23T04:45:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4dd2d10 0x55e5f4dcf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:50.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f6ff37fe700 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4dd2d10 0x55e5f4dcf410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11257f5725384d2 server_cookie=7da90280cf24856e in_seq=0 out_seq=0 2023-09-23T04:45:50.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f6fd37fe700 10 client.15399 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f6fd37fe700 1 -- 172.21.15.5:0/1290595769 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e5f4cacae0 con 0x55e5f4dd2d10 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f6fd37fe700 10 client.15399 handle_command_reply: tid=0 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.378+0000 7f700ddd9b80 1 client.15399 shutdown 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 msgr2=0x55e5f4d6fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6fe409af60 tx=0x7f6fe409aec0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 shutdown_connections 2023-09-23T04:45:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e5f4dd2d10 0x55e5f4dcf410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5f4d62d50 0x55e5f4d761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f4d61550 0x55e5f4d6fb50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f6fd37fe700 0 client.15399 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:45:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 --2- 172.21.15.5:0/1290595769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e5f4d61eb0 0x55e5f4d75c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 >> 172.21.15.5:0/1290595769 conn(0x55e5f4cd5210 msgr2=0x55e5f4d690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 shutdown_connections 2023-09-23T04:45:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 1 -- 172.21.15.5:0/1290595769 wait complete. 2023-09-23T04:45:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.379+0000 7f700ddd9b80 20 client.15399 trim_cache size 0 max 16384 2023-09-23T04:45:50.395 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:45:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "a18d5e54-59ca-11ee-8db4-212e2dc638e7", 2023-09-23T04:45:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-09-23T04:45:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24577, 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 122.790848952, 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2023-09-23T04:45:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2023-09-23T04:45:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 87, 2023-09-23T04:45:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 135.32567978700001 2023-09-23T04:45:50.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:45:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 msgr2=0x7f6fdc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 0x7f6fdc0a3a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0093c0 tx=0x7f6fec009920 comp rx=0 tx=0).stop 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 msgr2=0x7f6ff4196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009fd0 tx=0x7f6fe400b840 comp rx=0 tx=0).stop 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 shutdown_connections 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f6fdc0a15a0 0x7f6fdc0a3a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ff4102a50 0x7f6ff419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff4100eb0 0x7f6ff4196ca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 --2- 172.21.15.5:0/2917607855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ff4102090 0x7f6ff41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 >> 172.21.15.5:0/2917607855 conn(0x7f6ff40fc470 msgr2=0x7f6ff40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 shutdown_connections 2023-09-23T04:45:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:50.380+0000 7f6ffa3ff700 1 -- 172.21.15.5:0/2917607855 wait complete. 2023-09-23T04:45:50.402 INFO:teuthology.run_tasks:Running task kclient... 2023-09-23T04:45:50.416 INFO:tasks.kclient:Mounting kernel clients... 2023-09-23T04:45:50.416 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-23T04:45:50.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:50.416 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-09-23T04:45:50.460 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-23T04:45:50.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:50.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T04:45:50.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:50.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-09-23T04:45:50.693 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-09-23T04:45:50.693 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T04:45:50.693 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T04:45:50.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph fs ls 2023-09-23T04:45:50.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:50 smithi005 ceph-mon[97638]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:50 smithi069 ceph-mon[133931]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:50 smithi118 ceph-mon[134297]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-23T04:45:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T04:45:52.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 -- 172.21.15.5:0/3996187623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7201010a0 msgr2=0x7fc720101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:52.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 --2- 172.21.15.5:0/3996187623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7201010a0 0x7fc720101480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009990 tx=0x7fc71c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 -- 172.21.15.5:0/3996187623 shutdown_connections 2023-09-23T04:45:52.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 --2- 172.21.15.5:0/3996187623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720106180 0x7fc72010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 --2- 172.21.15.5:0/3996187623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7201010a0 0x7fc720101480 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 --2- 172.21.15.5:0/3996187623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101a50 0x7fc720105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 -- 172.21.15.5:0/3996187623 >> 172.21.15.5:0/3996187623 conn(0x7fc7200fc910 msgr2=0x7fc7200fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 -- 172.21.15.5:0/3996187623 shutdown_connections 2023-09-23T04:45:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.943+0000 7fc72727d700 1 -- 172.21.15.5:0/3996187623 wait complete. 2023-09-23T04:45:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.944+0000 7fc72727d700 1 Processor -- start 2023-09-23T04:45:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.944+0000 7fc72727d700 1 -- start start 2023-09-23T04:45:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.944+0000 7fc72727d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.944+0000 7fc72727d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720101a50 0x7fc7201974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.944+0000 7fc72727d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 0x7fc72019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc72727d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019bf40 con 0x7fc720106180 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc72727d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019c0b0 con 0x7fc7201010a0 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc72727d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019c270 con 0x7fc720101a50 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc72581a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 0x7fc72019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:36202/0 (socket says 172.21.15.5:36202) 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 -- 172.21.15.5:0/3020001289 learned_addr learned my addr 172.21.15.5:0/3020001289 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc724818700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720101a50 0x7fc7201974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 -- 172.21.15.5:0/3020001289 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720101a50 msgr2=0x7fc7201974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720101a50 0x7fc7201974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 -- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 msgr2=0x7fc72019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 0x7fc72019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc725019700 1 -- 172.21.15.5:0/3020001289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0095f0 con 0x7fc7201010a0 2023-09-23T04:45:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.945+0000 7fc72581a700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 0x7fc72019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc725019700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005910 tx=0x7fc71c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c01d070 con 0x7fc7201010a0 2023-09-23T04:45:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72019c4f0 con 0x7fc7201010a0 2023-09-23T04:45:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc71c022470 con 0x7fc7201010a0 2023-09-23T04:45:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017650 con 0x7fc7201010a0 2023-09-23T04:45:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.946+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7201a3db0 con 0x7fc7201010a0 2023-09-23T04:45:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.947+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72004eb20 con 0x7fc7201010a0 2023-09-23T04:45:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.949+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fc71c0225e0 con 0x7fc7201010a0 2023-09-23T04:45:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.950+0000 7fc7167fc700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 0x7fc70c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.950+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fc71c0ce520 con 0x7fc7201010a0 2023-09-23T04:45:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.950+0000 7fc724818700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 0x7fc70c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.951+0000 7fc724818700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 0x7fc70c0a3a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc7201985a0 tx=0x7fc710009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:52.953+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fc71c097750 con 0x7fc7201010a0 2023-09-23T04:45:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:52 smithi005 ceph-mon[97638]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:52 smithi118 ceph-mon[134297]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.208+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc720110de0 con 0x7fc7201010a0 2023-09-23T04:45:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.209+0000 7fc7167fc700 1 -- 172.21.15.5:0/3020001289 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc71c09ad70 con 0x7fc7201010a0 2023-09-23T04:45:53.210 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 msgr2=0x7fc70c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 0x7fc70c0a3a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc7201985a0 tx=0x7fc710009260 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 msgr2=0x7fc720196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005910 tx=0x7fc71c004710 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 shutdown_connections 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fc70c0a15a0 0x7fc70c0a3a60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc720106180 0x7fc72019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7201010a0 0x7fc720196f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 --2- 172.21.15.5:0/3020001289 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc720101a50 0x7fc7201974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 >> 172.21.15.5:0/3020001289 conn(0x7fc7200fc910 msgr2=0x7fc7200fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:53.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 shutdown_connections 2023-09-23T04:45:53.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:53.211+0000 7fc72727d700 1 -- 172.21.15.5:0/3020001289 wait complete. 2023-09-23T04:45:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:52 smithi069 ceph-mon[133931]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T04:45:53.713 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T04:45:53.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T04:45:53.714 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T04:45:53.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:53.714 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-09-23T04:45:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41665sec preferred_lft 41665sec 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T04:45:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-09-23T04:45:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-09-23T04:45:53.762 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T04:45:53.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T04:45:53.763 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:45:53.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:53 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3020001289' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:45:54.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.015 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T04:45:54.086 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-09-23T04:45:54.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.090 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-09-23T04:45:54.137 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-09-23T04:45:54.137 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T04:45:54.137 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-23T04:45:54.137 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-23T04:45:54.137 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T04:45:54.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-23T04:45:54.138 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:45:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:53 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3020001289' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:45:54.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:53 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3020001289' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-23T04:45:54.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.314 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-09-23T04:45:54.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.359 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2023-09-23T04:45:54.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.444 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:45:54.444 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:45:54.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T04:45:54.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-23T04:45:54.445 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:45:54.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-23T04:45:54.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.631 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:45:54.631 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:45:54.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-23T04:45:54.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T04:45:54.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-23T04:45:54.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-23T04:45:54.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-23T04:45:54.632 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:45:54.839 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:54 smithi005 ceph-mon[97638]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:54.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:54.885 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-09-23T04:45:54.886 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-09-23T04:45:54.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2023-09-23T04:45:54.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T04:45:54.886 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-09-23T04:45:55.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T04:45:55.059 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T04:45:55.102 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T04:45:55.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T04:45:55.103 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T04:45:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:54 smithi118 ceph-mon[134297]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:55.185 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-23T04:45:55.186 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-23T04:45:55.186 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/089c9db7-dbed-4b4e-aa4f-fbdfdd050c26 2023-09-23T04:45:55.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:55.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/089c9db7-dbed-4b4e-aa4f-fbdfdd050c26 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-09-23T04:45:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:54 smithi069 ceph-mon[133931]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:45:55.332 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-09-23T04:45:55.333 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-09-23T04:45:55.793 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-09-23T04:45:55.793 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-09-23T04:45:55.793 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.69:6789,172.21.15.118:6789" 2023-09-23T04:45:55.793 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@a18d5e54-59ca-11ee-8db4-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/089c9db7-dbed-4b4e-aa4f-fbdfdd050c26 2023-09-23T04:45:55.794 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.5:6789/172.21.15.69:6789/172.21.15.118:6789" will pass to kernel 2023-09-23T04:45:55.799 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-23T04:45:55.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:55.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T04:45:55.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:55.876 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T04:45:55.962 INFO:teuthology.orchestra.run.smithi005.stdout:client15408 2023-09-23T04:45:55.962 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:45:55.963 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T04:45:56.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:56.007 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T04:45:56.133 INFO:teuthology.orchestra.run.smithi005.stdout:client15408 2023-09-23T04:45:56.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:56.133 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-23T04:45:56.219 INFO:teuthology.orchestra.run.smithi005.stdout:client15408 2023-09-23T04:45:56.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T04:45:56.219 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7.client15408/status') 2023-09-23T04:45:56.328 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.15408 (3)192.168.0.1:0/1189186331 2023-09-23T04:45:56.328 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2023-09-23T04:45:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2023-09-23T04:45:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2023-09-23T04:45:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 5.5252e-05 s, 1.3 MB/s 2023-09-23T04:45:56.331 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-23T04:45:56.343 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-23T04:45:56.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:56.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:56 smithi005 ceph-mon[97638]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 -- 172.21.15.5:0/3627585849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 msgr2=0x7f9340109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 --2- 172.21.15.5:0/3627585849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f9340109360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f933800b0a0 tx=0x7f933800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 -- 172.21.15.5:0/3627585849 shutdown_connections 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 --2- 172.21.15.5:0/3627585849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f9340109360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 --2- 172.21.15.5:0/3627585849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 0x7f93401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 --2- 172.21.15.5:0/3627585849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 -- 172.21.15.5:0/3627585849 >> 172.21.15.5:0/3627585849 conn(0x7f93400fc480 msgr2=0x7f93400fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 -- 172.21.15.5:0/3627585849 shutdown_connections 2023-09-23T04:45:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.622+0000 7f9346dff700 1 -- 172.21.15.5:0/3627585849 wait complete. 2023-09-23T04:45:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.623+0000 7f9346dff700 1 Processor -- start 2023-09-23T04:45:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 -- start start 2023-09-23T04:45:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 0x7f9340198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f93401a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0a90 con 0x7f9340102a40 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0bd0 con 0x7f9340102080 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9346dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401a0d60 con 0x7f9340100ea0 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f933ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f933ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:36222/0 (socket says 172.21.15.5:36222) 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f933ffff700 1 -- 172.21.15.5:0/641221360 learned_addr learned my addr 172.21.15.5:0/641221360 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f934539c700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f93401a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.624+0000 7f9344b9b700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 0x7f9340198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 -- 172.21.15.5:0/641221360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 msgr2=0x7f9340198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 0x7f9340198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 -- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 msgr2=0x7f93401a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f93401a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 -- 172.21.15.5:0/641221360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009d00 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933ffff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e970 tx=0x7f932c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c009790 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93401a0fe0 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c00c7e0 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c003c70 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.625+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93401a1510 con 0x7f9340102080 2023-09-23T04:45:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.626+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934010c760 con 0x7f9340102080 2023-09-23T04:45:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.628+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f932c00cc50 con 0x7f9340102080 2023-09-23T04:45:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.630+0000 7f933dffb700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 0x7f93300a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.630+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f932c014070 con 0x7f9340102080 2023-09-23T04:45:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.630+0000 7f9344b9b700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 0x7f93300a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.631+0000 7f9344b9b700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 0x7f93300a3a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9340101bf0 tx=0x7f9334009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.632+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f932c096670 con 0x7f9340102080 2023-09-23T04:45:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.847+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93401a1760 con 0x7f9340102080 2023-09-23T04:45:56.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.849+0000 7f933dffb700 1 -- 172.21.15.5:0/641221360 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7f932c003dd0 con 0x7f9340102080 2023-09-23T04:45:56.850 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:56.850 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 msgr2=0x7f93300a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 0x7f93300a3a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9340101bf0 tx=0x7f9334009260 comp rx=0 tx=0).stop 2023-09-23T04:45:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 msgr2=0x7f9340199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f932c00e970 tx=0x7f932c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 shutdown_connections 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93300a15a0 0x7f93300a3a60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9340102a40 0x7f93401a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9340100ea0 0x7f9340198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 --2- 172.21.15.5:0/641221360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9340102080 0x7f9340199450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 >> 172.21.15.5:0/641221360 conn(0x7f93400fc480 msgr2=0x7f93400fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 shutdown_connections 2023-09-23T04:45:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:56.851+0000 7f9346dff700 1 -- 172.21.15.5:0/641221360 wait complete. 2023-09-23T04:45:56.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:56.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:56.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:56 smithi118 ceph-mon[134297]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:56 smithi069 ceph-mon[133931]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-23T04:45:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 -- 172.21.15.5:0/2238419223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 msgr2=0x7f59d8106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 --2- 172.21.15.5:0/2238419223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d8106690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009990 tx=0x7f59c4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 -- 172.21.15.5:0/2238419223 shutdown_connections 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 --2- 172.21.15.5:0/2238419223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 0x7f59d8109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 --2- 172.21.15.5:0/2238419223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 --2- 172.21.15.5:0/2238419223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d8106690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 -- 172.21.15.5:0/2238419223 >> 172.21.15.5:0/2238419223 conn(0x7f59d80fb3a0 msgr2=0x7f59d80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.078+0000 7f59dedff700 1 -- 172.21.15.5:0/2238419223 shutdown_connections 2023-09-23T04:45:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.079+0000 7f59dedff700 1 -- 172.21.15.5:0/2238419223 wait complete. 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 Processor -- start 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 -- start start 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 0x7f59d81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d81a08f0 con 0x7f59d81042a0 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d81a0a60 con 0x7f59d8101970 2023-09-23T04:45:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dedff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d81a0c20 con 0x7f59d8106bd0 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55336/0 (socket says 172.21.15.5:55336) 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59d7fff700 1 -- 172.21.15.5:0/4012339341 learned_addr learned my addr 172.21.15.5:0/4012339341 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.080+0000 7f59dcb9b700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dd39c700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 0x7f59d81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 -- 172.21.15.5:0/4012339341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 msgr2=0x7f59d81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 0x7f59d81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 -- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 msgr2=0x7f59d81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 -- 172.21.15.5:0/4012339341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c40095f0 con 0x7f59d8101970 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59d7fff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dcb9b700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8198db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e9f0 tx=0x7f59cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00c9c0 con 0x7f59d8101970 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d81a0f00 con 0x7f59d8101970 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59cc010430 con 0x7f59d8101970 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc018680 con 0x7f59d8101970 2023-09-23T04:45:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.081+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d81a1320 con 0x7f59d8101970 2023-09-23T04:45:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.083+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d81087a0 con 0x7f59d8101970 2023-09-23T04:45:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.085+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f59cc010860 con 0x7f59d8101970 2023-09-23T04:45:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.086+0000 7f59d5ffb700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 0x7f59c80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.086+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f59cc014070 con 0x7f59d8101970 2023-09-23T04:45:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.086+0000 7f59d7fff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 0x7f59c80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.087+0000 7f59d7fff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 0x7f59c80a3a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006010 tx=0x7f59c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.088+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f59cc0965d0 con 0x7f59d8101970 2023-09-23T04:45:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.301+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59d810c890 con 0x7f59d8101970 2023-09-23T04:45:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.302+0000 7f59d5ffb700 1 -- 172.21.15.5:0/4012339341 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59cc099bf0 con 0x7f59d8101970 2023-09-23T04:45:57.303 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 msgr2=0x7f59c80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 0x7f59c80a3a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59c4006010 tx=0x7f59c4005960 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 msgr2=0x7f59d8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8198db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e9f0 tx=0x7f59cc00ed00 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 shutdown_connections 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f59c80a15a0 0x7f59c80a3a60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f59d8106bd0 0x7f59d81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59d8101970 0x7f59d8198db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 --2- 172.21.15.5:0/4012339341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d81042a0 0x7f59d81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 >> 172.21.15.5:0/4012339341 conn(0x7f59d80fb3a0 msgr2=0x7f59d81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 shutdown_connections 2023-09-23T04:45:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.304+0000 7f59dedff700 1 -- 172.21.15.5:0/4012339341 wait complete. 2023-09-23T04:45:57.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.531+0000 7f7162fff700 1 -- 172.21.15.5:0/215445672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 msgr2=0x7f715c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.531+0000 7f7162fff700 1 --2- 172.21.15.5:0/215445672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c109370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f715400b0a0 tx=0x7f715400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 -- 172.21.15.5:0/215445672 shutdown_connections 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 --2- 172.21.15.5:0/215445672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c109370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 --2- 172.21.15.5:0/215445672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c100eb0 0x7f715c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 --2- 172.21.15.5:0/215445672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c102090 0x7f715c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 -- 172.21.15.5:0/215445672 >> 172.21.15.5:0/215445672 conn(0x7f715c0fc470 msgr2=0x7f715c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 -- 172.21.15.5:0/215445672 shutdown_connections 2023-09-23T04:45:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.532+0000 7f7162fff700 1 -- 172.21.15.5:0/215445672 wait complete. 2023-09-23T04:45:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 Processor -- start 2023-09-23T04:45:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 -- start start 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 0x7f715c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c102090 0x7f715c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1a0a80 con 0x7f715c102a50 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1a0bc0 con 0x7f715c100eb0 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.533+0000 7f7162fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1a0d50 con 0x7f715c102090 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f7160d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 0x7f715c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55356/0 (socket says 172.21.15.5:55356) 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 -- 172.21.15.5:0/2042431119 learned_addr learned my addr 172.21.15.5:0/2042431119 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f715bfff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c102090 0x7f715c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 -- 172.21.15.5:0/2042431119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c102090 msgr2=0x7f715c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c102090 0x7f715c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 -- 172.21.15.5:0/2042431119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 msgr2=0x7f715c198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 0x7f715c198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 -- 172.21.15.5:0/2042431119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009d00 con 0x7f715c102a50 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f7160d9b700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 0x7f715c198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.534+0000 7f716159c700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7154009fd0 tx=0x7f715400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.535+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e040 con 0x7f715c102a50 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.535+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71540094a0 con 0x7f715c102a50 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.535+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c1a0fd0 con 0x7f715c102a50 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.535+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715401b6c0 con 0x7f715c102a50 2023-09-23T04:45:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.535+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c1a13f0 con 0x7f715c102a50 2023-09-23T04:45:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.536+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c10c770 con 0x7f715c102a50 2023-09-23T04:45:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.538+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f715401d470 con 0x7f715c102a50 2023-09-23T04:45:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.539+0000 7f7159ffb700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 0x7f714c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.539+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f71540ce260 con 0x7f715c102a50 2023-09-23T04:45:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.539+0000 7f7160d9b700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 0x7f714c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.540+0000 7f7160d9b700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 0x7f714c0a3ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f715c101c00 tx=0x7f7150009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.543+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7154097490 con 0x7f715c102a50 2023-09-23T04:45:57.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:57 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/641221360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:57.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:57 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4012339341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.758+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f715c1a15e0 con 0x7f715c102a50 2023-09-23T04:45:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.758+0000 7f7159ffb700 1 -- 172.21.15.5:0/2042431119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f715409aab0 con 0x7f715c102a50 2023-09-23T04:45:57.760 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.760+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 msgr2=0x7f714c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.760+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 0x7f714c0a3ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f715c101c00 tx=0x7f7150009260 comp rx=0 tx=0).stop 2023-09-23T04:45:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.760+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 msgr2=0x7f715c1a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.760+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7154009fd0 tx=0x7f715400bce0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 shutdown_connections 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f714c0a1620 0x7f714c0a3ae0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f715c102a50 0x7f715c1a04b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f715c100eb0 0x7f715c198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 --2- 172.21.15.5:0/2042431119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f715c102090 0x7f715c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 >> 172.21.15.5:0/2042431119 conn(0x7f715c0fc470 msgr2=0x7f715c107960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 shutdown_connections 2023-09-23T04:45:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.761+0000 7f7162fff700 1 -- 172.21.15.5:0/2042431119 wait complete. 2023-09-23T04:45:57.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:45:57.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:57 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/641221360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:57.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:57 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4012339341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.987+0000 7f1263bff700 1 -- 172.21.15.5:0/3774489792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102a50 msgr2=0x7f125c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.987+0000 7f1263bff700 1 --2- 172.21.15.5:0/3774489792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102a50 0x7f125c109370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1254009f90 tx=0x7f125400b330 comp rx=0 tx=0).stop 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 -- 172.21.15.5:0/3774489792 shutdown_connections 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 --2- 172.21.15.5:0/3774489792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102a50 0x7f125c109370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 --2- 172.21.15.5:0/3774489792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c100eb0 0x7f125c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 --2- 172.21.15.5:0/3774489792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c102090 0x7f125c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 -- 172.21.15.5:0/3774489792 >> 172.21.15.5:0/3774489792 conn(0x7f125c0fc470 msgr2=0x7f125c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 -- 172.21.15.5:0/3774489792 shutdown_connections 2023-09-23T04:45:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.988+0000 7f1263bff700 1 -- 172.21.15.5:0/3774489792 wait complete. 2023-09-23T04:45:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 Processor -- start 2023-09-23T04:45:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 -- start start 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c100eb0 0x7f125c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102090 0x7f125c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1a0a50 con 0x7f125c102a50 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1a0b90 con 0x7f125c102090 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f1263bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1a0d20 con 0x7f125c100eb0 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f126219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f126219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55374/0 (socket says 172.21.15.5:55374) 2023-09-23T04:45:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.989+0000 7f126219c700 1 -- 172.21.15.5:0/630884821 learned_addr learned my addr 172.21.15.5:0/630884821 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126199b700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c100eb0 0x7f125c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126119a700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102090 0x7f125c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 -- 172.21.15.5:0/630884821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c100eb0 msgr2=0x7f125c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c100eb0 0x7f125c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 -- 172.21.15.5:0/630884821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102090 msgr2=0x7f125c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102090 0x7f125c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 -- 172.21.15.5:0/630884821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254009c40 con 0x7f125c102a50 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f126219c700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1254009f60 tx=0x7f1254004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125400e040 con 0x7f125c102a50 2023-09-23T04:45:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125400bbf0 con 0x7f125c102a50 2023-09-23T04:45:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c1a0fa0 con 0x7f125c102a50 2023-09-23T04:45:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.990+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12540052a0 con 0x7f125c102a50 2023-09-23T04:45:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.991+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c1a13c0 con 0x7f125c102a50 2023-09-23T04:45:57.994 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:57 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/641221360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:57.994 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:57 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4012339341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:45:57.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.992+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125c10c720 con 0x7f125c102a50 2023-09-23T04:45:57.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.994+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f125401d470 con 0x7f125c102a50 2023-09-23T04:45:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.995+0000 7f1252ffd700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 0x7f12480a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.995+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f12540ce7b0 con 0x7f125c102a50 2023-09-23T04:45:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.995+0000 7f126199b700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 0x7f12480a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.996+0000 7f126199b700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 0x7f12480a3a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f125c101c00 tx=0x7f1258009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:57.999+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f12540979e0 con 0x7f125c102a50 2023-09-23T04:45:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.211+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f125c1a15b0 con 0x7f125c102a50 2023-09-23T04:45:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.212+0000 7f1252ffd700 1 -- 172.21.15.5:0/630884821 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f125409b000 con 0x7f125c102a50 2023-09-23T04:45:58.213 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 msgr2=0x7f12480a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 0x7f12480a3a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f125c101c00 tx=0x7f1258009310 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 msgr2=0x7f125c1a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1254009f60 tx=0x7f1254004d90 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 shutdown_connections 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f12480a15a0 0x7f12480a3a60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c102a50 0x7f125c1a0480 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f125c100eb0 0x7f125c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 --2- 172.21.15.5:0/630884821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f125c102090 0x7f125c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 >> 172.21.15.5:0/630884821 conn(0x7f125c0fc470 msgr2=0x7f125c107960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 shutdown_connections 2023-09-23T04:45:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.215+0000 7f1263bff700 1 -- 172.21.15.5:0/630884821 wait complete. 2023-09-23T04:45:58.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:45:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 -- 172.21.15.5:0/112429115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0102090 msgr2=0x7f49d0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 --2- 172.21.15.5:0/112429115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0102090 0x7f49d0102510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009990 tx=0x7f49c4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 -- 172.21.15.5:0/112429115 shutdown_connections 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 --2- 172.21.15.5:0/112429115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102a50 0x7f49d0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 --2- 172.21.15.5:0/112429115 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0100eb0 0x7f49d01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 --2- 172.21.15.5:0/112429115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0102090 0x7f49d0102510 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 -- 172.21.15.5:0/112429115 >> 172.21.15.5:0/112429115 conn(0x7f49d00fc470 msgr2=0x7f49d00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 -- 172.21.15.5:0/112429115 shutdown_connections 2023-09-23T04:45:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.440+0000 7f49d49ff700 1 -- 172.21.15.5:0/112429115 wait complete. 2023-09-23T04:45:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.441+0000 7f49d49ff700 1 Processor -- start 2023-09-23T04:45:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.441+0000 7f49d49ff700 1 -- start start 2023-09-23T04:45:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102090 0x7f49d0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 0x7f49d0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0073ef0 con 0x7f49d0100eb0 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d0074060 con 0x7f49d0102090 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49d49ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d01a0c70 con 0x7f49d0102a50 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55386/0 (socket says 172.21.15.5:55386) 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 -- 172.21.15.5:0/2426754260 learned_addr learned my addr 172.21.15.5:0/2426754260 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49cdd9b700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102090 0x7f49d0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ced9d700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 0x7f49d0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 -- 172.21.15.5:0/2426754260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 msgr2=0x7f49d0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 0x7f49d0073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 -- 172.21.15.5:0/2426754260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102090 msgr2=0x7f49d0073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102090 0x7f49d0073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ce59c700 1 -- 172.21.15.5:0/2426754260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40095f0 con 0x7f49d0100eb0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.442+0000 7f49ced9d700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 0x7f49d0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49ce59c700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e970 tx=0x7f49c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009790 con 0x7f49d0100eb0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49c000eec0 con 0x7f49d0100eb0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d01a0f50 con 0x7f49d0100eb0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0003cf0 con 0x7f49d0100eb0 2023-09-23T04:45:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.443+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d01a1370 con 0x7f49d0100eb0 2023-09-23T04:45:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.444+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d0107eb0 con 0x7f49d0100eb0 2023-09-23T04:45:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.446+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f49c000ccd0 con 0x7f49d0100eb0 2023-09-23T04:45:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.447+0000 7f49bf7fe700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 0x7f49b80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.448+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f49c0014070 con 0x7f49d0100eb0 2023-09-23T04:45:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.448+0000 7f49cdd9b700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 0x7f49b80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.448+0000 7f49cdd9b700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 0x7f49b80a3a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b2e0 tx=0x7f49c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.452+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f49c0096740 con 0x7f49d0100eb0 2023-09-23T04:45:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.665+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f49d002d2a0 con 0x7f49d0100eb0 2023-09-23T04:45:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.665+0000 7f49bf7fe700 1 -- 172.21.15.5:0/2426754260 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f49c0099d60 con 0x7f49d0100eb0 2023-09-23T04:45:58.666 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 msgr2=0x7f49b80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 0x7f49b80a3a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b2e0 tx=0x7f49c4005fb0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 msgr2=0x7f49d0074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e970 tx=0x7f49c000ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 shutdown_connections 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49b80a15a0 0x7f49b80a3a60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49d0102a50 0x7f49d0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d0100eb0 0x7f49d0074bd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 --2- 172.21.15.5:0/2426754260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49d0102090 0x7f49d0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 >> 172.21.15.5:0/2426754260 conn(0x7f49d00fc470 msgr2=0x7f49d00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 shutdown_connections 2023-09-23T04:45:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.668+0000 7f49d49ff700 1 -- 172.21.15.5:0/2426754260 wait complete. 2023-09-23T04:45:58.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:45:58.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:58 smithi005 ceph-mon[97638]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-23T04:45:58.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2042431119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:58.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:58 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/630884821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.891+0000 7f838ddff700 1 -- 172.21.15.5:0/1011880726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388100e90 msgr2=0x7f83881012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.891+0000 7f838ddff700 1 --2- 172.21.15.5:0/1011880726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388100e90 0x7f83881012b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83780098f0 tx=0x7f8378009c00 comp rx=0 tx=0).stop 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 -- 172.21.15.5:0/1011880726 shutdown_connections 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 --2- 172.21.15.5:0/1011880726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388102a30 0x7f8388109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 --2- 172.21.15.5:0/1011880726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388100e90 0x7f83881012b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 --2- 172.21.15.5:0/1011880726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f83881024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 -- 172.21.15.5:0/1011880726 >> 172.21.15.5:0/1011880726 conn(0x7f83880fc450 msgr2=0x7f83880fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 -- 172.21.15.5:0/1011880726 shutdown_connections 2023-09-23T04:45:58.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.892+0000 7f838ddff700 1 -- 172.21.15.5:0/1011880726 wait complete. 2023-09-23T04:45:58.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 Processor -- start 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 -- start start 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 0x7f8388074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388102a30 0x7f8388073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388073ef0 con 0x7f8388102070 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388074060 con 0x7f8388100e90 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f838ddff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83881a0cf0 con 0x7f8388102a30 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f8386ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f8386ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55396/0 (socket says 172.21.15.5:55396) 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f8386ffd700 1 -- 172.21.15.5:0/363579618 learned_addr learned my addr 172.21.15.5:0/363579618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:58.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f83877fe700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 0x7f8388074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.893+0000 7f8387fff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388102a30 0x7f8388073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 -- 172.21.15.5:0/363579618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388102a30 msgr2=0x7f8388073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388102a30 0x7f8388073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 -- 172.21.15.5:0/363579618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 msgr2=0x7f8388074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 0x7f8388074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 -- 172.21.15.5:0/363579618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c0095f0 con 0x7f8388102070 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f83877fe700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 0x7f8388074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8386ffd700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f837c00ebf0 tx=0x7f837c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c009a10 con 0x7f8388102070 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8378009520 con 0x7f8388102070 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f837c00cc90 con 0x7f8388102070 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.894+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c018750 con 0x7f8388102070 2023-09-23T04:45:58.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.895+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83881a1310 con 0x7f8388102070 2023-09-23T04:45:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.895+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8388107ef0 con 0x7f8388102070 2023-09-23T04:45:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.898+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f837c0188b0 con 0x7f8388102070 2023-09-23T04:45:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.899+0000 7f8384ff9700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 0x7f83700a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.899+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f837c014070 con 0x7f8388102070 2023-09-23T04:45:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.899+0000 7f83877fe700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 0x7f83700a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.900+0000 7f83877fe700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 0x7f83700a3ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f8378019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:58.901+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f837c097130 con 0x7f8388102070 2023-09-23T04:45:58.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:58 smithi118 ceph-mon[134297]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-23T04:45:58.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:58 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2042431119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:58.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:58 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/630884821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:58 smithi069 ceph-mon[133931]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-23T04:45:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:58 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2042431119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:45:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:58 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/630884821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:45:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.117+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f838810c750 con 0x7f8388102070 2023-09-23T04:45:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.118+0000 7f8384ff9700 1 -- 172.21.15.5:0/363579618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7f837c09a750 con 0x7f8388102070 2023-09-23T04:45:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:45:59.120 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:45:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 msgr2=0x7f83700a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 0x7f83700a3ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f8378019040 comp rx=0 tx=0).stop 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 msgr2=0x7f8388073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f837c00ebf0 tx=0x7f837c00ef00 comp rx=0 tx=0).stop 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 shutdown_connections 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f83700a1620 0x7f83700a3ae0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8388102a30 0x7f8388073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8388100e90 0x7f8388074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 --2- 172.21.15.5:0/363579618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8388102070 0x7f8388073210 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 >> 172.21.15.5:0/363579618 conn(0x7f83880fc450 msgr2=0x7f83880fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 shutdown_connections 2023-09-23T04:45:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.121+0000 7f838ddff700 1 -- 172.21.15.5:0/363579618 wait complete. 2023-09-23T04:45:59.141 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:45:59.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.346+0000 7efcc6bff700 1 -- 172.21.15.5:0/3936114625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102a50 msgr2=0x7efcc0109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.346+0000 7efcc6bff700 1 --2- 172.21.15.5:0/3936114625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102a50 0x7efcc0109370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b0a0 tx=0x7efcb800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 -- 172.21.15.5:0/3936114625 shutdown_connections 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 --2- 172.21.15.5:0/3936114625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102a50 0x7efcc0109370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 --2- 172.21.15.5:0/3936114625 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0100eb0 0x7efcc01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 --2- 172.21.15.5:0/3936114625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0102090 0x7efcc0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 -- 172.21.15.5:0/3936114625 >> 172.21.15.5:0/3936114625 conn(0x7efcc00fc470 msgr2=0x7efcc00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 -- 172.21.15.5:0/3936114625 shutdown_connections 2023-09-23T04:45:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.347+0000 7efcc6bff700 1 -- 172.21.15.5:0/3936114625 wait complete. 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 Processor -- start 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 -- start start 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 0x7efcc0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 0x7efcc019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc019e850 con 0x7efcc0102090 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc019e9c0 con 0x7efcc0100eb0 2023-09-23T04:45:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.348+0000 7efcc6bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc019eb80 con 0x7efcc0102a50 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcbffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 0x7efcc0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc519c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 0x7efcc019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:36334/0 (socket says 172.21.15.5:36334) 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 -- 172.21.15.5:0/183883787 learned_addr learned my addr 172.21.15.5:0/183883787 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 -- 172.21.15.5:0/183883787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 msgr2=0x7efcc019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 0x7efcc019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 -- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 msgr2=0x7efcc0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 0x7efcc0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 -- 172.21.15.5:0/183883787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8009d00 con 0x7efcc0100eb0 2023-09-23T04:45:59.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc519c700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 0x7efcc019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcbffff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 0x7efcc0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.349+0000 7efcc499b700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e970 tx=0x7efcb400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.350+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4009790 con 0x7efcc0100eb0 2023-09-23T04:45:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.350+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc019ee60 con 0x7efcc0100eb0 2023-09-23T04:45:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.350+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcb400c7e0 con 0x7efcc0100eb0 2023-09-23T04:45:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.350+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4003c70 con 0x7efcc0100eb0 2023-09-23T04:45:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.350+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc019f290 con 0x7efcc0100eb0 2023-09-23T04:45:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.351+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc0107dc0 con 0x7efcc0100eb0 2023-09-23T04:45:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.353+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7efcb400cc50 con 0x7efcc0100eb0 2023-09-23T04:45:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.354+0000 7efcbdffb700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 0x7efcb00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.354+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7efcb4014070 con 0x7efcc0100eb0 2023-09-23T04:45:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.355+0000 7efcbffff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 0x7efcb00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.355+0000 7efcbffff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 0x7efcb00a3ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efcc0100bd0 tx=0x7efcac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.356+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efcb4096620 con 0x7efcc0100eb0 2023-09-23T04:45:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.569+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcc010c9c0 con 0x7efcc0100eb0 2023-09-23T04:45:59.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.570+0000 7efcbdffb700 1 -- 172.21.15.5:0/183883787 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcb4099c40 con 0x7efcc0100eb0 2023-09-23T04:45:59.604 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:45:59.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 msgr2=0x7efcb00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 0x7efcb00a3ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efcc0100bd0 tx=0x7efcac009260 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 msgr2=0x7efcc0196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e970 tx=0x7efcb400ec80 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 shutdown_connections 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efcb00a1620 0x7efcb00a3ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcc0102a50 0x7efcc019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcc0100eb0 0x7efcc0196d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 --2- 172.21.15.5:0/183883787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102090 0x7efcc0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 >> 172.21.15.5:0/183883787 conn(0x7efcc00fc470 msgr2=0x7efcc00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 shutdown_connections 2023-09-23T04:45:59.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.572+0000 7efcc6bff700 1 -- 172.21.15.5:0/183883787 wait complete. 2023-09-23T04:45:59.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:45:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 -- 172.21.15.5:0/310042624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 msgr2=0x7f4340102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 --2- 172.21.15.5:0/310042624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340102510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4334009990 tx=0x7f4334009ca0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 -- 172.21.15.5:0/310042624 shutdown_connections 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 --2- 172.21.15.5:0/310042624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 --2- 172.21.15.5:0/310042624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f43401012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 --2- 172.21.15.5:0/310042624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340102510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 -- 172.21.15.5:0/310042624 >> 172.21.15.5:0/310042624 conn(0x7f43400fc490 msgr2=0x7f43400fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 -- 172.21.15.5:0/310042624 shutdown_connections 2023-09-23T04:45:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.818+0000 7f43457ff700 1 -- 172.21.15.5:0/310042624 wait complete. 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.819+0000 7f43457ff700 1 Processor -- start 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.819+0000 7f43457ff700 1 -- start start 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f4340074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4340073ef0 con 0x7f4340102a50 2023-09-23T04:45:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4340074060 con 0x7f4340100eb0 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f43457ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401a0c50 con 0x7f4340102090 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55450/0 (socket says 172.21.15.5:55450) 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f433effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f4340074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f433f7fe700 1 -- 172.21.15.5:0/3199868754 learned_addr learned my addr 172.21.15.5:0/3199868754 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.820+0000 7f433e7fc700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 -- 172.21.15.5:0/3199868754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 msgr2=0x7f4340073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 -- 172.21.15.5:0/3199868754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 msgr2=0x7f4340074bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f4340074bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:45:59.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 -- 172.21.15.5:0/3199868754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340095f0 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433effd700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f4340074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f433f7fe700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f433800ef30 tx=0x7f433800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338018070 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43401a0ed0 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f433800f040 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800cd00 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.821+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43401a1300 con 0x7f4340102a50 2023-09-23T04:45:59.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.822+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434010c720 con 0x7f4340102a50 2023-09-23T04:45:59.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.824+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4338003a00 con 0x7f4340102a50 2023-09-23T04:45:59.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.826+0000 7f4327fff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 0x7f43280a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:45:59.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.826+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f43380cce90 con 0x7f4340102a50 2023-09-23T04:45:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.826+0000 7f433effd700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 0x7f43280a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:45:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.826+0000 7f433effd700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 0x7f43280a3a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4340101c00 tx=0x7f4330009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:45:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:45:59.830+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f43380960c0 con 0x7f4340102a50 2023-09-23T04:45:59.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:59 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2426754260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:59.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:45:59 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/363579618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:59.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:59 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2426754260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:59.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:45:59 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/363579618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:45:59.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:59 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2426754260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:45:59.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:45:59 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/363579618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.047+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4340107eb0 con 0x7f4340102a50 2023-09-23T04:46:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.048+0000 7f4327fff700 1 -- 172.21.15.5:0/3199868754 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43380996e0 con 0x7f4340102a50 2023-09-23T04:46:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:46:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 msgr2=0x7f43280a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 0x7f43280a3a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4340101c00 tx=0x7f4330009260 comp rx=0 tx=0).stop 2023-09-23T04:46:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 msgr2=0x7f4340073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f433800ef30 tx=0x7f433800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 shutdown_connections 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f43280a15a0 0x7f43280a3a60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4340102a50 0x7f4340073750 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4340100eb0 0x7f4340074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 --2- 172.21.15.5:0/3199868754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4340102090 0x7f4340073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.050+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 >> 172.21.15.5:0/3199868754 conn(0x7f43400fc490 msgr2=0x7f43400fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.051+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 shutdown_connections 2023-09-23T04:46:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.051+0000 7f43457ff700 1 -- 172.21.15.5:0/3199868754 wait complete. 2023-09-23T04:46:00.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:46:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 -- 172.21.15.5:0/994501335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 msgr2=0x7fbf1c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 --2- 172.21.15.5:0/994501335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c1012d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009990 tx=0x7fbf0c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 -- 172.21.15.5:0/994501335 shutdown_connections 2023-09-23T04:46:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 --2- 172.21.15.5:0/994501335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102a50 0x7fbf1c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 --2- 172.21.15.5:0/994501335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c1012d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 --2- 172.21.15.5:0/994501335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102090 0x7fbf1c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 -- 172.21.15.5:0/994501335 >> 172.21.15.5:0/994501335 conn(0x7fbf1c0fc470 msgr2=0x7fbf1c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 -- 172.21.15.5:0/994501335 shutdown_connections 2023-09-23T04:46:00.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.278+0000 7fbf20fff700 1 -- 172.21.15.5:0/994501335 wait complete. 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.279+0000 7fbf20fff700 1 Processor -- start 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.279+0000 7fbf20fff700 1 -- start start 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102090 0x7fbf1c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102a50 0x7fbf1c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c073ef0 con 0x7fbf1c100eb0 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c074060 con 0x7fbf1c102a50 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf20fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1a0c90 con 0x7fbf1c102090 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55474/0 (socket says 172.21.15.5:55474) 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 -- 172.21.15.5:0/2935241686 learned_addr learned my addr 172.21.15.5:0/2935241686 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1ad9d700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102a50 0x7fbf1c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf19d9b700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102090 0x7fbf1c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 -- 172.21.15.5:0/2935241686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102090 msgr2=0x7fbf1c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102090 0x7fbf1c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 -- 172.21.15.5:0/2935241686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102a50 msgr2=0x7fbf1c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102a50 0x7fbf1c073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.280+0000 7fbf1a59c700 1 -- 172.21.15.5:0/2935241686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009520 con 0x7fbf1c100eb0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf1a59c700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005720 tx=0x7fbf0c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c01d070 con 0x7fbf1c100eb0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c0095f0 con 0x7fbf1c100eb0 2023-09-23T04:46:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf0c021410 con 0x7fbf1c100eb0 2023-09-23T04:46:00.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c017650 con 0x7fbf1c100eb0 2023-09-23T04:46:00.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.281+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1a11b0 con 0x7fbf1c100eb0 2023-09-23T04:46:00.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.282+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c10c770 con 0x7fbf1c100eb0 2023-09-23T04:46:00.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.284+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fbf0c021580 con 0x7fbf1c100eb0 2023-09-23T04:46:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.285+0000 7fbf0b7fe700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 0x7fbf040a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.285+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fbf0c0ce5b0 con 0x7fbf1c100eb0 2023-09-23T04:46:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.285+0000 7fbf19d9b700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 0x7fbf040a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.286+0000 7fbf19d9b700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 0x7fbf040a3a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c100bd0 tx=0x7fbf10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.289+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbf0c097900 con 0x7fbf1c100eb0 2023-09-23T04:46:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.501+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf1c107e80 con 0x7fbf1c100eb0 2023-09-23T04:46:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.502+0000 7fbf0b7fe700 1 -- 172.21.15.5:0/2935241686 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf0c026020 con 0x7fbf1c100eb0 2023-09-23T04:46:00.503 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:46:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 msgr2=0x7fbf040a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 0x7fbf040a3a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c100bd0 tx=0x7fbf10009260 comp rx=0 tx=0).stop 2023-09-23T04:46:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 msgr2=0x7fbf1c074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005720 tx=0x7fbf0c009580 comp rx=0 tx=0).stop 2023-09-23T04:46:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 shutdown_connections 2023-09-23T04:46:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbf040a15a0 0x7fbf040a3a60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf1c102a50 0x7fbf1c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf1c100eb0 0x7fbf1c074bd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 --2- 172.21.15.5:0/2935241686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbf1c102090 0x7fbf1c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 >> 172.21.15.5:0/2935241686 conn(0x7fbf1c0fc470 msgr2=0x7fbf1c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.504+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 shutdown_connections 2023-09-23T04:46:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.505+0000 7fbf20fff700 1 -- 172.21.15.5:0/2935241686 wait complete. 2023-09-23T04:46:00.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:46:00.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 -- 172.21.15.5:0/3329604167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 msgr2=0x7f6144102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 --2- 172.21.15.5:0/3329604167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144102510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f612c009990 tx=0x7f612c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 -- 172.21.15.5:0/3329604167 shutdown_connections 2023-09-23T04:46:00.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 --2- 172.21.15.5:0/3329604167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144102a50 0x7f6144109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 --2- 172.21.15.5:0/3329604167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144100eb0 0x7f61441012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 --2- 172.21.15.5:0/3329604167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144102510 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 -- 172.21.15.5:0/3329604167 >> 172.21.15.5:0/3329604167 conn(0x7f61440fc470 msgr2=0x7f61440fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:00.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 -- 172.21.15.5:0/3329604167 shutdown_connections 2023-09-23T04:46:00.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.732+0000 7f6148fff700 1 -- 172.21.15.5:0/3329604167 wait complete. 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.733+0000 7f6148fff700 1 Processor -- start 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.733+0000 7f6148fff700 1 -- start start 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144102a50 0x7f61441a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61441a09b0 con 0x7f6144102090 2023-09-23T04:46:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61441a0af0 con 0x7f6144100eb0 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6148fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61441a0c80 con 0x7f6144102a50 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f614259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f614259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:36366/0 (socket says 172.21.15.5:36366) 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f614259c700 1 -- 172.21.15.5:0/3885840977 learned_addr learned my addr 172.21.15.5:0/3885840977 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f6142d9d700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144102a50 0x7f61441a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 -- 172.21.15.5:0/3885840977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144102a50 msgr2=0x7f61441a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144102a50 0x7f61441a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 -- 172.21.15.5:0/3885840977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 msgr2=0x7f6144198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f613bfff700 1 -- 172.21.15.5:0/3885840977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0095f0 con 0x7f6144102090 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.734+0000 7f614259c700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f613bfff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144199370 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f612c0098e0 tx=0x7f612c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c01d070 con 0x7f6144102090 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f612c021410 con 0x7f6144102090 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61441a0f00 con 0x7f6144102090 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c005720 con 0x7f6144102090 2023-09-23T04:46:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.735+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61441a1270 con 0x7f6144102090 2023-09-23T04:46:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.736+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614410c770 con 0x7f6144102090 2023-09-23T04:46:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.738+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f612c021580 con 0x7f6144102090 2023-09-23T04:46:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.740+0000 7f613b7fe700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 0x7f61300a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.740+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f612c0ce410 con 0x7f6144102090 2023-09-23T04:46:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.740+0000 7f614259c700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 0x7f61300a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.740+0000 7f614259c700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 0x7f61300a3a60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6134005950 tx=0x7f6134009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.744+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f612c097760 con 0x7f6144102090 2023-09-23T04:46:00.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/183883787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:00.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:00 smithi005 ceph-mon[97638]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-23T04:46:00.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:00 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3199868754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:00.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/183883787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:00 smithi118 ceph-mon[134297]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-23T04:46:00.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:00 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3199868754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.954+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f61441a1520 con 0x7f6144102090 2023-09-23T04:46:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.955+0000 7f613b7fe700 1 -- 172.21.15.5:0/3885840977 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f612c0260a0 con 0x7f6144102090 2023-09-23T04:46:00.956 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 msgr2=0x7f61300a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 0x7f61300a3a60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6134005950 tx=0x7f6134009260 comp rx=0 tx=0).stop 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 msgr2=0x7f6144199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144199370 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f612c0098e0 tx=0x7f612c009580 comp rx=0 tx=0).stop 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 shutdown_connections 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f61300a15a0 0x7f61300a3a60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6144102a50 0x7f61441a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6144100eb0 0x7f6144198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 --2- 172.21.15.5:0/3885840977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6144102090 0x7f6144199370 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 >> 172.21.15.5:0/3885840977 conn(0x7f61440fc470 msgr2=0x7f61440fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 shutdown_connections 2023-09-23T04:46:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:00.957+0000 7f6148fff700 1 -- 172.21.15.5:0/3885840977 wait complete. 2023-09-23T04:46:00.977 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-23T04:46:00.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:46:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/183883787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:00 smithi069 ceph-mon[133931]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-23T04:46:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:00 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3199868754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 -- 172.21.15.5:0/3445627215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130102a40 msgr2=0x7f4130109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 --2- 172.21.15.5:0/3445627215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130102a40 0x7f4130109360 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f412800b0a0 tx=0x7f412800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 -- 172.21.15.5:0/3445627215 shutdown_connections 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 --2- 172.21.15.5:0/3445627215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130102a40 0x7f4130109360 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 --2- 172.21.15.5:0/3445627215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130100ea0 0x7f41301012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 --2- 172.21.15.5:0/3445627215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 0x7f4130102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 -- 172.21.15.5:0/3445627215 >> 172.21.15.5:0/3445627215 conn(0x7f41300fc480 msgr2=0x7f41300fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 -- 172.21.15.5:0/3445627215 shutdown_connections 2023-09-23T04:46:01.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.181+0000 7f41357ff700 1 -- 172.21.15.5:0/3445627215 wait complete. 2023-09-23T04:46:01.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.182+0000 7f41357ff700 1 Processor -- start 2023-09-23T04:46:01.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 -- start start 2023-09-23T04:46:01.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 0x7f4130199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130102a40 0x7f41301a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41301a0aa0 con 0x7f4130100ea0 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41301a0be0 con 0x7f4130102a40 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f41357ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41301a0d70 con 0x7f4130102080 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f412effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f412effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40980/0 (socket says 172.21.15.5:40980) 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f412effd700 1 -- 172.21.15.5:0/4057530180 learned_addr learned my addr 172.21.15.5:0/4057530180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f412f7fe700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130102a40 0x7f41301a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.183+0000 7f412e7fc700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 0x7f4130199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 -- 172.21.15.5:0/4057530180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 msgr2=0x7f4130199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 0x7f4130199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 -- 172.21.15.5:0/4057530180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130102a40 msgr2=0x7f41301a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130102a40 0x7f41301a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 -- 172.21.15.5:0/4057530180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120009520 con 0x7f4130100ea0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f412effd700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f412000ea50 tx=0x7f412000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000cbd0 con 0x7f4130100ea0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128009d00 con 0x7f4130100ea0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412000cd30 con 0x7f4130100ea0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120018680 con 0x7f4130100ea0 2023-09-23T04:46:01.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.184+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41301a12e0 con 0x7f4130100ea0 2023-09-23T04:46:01.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.185+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413010c760 con 0x7f4130100ea0 2023-09-23T04:46:01.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.187+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4120010430 con 0x7f4130100ea0 2023-09-23T04:46:01.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.189+0000 7f4117fff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 0x7f41180a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.189+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f4120014070 con 0x7f4130100ea0 2023-09-23T04:46:01.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.189+0000 7f412e7fc700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 0x7f41180a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.189+0000 7f412e7fc700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 0x7f41180a3a60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4130100bc0 tx=0x7f4124009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:01.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.192+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f41200969a0 con 0x7f4130100ea0 2023-09-23T04:46:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.451+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41301a16d0 con 0x7f4130100ea0 2023-09-23T04:46:01.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.452+0000 7f4117fff700 1 -- 172.21.15.5:0/4057530180 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f4120099fc0 con 0x7f4130100ea0 2023-09-23T04:46:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:01.453 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:46:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 msgr2=0x7f41180a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 0x7f41180a3a60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4130100bc0 tx=0x7f4124009260 comp rx=0 tx=0).stop 2023-09-23T04:46:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 msgr2=0x7f4130198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f412000ea50 tx=0x7f412000ed60 comp rx=0 tx=0).stop 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 shutdown_connections 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f41180a15a0 0x7f41180a3a60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4130102a40 0x7f41301a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4130100ea0 0x7f4130198f20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 --2- 172.21.15.5:0/4057530180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4130102080 0x7f4130199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 >> 172.21.15.5:0/4057530180 conn(0x7f41300fc480 msgr2=0x7f41300fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.454+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 shutdown_connections 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.455+0000 7f41357ff700 1 -- 172.21.15.5:0/4057530180 wait complete. 2023-09-23T04:46:01.456 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:46:01.474 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-23T04:46:01.475 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-23T04:46:01.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 -- 172.21.15.5:0/808232651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 msgr2=0x7f7970109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 --2- 172.21.15.5:0/808232651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970109360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f796800b0a0 tx=0x7f796800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 -- 172.21.15.5:0/808232651 shutdown_connections 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 --2- 172.21.15.5:0/808232651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970109360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 --2- 172.21.15.5:0/808232651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970100ea0 0x7f79701012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 --2- 172.21.15.5:0/808232651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970102080 0x7f7970102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 -- 172.21.15.5:0/808232651 >> 172.21.15.5:0/808232651 conn(0x7f79700fc480 msgr2=0x7f79700fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 -- 172.21.15.5:0/808232651 shutdown_connections 2023-09-23T04:46:01.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.679+0000 7f7975fff700 1 -- 172.21.15.5:0/808232651 wait complete. 2023-09-23T04:46:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 Processor -- start 2023-09-23T04:46:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 -- start start 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970102080 0x7f79701071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970103fd0 con 0x7f7970102a40 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970104140 con 0x7f7970100ea0 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.680+0000 7f7975fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970104300 con 0x7f7970102080 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f7967fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970102080 0x7f79701071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59752/0 (socket says 172.21.15.5:59752) 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796f7fe700 1 -- 172.21.15.5:0/812100125 learned_addr learned my addr 172.21.15.5:0/812100125 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:01.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 -- 172.21.15.5:0/812100125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970102080 msgr2=0x7f79701071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970102080 0x7f79701071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 -- 172.21.15.5:0/812100125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 msgr2=0x7f7970106ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 -- 172.21.15.5:0/812100125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968009d00 con 0x7f7970102a40 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796f7fe700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796ffff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970103830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7968009fd0 tx=0x7f7968007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.681+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800e040 con 0x7f7970102a40 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.682+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79701a53b0 con 0x7f7970102a40 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.682+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7968004c50 con 0x7f7970102a40 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.682+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796801b6c0 con 0x7f7970102a40 2023-09-23T04:46:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.682+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701a5830 con 0x7f7970102a40 2023-09-23T04:46:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.683+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797010c760 con 0x7f7970102a40 2023-09-23T04:46:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.685+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f796801d470 con 0x7f7970102a40 2023-09-23T04:46:01.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.687+0000 7f796d7fa700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 0x7f795c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:01.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.687+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f79680ce240 con 0x7f7970102a40 2023-09-23T04:46:01.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.687+0000 7f796f7fe700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 0x7f795c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:01.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.688+0000 7f796f7fe700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 0x7f795c0a3ae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7960005950 tx=0x7f7960009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:01.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.691+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f79680974d0 con 0x7f7970102a40 2023-09-23T04:46:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:01 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2935241686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:01 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3885840977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:01 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4057530180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:01.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.905+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79701a5ae0 con 0x7f795c0a1620 2023-09-23T04:46:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:01 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2935241686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:01 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3885840977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:01 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4057530180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.919+0000 7f796d7fa700 1 -- 172.21.15.5:0/812100125 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427809 (secure 0 0 0) 0x7f79701a5ae0 con 0x7f795c0a1620 2023-09-23T04:46:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.924+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 msgr2=0x7f795c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.924+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 0x7f795c0a3ae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7960005950 tx=0x7f7960009260 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.924+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 msgr2=0x7f7970103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.924+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970103830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7968009fd0 tx=0x7f7968007930 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 shutdown_connections 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f795c0a1620 0x7f795c0a3ae0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7970102a40 0x7f7970103830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970100ea0 0x7f7970106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 --2- 172.21.15.5:0/812100125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7970102080 0x7f79701071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 >> 172.21.15.5:0/812100125 conn(0x7f79700fc480 msgr2=0x7f79700fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 shutdown_connections 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:01.925+0000 7f7975fff700 1 -- 172.21.15.5:0/812100125 wait complete. 2023-09-23T04:46:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:46:01.944 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2023-09-23T04:46:01.665346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659420,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1163,"num_read_kb":707,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181458,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336544,"kb_used_data":12284,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605280,"statfs":{"total":1151940427776,"available":1151595806720,"internally_reserved":0,"allocated":12578816,"data_stored":8046901,"data_compressed":392895,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796321+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:45:25.796321+0000","last_peered":"2023-09-23T04:45:25.796321+0000","last_clean":"2023-09-23T04:45:25.796321+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:45:25.796321+0000","last_undegraded":"2023-09-23T04:45:25.796321+0000","last_fullsized":"2023-09-23T04:45:25.796321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791073+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:45:25.791073+0000","last_peered":"2023-09-23T04:45:25.791073+0000","last_clean":"2023-09-23T04:45:25.791073+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:45:25.791073+0000","last_undegraded":"2023-09-23T04:45:25.791073+0000","last_fullsized":"2023-09-23T04:45:25.791073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790918+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:45:25.790918+0000","last_peered":"2023-09-23T04:45:25.790918+0000","last_clean":"2023-09-23T04:45:25.790918+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:45:25.790918+0000","last_undegraded":"2023-09-23T04:45:25.790918+0000","last_fullsized":"2023-09-23T04:45:25.790918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791433+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:45:25.791433+0000","last_peered":"2023-09-23T04:45:25.791433+0000","last_clean":"2023-09-23T04:45:25.791433+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:45:25.791433+0000","last_undegraded":"2023-09-23T04:45:25.791433+0000","last_fullsized":"2023-09-23T04:45:25.791433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796437+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:45:24.796437+0000","last_peered":"2023-09-23T04:45:24.796437+0000","last_clean":"2023-09-23T04:45:24.796437+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:45:24.796437+0000","last_undegraded":"2023-09-23T04:45:24.796437+0000","last_fullsized":"2023-09-23T04:45:24.796437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788454+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:45:24.788454+0000","last_peered":"2023-09-23T04:45:24.788454+0000","last_clean":"2023-09-23T04:45:24.788454+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:45:24.788454+0000","last_undegraded":"2023-09-23T04:45:24.788454+0000","last_fullsized":"2023-09-23T04:45:24.788454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791116+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:45:24.791116+0000","last_peered":"2023-09-23T04:45:24.791116+0000","last_clean":"2023-09-23T04:45:24.791116+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:45:24.791116+0000","last_undegraded":"2023-09-23T04:45:24.791116+0000","last_fullsized":"2023-09-23T04:45:24.791116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796359+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:45:24.796359+0000","last_peered":"2023-09-23T04:45:24.796359+0000","last_clean":"2023-09-23T04:45:24.796359+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:45:24.796359+0000","last_undegraded":"2023-09-23T04:45:24.796359+0000","last_fullsized":"2023-09-23T04:45:24.796359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794018+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:45:24.794018+0000","last_peered":"2023-09-23T04:45:24.794018+0000","last_clean":"2023-09-23T04:45:24.794018+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:45:24.794018+0000","last_undegraded":"2023-09-23T04:45:24.794018+0000","last_fullsized":"2023-09-23T04:45:24.794018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.789806+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:45:24.789806+0000","last_peered":"2023-09-23T04:45:24.789806+0000","last_clean":"2023-09-23T04:45:24.789806+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:45:24.789806+0000","last_undegraded":"2023-09-23T04:45:24.789806+0000","last_fullsized":"2023-09-23T04:45:24.789806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791821+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:45:24.791821+0000","last_peered":"2023-09-23T04:45:24.791821+0000","last_clean":"2023-09-23T04:45:24.791821+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:45:24.791821+0000","last_undegraded":"2023-09-23T04:45:24.791821+0000","last_fullsized":"2023-09-23T04:45:24.791821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790570+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:45:24.790570+0000","last_peered":"2023-09-23T04:45:24.790570+0000","last_clean":"2023-09-23T04:45:24.790570+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:45:24.790570+0000","last_undegraded":"2023-09-23T04:45:24.790570+0000","last_fullsized":"2023-09-23T04:45:24.790570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788085+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:45:24.788085+0000","last_peered":"2023-09-23T04:45:24.788085+0000","last_clean":"2023-09-23T04:45:24.788085+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:45:24.788085+0000","last_undegraded":"2023-09-23T04:45:24.788085+0000","last_fullsized":"2023-09-23T04:45:24.788085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793195+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:45:24.793195+0000","last_peered":"2023-09-23T04:45:24.793195+0000","last_clean":"2023-09-23T04:45:24.793195+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:45:24.793195+0000","last_undegraded":"2023-09-23T04:45:24.793195+0000","last_fullsized":"2023-09-23T04:45:24.793195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793816+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:45:24.793816+0000","last_peered":"2023-09-23T04:45:24.793816+0000","last_clean":"2023-09-23T04:45:24.793816+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:45:24.793816+0000","last_undegraded":"2023-09-23T04:45:24.793816+0000","last_fullsized":"2023-09-23T04:45:24.793816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792675+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:45:24.792675+0000","last_peered":"2023-09-23T04:45:24.792675+0000","last_clean":"2023-09-23T04:45:24.792675+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:45:24.792675+0000","last_undegraded":"2023-09-23T04:45:24.792675+0000","last_fullsized":"2023-09-23T04:45:24.792675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793222+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:45:24.793222+0000","last_peered":"2023-09-23T04:45:24.793222+0000","last_clean":"2023-09-23T04:45:24.793222+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:45:24.793222+0000","last_undegraded":"2023-09-23T04:45:24.793222+0000","last_fullsized":"2023-09-23T04:45:24.793222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793658+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:45:24.793658+0000","last_peered":"2023-09-23T04:45:24.793658+0000","last_clean":"2023-09-23T04:45:24.793658+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:45:24.793658+0000","last_undegraded":"2023-09-23T04:45:24.793658+0000","last_fullsized":"2023-09-23T04:45:24.793658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.962541+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:45:56.962541+0000","last_peered":"2023-09-23T04:45:56.962541+0000","last_clean":"2023-09-23T04:45:56.962541+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:45:56.962541+0000","last_undegraded":"2023-09-23T04:45:56.962541+0000","last_fullsized":"2023-09-23T04:45:56.962541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947303+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:45:25.947303+0000","last_peered":"2023-09-23T04:45:25.947303+0000","last_clean":"2023-09-23T04:45:25.947303+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:45:25.947303+0000","last_undegraded":"2023-09-23T04:45:25.947303+0000","last_fullsized":"2023-09-23T04:45:25.947303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787715+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:45:24.787715+0000","last_peered":"2023-09-23T04:45:24.787715+0000","last_clean":"2023-09-23T04:45:24.787715+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:45:24.787715+0000","last_undegraded":"2023-09-23T04:45:24.787715+0000","last_fullsized":"2023-09-23T04:45:24.787715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947680+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:45:25.947680+0000","last_peered":"2023-09-23T04:45:25.947680+0000","last_clean":"2023-09-23T04:45:25.947680+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:45:25.947680+0000","last_undegraded":"2023-09-23T04:45:25.947680+0000","last_fullsized":"2023-09-23T04:45:25.947680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791138+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:45:25.791138+0000","last_peered":"2023-09-23T04:45:25.791138+0000","last_clean":"2023-09-23T04:45:25.791138+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:45:25.791138+0000","last_undegraded":"2023-09-23T04:45:25.791138+0000","last_fullsized":"2023-09-23T04:45:25.791138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788666+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:45:24.788666+0000","last_peered":"2023-09-23T04:45:24.788666+0000","last_clean":"2023-09-23T04:45:24.788666+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:45:24.788666+0000","last_undegraded":"2023-09-23T04:45:24.788666+0000","last_fullsized":"2023-09-23T04:45:24.788666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796632+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:45:24.796632+0000","last_peered":"2023-09-23T04:45:24.796632+0000","last_clean":"2023-09-23T04:45:24.796632+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:45:24.796632+0000","last_undegraded":"2023-09-23T04:45:24.796632+0000","last_fullsized":"2023-09-23T04:45:24.796632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793988+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:45:24.793988+0000","last_peered":"2023-09-23T04:45:24.793988+0000","last_clean":"2023-09-23T04:45:24.793988+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:45:24.793988+0000","last_undegraded":"2023-09-23T04:45:24.793988+0000","last_fullsized":"2023-09-23T04:45:24.793988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788201+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:45:24.788201+0000","last_peered":"2023-09-23T04:45:24.788201+0000","last_clean":"2023-09-23T04:45:24.788201+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:45:24.788201+0000","last_undegraded":"2023-09-23T04:45:24.788201+0000","last_fullsized":"2023-09-23T04:45:24.788201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796586+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:45:24.796586+0000","last_peered":"2023-09-23T04:45:24.796586+0000","last_clean":"2023-09-23T04:45:24.796586+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:45:24.796586+0000","last_undegraded":"2023-09-23T04:45:24.796586+0000","last_fullsized":"2023-09-23T04:45:24.796586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796924+0000","last_change":"2023-09-23T04:45:25.796924+0000","last_active":"2023-09-23T04:45:25.796924+0000","last_peered":"2023-09-23T04:45:25.796924+0000","last_clean":"2023-09-23T04:45:25.796924+0000","last_became_active":"2023-09-23T04:45:25.795995+0000","last_became_peered":"2023-09-23T04:45:25.795995+0000","last_unstale":"2023-09-23T04:45:25.796924+0000","last_undegraded":"2023-09-23T04:45:25.796924+0000","last_fullsized":"2023-09-23T04:45:25.796924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:43.679364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,2],"acting":[5,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793038+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:45:24.793038+0000","last_peered":"2023-09-23T04:45:24.793038+0000","last_clean":"2023-09-23T04:45:24.793038+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:45:24.793038+0000","last_undegraded":"2023-09-23T04:45:24.793038+0000","last_fullsized":"2023-09-23T04:45:24.793038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.602592+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:45:58.602592+0000","last_peered":"2023-09-23T04:45:58.602592+0000","last_clean":"2023-09-23T04:45:58.602592+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:45:58.602592+0000","last_undegraded":"2023-09-23T04:45:58.602592+0000","last_fullsized":"2023-09-23T04:45:58.602592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796301+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:45:24.796301+0000","last_peered":"2023-09-23T04:45:24.796301+0000","last_clean":"2023-09-23T04:45:24.796301+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:45:24.796301+0000","last_undegraded":"2023-09-23T04:45:24.796301+0000","last_fullsized":"2023-09-23T04:45:24.796301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.799726+0000","last_change":"2023-09-23T04:45:25.799726+0000","last_active":"2023-09-23T04:45:25.799726+0000","last_peered":"2023-09-23T04:45:25.799726+0000","last_clean":"2023-09-23T04:45:25.799726+0000","last_became_active":"2023-09-23T04:45:25.799273+0000","last_became_peered":"2023-09-23T04:45:25.799273+0000","last_unstale":"2023-09-23T04:45:25.799726+0000","last_undegraded":"2023-09-23T04:45:25.799726+0000","last_fullsized":"2023-09-23T04:45:25.799726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:19.330354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788266+0000","last_change":"2023-09-23T04:44:24.819153+0000","last_active":"2023-09-23T04:45:24.788266+0000","last_peered":"2023-09-23T04:45:24.788266+0000","last_clean":"2023-09-23T04:45:24.788266+0000","last_became_active":"2023-09-23T04:44:24.809881+0000","last_became_peered":"2023-09-23T04:44:24.809881+0000","last_unstale":"2023-09-23T04:45:24.788266+0000","last_undegraded":"2023-09-23T04:45:24.788266+0000","last_fullsized":"2023-09-23T04:45:24.788266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:38.014881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.798084+0000","last_change":"2023-09-23T04:45:25.798084+0000","last_active":"2023-09-23T04:45:25.798084+0000","last_peered":"2023-09-23T04:45:25.798084+0000","last_clean":"2023-09-23T04:45:25.798084+0000","last_became_active":"2023-09-23T04:45:25.797177+0000","last_became_peered":"2023-09-23T04:45:25.797177+0000","last_unstale":"2023-09-23T04:45:25.798084+0000","last_undegraded":"2023-09-23T04:45:25.798084+0000","last_fullsized":"2023-09-23T04:45:25.798084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:15.059834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,7],"acting":[5,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796126+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:45:25.796126+0000","last_peered":"2023-09-23T04:45:25.796126+0000","last_clean":"2023-09-23T04:45:25.796126+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:45:25.796126+0000","last_undegraded":"2023-09-23T04:45:25.796126+0000","last_fullsized":"2023-09-23T04:45:25.796126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787803+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:45:24.787803+0000","last_peered":"2023-09-23T04:45:24.787803+0000","last_clean":"2023-09-23T04:45:24.787803+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:45:24.787803+0000","last_undegraded":"2023-09-23T04:45:24.787803+0000","last_fullsized":"2023-09-23T04:45:24.787803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800083+0000","last_change":"2023-09-23T04:45:25.798999+0000","last_active":"2023-09-23T04:45:25.800083+0000","last_peered":"2023-09-23T04:45:25.800083+0000","last_clean":"2023-09-23T04:45:25.800083+0000","last_became_active":"2023-09-23T04:45:25.798608+0000","last_became_peered":"2023-09-23T04:45:25.798608+0000","last_unstale":"2023-09-23T04:45:25.800083+0000","last_undegraded":"2023-09-23T04:45:25.800083+0000","last_fullsized":"2023-09-23T04:45:25.800083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:31:20.866274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791022+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:45:25.791022+0000","last_peered":"2023-09-23T04:45:25.791022+0000","last_clean":"2023-09-23T04:45:25.791022+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:45:25.791022+0000","last_undegraded":"2023-09-23T04:45:25.791022+0000","last_fullsized":"2023-09-23T04:45:25.791022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792901+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:45:24.792901+0000","last_peered":"2023-09-23T04:45:24.792901+0000","last_clean":"2023-09-23T04:45:24.792901+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:45:24.792901+0000","last_undegraded":"2023-09-23T04:45:24.792901+0000","last_fullsized":"2023-09-23T04:45:24.792901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791370+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:45:24.791370+0000","last_peered":"2023-09-23T04:45:24.791370+0000","last_clean":"2023-09-23T04:45:24.791370+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:45:24.791370+0000","last_undegraded":"2023-09-23T04:45:24.791370+0000","last_fullsized":"2023-09-23T04:45:24.791370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796834+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:45:24.796834+0000","last_peered":"2023-09-23T04:45:24.796834+0000","last_clean":"2023-09-23T04:45:24.796834+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:45:24.796834+0000","last_undegraded":"2023-09-23T04:45:24.796834+0000","last_fullsized":"2023-09-23T04:45:24.796834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":223,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601638+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:45:58.601638+0000","last_peered":"2023-09-23T04:45:58.601638+0000","last_clean":"2023-09-23T04:45:58.601638+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:45:58.601638+0000","last_undegraded":"2023-09-23T04:45:58.601638+0000","last_fullsized":"2023-09-23T04:45:58.601638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793929+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:45:24.793929+0000","last_peered":"2023-09-23T04:45:24.793929+0000","last_clean":"2023-09-23T04:45:24.793929+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:45:24.793929+0000","last_undegraded":"2023-09-23T04:45:24.793929+0000","last_fullsized":"2023-09-23T04:45:24.793929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792058+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:45:24.792058+0000","last_peered":"2023-09-23T04:45:24.792058+0000","last_clean":"2023-09-23T04:45:24.792058+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:45:24.792058+0000","last_undegraded":"2023-09-23T04:45:24.792058+0000","last_fullsized":"2023-09-23T04:45:24.792058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791812+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:45:24.791812+0000","last_peered":"2023-09-23T04:45:24.791812+0000","last_clean":"2023-09-23T04:45:24.791812+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:45:24.791812+0000","last_undegraded":"2023-09-23T04:45:24.791812+0000","last_fullsized":"2023-09-23T04:45:24.791812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796210+0000","last_change":"2023-09-23T04:45:25.796210+0000","last_active":"2023-09-23T04:45:25.796210+0000","last_peered":"2023-09-23T04:45:25.796210+0000","last_clean":"2023-09-23T04:45:25.796210+0000","last_became_active":"2023-09-23T04:45:25.795713+0000","last_became_peered":"2023-09-23T04:45:25.795713+0000","last_unstale":"2023-09-23T04:45:25.796210+0000","last_undegraded":"2023-09-23T04:45:25.796210+0000","last_fullsized":"2023-09-23T04:45:25.796210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:02.588278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788864+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:45:24.788864+0000","last_peered":"2023-09-23T04:45:24.788864+0000","last_clean":"2023-09-23T04:45:24.788864+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:45:24.788864+0000","last_undegraded":"2023-09-23T04:45:24.788864+0000","last_fullsized":"2023-09-23T04:45:24.788864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792638+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:45:24.792638+0000","last_peered":"2023-09-23T04:45:24.792638+0000","last_clean":"2023-09-23T04:45:24.792638+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:45:24.792638+0000","last_undegraded":"2023-09-23T04:45:24.792638+0000","last_fullsized":"2023-09-23T04:45:24.792638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793884+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:45:24.793884+0000","last_peered":"2023-09-23T04:45:24.793884+0000","last_clean":"2023-09-23T04:45:24.793884+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:45:24.793884+0000","last_undegraded":"2023-09-23T04:45:24.793884+0000","last_fullsized":"2023-09-23T04:45:24.793884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791564+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:45:24.791564+0000","last_peered":"2023-09-23T04:45:24.791564+0000","last_clean":"2023-09-23T04:45:24.791564+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:45:24.791564+0000","last_undegraded":"2023-09-23T04:45:24.791564+0000","last_fullsized":"2023-09-23T04:45:24.791564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":267,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601841+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:45:58.601841+0000","last_peered":"2023-09-23T04:45:58.601841+0000","last_clean":"2023-09-23T04:45:58.601841+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:45:58.601841+0000","last_undegraded":"2023-09-23T04:45:58.601841+0000","last_fullsized":"2023-09-23T04:45:58.601841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61006,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":163,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796923+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:45:24.796923+0000","last_peered":"2023-09-23T04:45:24.796923+0000","last_clean":"2023-09-23T04:45:24.796923+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:45:24.796923+0000","last_undegraded":"2023-09-23T04:45:24.796923+0000","last_fullsized":"2023-09-23T04:45:24.796923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793813+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:45:24.793813+0000","last_peered":"2023-09-23T04:45:24.793813+0000","last_clean":"2023-09-23T04:45:24.793813+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:45:24.793813+0000","last_undegraded":"2023-09-23T04:45:24.793813+0000","last_fullsized":"2023-09-23T04:45:24.793813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792648+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:45:24.792648+0000","last_peered":"2023-09-23T04:45:24.792648+0000","last_clean":"2023-09-23T04:45:24.792648+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:45:24.792648+0000","last_undegraded":"2023-09-23T04:45:24.792648+0000","last_fullsized":"2023-09-23T04:45:24.792648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788136+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:45:24.788136+0000","last_peered":"2023-09-23T04:45:24.788136+0000","last_clean":"2023-09-23T04:45:24.788136+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:45:24.788136+0000","last_undegraded":"2023-09-23T04:45:24.788136+0000","last_fullsized":"2023-09-23T04:45:24.788136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800896+0000","last_change":"2023-09-23T04:45:25.800896+0000","last_active":"2023-09-23T04:45:25.800896+0000","last_peered":"2023-09-23T04:45:25.800896+0000","last_clean":"2023-09-23T04:45:25.800896+0000","last_became_active":"2023-09-23T04:45:25.800479+0000","last_became_peered":"2023-09-23T04:45:25.800479+0000","last_unstale":"2023-09-23T04:45:25.800896+0000","last_undegraded":"2023-09-23T04:45:25.800896+0000","last_fullsized":"2023-09-23T04:45:25.800896+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:31.746725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794928+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:45:24.794928+0000","last_peered":"2023-09-23T04:45:24.794928+0000","last_clean":"2023-09-23T04:45:24.794928+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:45:24.794928+0000","last_undegraded":"2023-09-23T04:45:24.794928+0000","last_fullsized":"2023-09-23T04:45:24.794928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791908+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:45:24.791908+0000","last_peered":"2023-09-23T04:45:24.791908+0000","last_clean":"2023-09-23T04:45:24.791908+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:45:24.791908+0000","last_undegraded":"2023-09-23T04:45:24.791908+0000","last_fullsized":"2023-09-23T04:45:24.791908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791302+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:45:24.791302+0000","last_peered":"2023-09-23T04:45:24.791302+0000","last_clean":"2023-09-23T04:45:24.791302+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:45:24.791302+0000","last_undegraded":"2023-09-23T04:45:24.791302+0000","last_fullsized":"2023-09-23T04:45:24.791302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.635371+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:45:59.635371+0000","last_peered":"2023-09-23T04:45:59.635371+0000","last_clean":"2023-09-23T04:45:59.635371+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:45:59.635371+0000","last_undegraded":"2023-09-23T04:45:59.635371+0000","last_fullsized":"2023-09-23T04:45:59.635371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.797255+0000","last_change":"2023-09-23T04:45:25.797255+0000","last_active":"2023-09-23T04:45:25.797255+0000","last_peered":"2023-09-23T04:45:25.797255+0000","last_clean":"2023-09-23T04:45:25.797255+0000","last_became_active":"2023-09-23T04:45:25.796320+0000","last_became_peered":"2023-09-23T04:45:25.796320+0000","last_unstale":"2023-09-23T04:45:25.797255+0000","last_undegraded":"2023-09-23T04:45:25.797255+0000","last_fullsized":"2023-09-23T04:45:25.797255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:22:13.933638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788018+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:45:24.788018+0000","last_peered":"2023-09-23T04:45:24.788018+0000","last_clean":"2023-09-23T04:45:24.788018+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:45:24.788018+0000","last_undegraded":"2023-09-23T04:45:24.788018+0000","last_fullsized":"2023-09-23T04:45:24.788018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796424+0000","last_change":"2023-09-23T04:44:24.830212+0000","last_active":"2023-09-23T04:45:24.796424+0000","last_peered":"2023-09-23T04:45:24.796424+0000","last_clean":"2023-09-23T04:45:24.796424+0000","last_became_active":"2023-09-23T04:44:24.809351+0000","last_became_peered":"2023-09-23T04:44:24.809351+0000","last_unstale":"2023-09-23T04:45:24.796424+0000","last_undegraded":"2023-09-23T04:45:24.796424+0000","last_fullsized":"2023-09-23T04:45:24.796424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:07:32.566651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794396+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:45:24.794396+0000","last_peered":"2023-09-23T04:45:24.794396+0000","last_clean":"2023-09-23T04:45:24.794396+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:45:24.794396+0000","last_undegraded":"2023-09-23T04:45:24.794396+0000","last_fullsized":"2023-09-23T04:45:24.794396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794287+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:45:24.794287+0000","last_peered":"2023-09-23T04:45:24.794287+0000","last_clean":"2023-09-23T04:45:24.794287+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:45:24.794287+0000","last_undegraded":"2023-09-23T04:45:24.794287+0000","last_fullsized":"2023-09-23T04:45:24.794287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796031+0000","last_change":"2023-09-23T04:44:24.833679+0000","last_active":"2023-09-23T04:45:25.796031+0000","last_peered":"2023-09-23T04:45:25.796031+0000","last_clean":"2023-09-23T04:45:25.796031+0000","last_became_active":"2023-09-23T04:44:24.814917+0000","last_became_peered":"2023-09-23T04:44:24.814917+0000","last_unstale":"2023-09-23T04:45:25.796031+0000","last_undegraded":"2023-09-23T04:45:25.796031+0000","last_fullsized":"2023-09-23T04:45:25.796031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:18:28.868416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794076+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:45:24.794076+0000","last_peered":"2023-09-23T04:45:24.794076+0000","last_clean":"2023-09-23T04:45:24.794076+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:45:24.794076+0000","last_undegraded":"2023-09-23T04:45:24.794076+0000","last_fullsized":"2023-09-23T04:45:24.794076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794710+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:45:25.794710+0000","last_peered":"2023-09-23T04:45:25.794710+0000","last_clean":"2023-09-23T04:45:25.794710+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:45:25.794710+0000","last_undegraded":"2023-09-23T04:45:25.794710+0000","last_fullsized":"2023-09-23T04:45:25.794710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789444+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:45:25.789444+0000","last_peered":"2023-09-23T04:45:25.789444+0000","last_clean":"2023-09-23T04:45:25.789444+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:45:25.789444+0000","last_undegraded":"2023-09-23T04:45:25.789444+0000","last_fullsized":"2023-09-23T04:45:25.789444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796417+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:45:25.796417+0000","last_peered":"2023-09-23T04:45:25.796417+0000","last_clean":"2023-09-23T04:45:25.796417+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:45:25.796417+0000","last_undegraded":"2023-09-23T04:45:25.796417+0000","last_fullsized":"2023-09-23T04:45:25.796417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794131+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:45:25.794131+0000","last_peered":"2023-09-23T04:45:25.794131+0000","last_clean":"2023-09-23T04:45:25.794131+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:45:25.794131+0000","last_undegraded":"2023-09-23T04:45:25.794131+0000","last_fullsized":"2023-09-23T04:45:25.794131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.965773+0000","last_change":"2023-09-23T04:44:24.821431+0000","last_active":"2023-09-23T04:45:58.965773+0000","last_peered":"2023-09-23T04:45:58.965773+0000","last_clean":"2023-09-23T04:45:58.965773+0000","last_became_active":"2023-09-23T04:44:24.812602+0000","last_became_peered":"2023-09-23T04:44:24.812602+0000","last_unstale":"2023-09-23T04:45:58.965773+0000","last_undegraded":"2023-09-23T04:45:58.965773+0000","last_fullsized":"2023-09-23T04:45:58.965773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:48:01.507032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796261+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:45:25.796261+0000","last_peered":"2023-09-23T04:45:25.796261+0000","last_clean":"2023-09-23T04:45:25.796261+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:45:25.796261+0000","last_undegraded":"2023-09-23T04:45:25.796261+0000","last_fullsized":"2023-09-23T04:45:25.796261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794308+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:45:25.794308+0000","last_peered":"2023-09-23T04:45:25.794308+0000","last_clean":"2023-09-23T04:45:25.794308+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:45:25.794308+0000","last_undegraded":"2023-09-23T04:45:25.794308+0000","last_fullsized":"2023-09-23T04:45:25.794308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.802136+0000","last_change":"2023-09-23T04:45:25.801027+0000","last_active":"2023-09-23T04:45:25.802136+0000","last_peered":"2023-09-23T04:45:25.802136+0000","last_clean":"2023-09-23T04:45:25.802136+0000","last_became_active":"2023-09-23T04:45:25.800573+0000","last_became_peered":"2023-09-23T04:45:25.800573+0000","last_unstale":"2023-09-23T04:45:25.802136+0000","last_undegraded":"2023-09-23T04:45:25.802136+0000","last_fullsized":"2023-09-23T04:45:25.802136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:03:10.223001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.634476+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:45:59.634476+0000","last_peered":"2023-09-23T04:45:59.634476+0000","last_clean":"2023-09-23T04:45:59.634476+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:45:59.634476+0000","last_undegraded":"2023-09-23T04:45:59.634476+0000","last_fullsized":"2023-09-23T04:45:59.634476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793344+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:45:25.793344+0000","last_peered":"2023-09-23T04:45:25.793344+0000","last_clean":"2023-09-23T04:45:25.793344+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:45:25.793344+0000","last_undegraded":"2023-09-23T04:45:25.793344+0000","last_fullsized":"2023-09-23T04:45:25.793344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947473+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:45:25.947473+0000","last_peered":"2023-09-23T04:45:25.947473+0000","last_clean":"2023-09-23T04:45:25.947473+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:45:25.947473+0000","last_undegraded":"2023-09-23T04:45:25.947473+0000","last_fullsized":"2023-09-23T04:45:25.947473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791581+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:45:25.791581+0000","last_peered":"2023-09-23T04:45:25.791581+0000","last_clean":"2023-09-23T04:45:25.791581+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:45:25.791581+0000","last_undegraded":"2023-09-23T04:45:25.791581+0000","last_fullsized":"2023-09-23T04:45:25.791581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792473+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:45:25.792473+0000","last_peered":"2023-09-23T04:45:25.792473+0000","last_clean":"2023-09-23T04:45:25.792473+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:45:25.792473+0000","last_undegraded":"2023-09-23T04:45:25.792473+0000","last_fullsized":"2023-09-23T04:45:25.792473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792278+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:45:24.792278+0000","last_peered":"2023-09-23T04:45:24.792278+0000","last_clean":"2023-09-23T04:45:24.792278+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:45:24.792278+0000","last_undegraded":"2023-09-23T04:45:24.792278+0000","last_fullsized":"2023-09-23T04:45:24.792278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793672+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:45:25.793672+0000","last_peered":"2023-09-23T04:45:25.793672+0000","last_clean":"2023-09-23T04:45:25.793672+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:45:25.793672+0000","last_undegraded":"2023-09-23T04:45:25.793672+0000","last_fullsized":"2023-09-23T04:45:25.793672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792657+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:45:25.792657+0000","last_peered":"2023-09-23T04:45:25.792657+0000","last_clean":"2023-09-23T04:45:25.792657+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:45:25.792657+0000","last_undegraded":"2023-09-23T04:45:25.792657+0000","last_fullsized":"2023-09-23T04:45:25.792657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794395+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:45:25.794395+0000","last_peered":"2023-09-23T04:45:25.794395+0000","last_clean":"2023-09-23T04:45:25.794395+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:45:25.794395+0000","last_undegraded":"2023-09-23T04:45:25.794395+0000","last_fullsized":"2023-09-23T04:45:25.794395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796099+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:45:25.796099+0000","last_peered":"2023-09-23T04:45:25.796099+0000","last_clean":"2023-09-23T04:45:25.796099+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:45:25.796099+0000","last_undegraded":"2023-09-23T04:45:25.796099+0000","last_fullsized":"2023-09-23T04:45:25.796099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794293+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:45:25.794293+0000","last_peered":"2023-09-23T04:45:25.794293+0000","last_clean":"2023-09-23T04:45:25.794293+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:45:25.794293+0000","last_undegraded":"2023-09-23T04:45:25.794293+0000","last_fullsized":"2023-09-23T04:45:25.794293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947273+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:45:25.947273+0000","last_peered":"2023-09-23T04:45:25.947273+0000","last_clean":"2023-09-23T04:45:25.947273+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:45:25.947273+0000","last_undegraded":"2023-09-23T04:45:25.947273+0000","last_fullsized":"2023-09-23T04:45:25.947273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789798+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:45:25.789798+0000","last_peered":"2023-09-23T04:45:25.789798+0000","last_clean":"2023-09-23T04:45:25.789798+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:45:25.789798+0000","last_undegraded":"2023-09-23T04:45:25.789798+0000","last_fullsized":"2023-09-23T04:45:25.789798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947990+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:45:25.947990+0000","last_peered":"2023-09-23T04:45:25.947990+0000","last_clean":"2023-09-23T04:45:25.947990+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:45:25.947990+0000","last_undegraded":"2023-09-23T04:45:25.947990+0000","last_fullsized":"2023-09-23T04:45:25.947990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792918+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:45:25.792918+0000","last_peered":"2023-09-23T04:45:25.792918+0000","last_clean":"2023-09-23T04:45:25.792918+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:45:25.792918+0000","last_undegraded":"2023-09-23T04:45:25.792918+0000","last_fullsized":"2023-09-23T04:45:25.792918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792003+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:45:25.792003+0000","last_peered":"2023-09-23T04:45:25.792003+0000","last_clean":"2023-09-23T04:45:25.792003+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:45:25.792003+0000","last_undegraded":"2023-09-23T04:45:25.792003+0000","last_fullsized":"2023-09-23T04:45:25.792003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790984+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:45:25.790984+0000","last_peered":"2023-09-23T04:45:25.790984+0000","last_clean":"2023-09-23T04:45:25.790984+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:45:25.790984+0000","last_undegraded":"2023-09-23T04:45:25.790984+0000","last_fullsized":"2023-09-23T04:45:25.790984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796715+0000","last_change":"2023-09-23T04:45:25.796715+0000","last_active":"2023-09-23T04:45:25.796715+0000","last_peered":"2023-09-23T04:45:25.796715+0000","last_clean":"2023-09-23T04:45:25.796715+0000","last_became_active":"2023-09-23T04:45:25.796226+0000","last_became_peered":"2023-09-23T04:45:25.796226+0000","last_unstale":"2023-09-23T04:45:25.796715+0000","last_undegraded":"2023-09-23T04:45:25.796715+0000","last_fullsized":"2023-09-23T04:45:25.796715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:29.286689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947031+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:45:25.947031+0000","last_peered":"2023-09-23T04:45:25.947031+0000","last_clean":"2023-09-23T04:45:25.947031+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:45:25.947031+0000","last_undegraded":"2023-09-23T04:45:25.947031+0000","last_fullsized":"2023-09-23T04:45:25.947031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794594+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:45:25.794594+0000","last_peered":"2023-09-23T04:45:25.794594+0000","last_clean":"2023-09-23T04:45:25.794594+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:45:25.794594+0000","last_undegraded":"2023-09-23T04:45:25.794594+0000","last_fullsized":"2023-09-23T04:45:25.794594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791471+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:45:24.791471+0000","last_peered":"2023-09-23T04:45:24.791471+0000","last_clean":"2023-09-23T04:45:24.791471+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:45:24.791471+0000","last_undegraded":"2023-09-23T04:45:24.791471+0000","last_fullsized":"2023-09-23T04:45:24.791471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947391+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:45:25.947391+0000","last_peered":"2023-09-23T04:45:25.947391+0000","last_clean":"2023-09-23T04:45:25.947391+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:45:25.947391+0000","last_undegraded":"2023-09-23T04:45:25.947391+0000","last_fullsized":"2023-09-23T04:45:25.947391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795516+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:45:25.795516+0000","last_peered":"2023-09-23T04:45:25.795516+0000","last_clean":"2023-09-23T04:45:25.795516+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:45:25.795516+0000","last_undegraded":"2023-09-23T04:45:25.795516+0000","last_fullsized":"2023-09-23T04:45:25.795516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790363+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:45:24.790363+0000","last_peered":"2023-09-23T04:45:24.790363+0000","last_clean":"2023-09-23T04:45:24.790363+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:45:24.790363+0000","last_undegraded":"2023-09-23T04:45:24.790363+0000","last_fullsized":"2023-09-23T04:45:24.790363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792496+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:45:25.792496+0000","last_peered":"2023-09-23T04:45:25.792496+0000","last_clean":"2023-09-23T04:45:25.792496+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:45:25.792496+0000","last_undegraded":"2023-09-23T04:45:25.792496+0000","last_fullsized":"2023-09-23T04:45:25.792496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797136+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:45:24.797136+0000","last_peered":"2023-09-23T04:45:24.797136+0000","last_clean":"2023-09-23T04:45:24.797136+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:45:24.797136+0000","last_undegraded":"2023-09-23T04:45:24.797136+0000","last_fullsized":"2023-09-23T04:45:24.797136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788514+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:45:24.788514+0000","last_peered":"2023-09-23T04:45:24.788514+0000","last_clean":"2023-09-23T04:45:24.788514+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:45:24.788514+0000","last_undegraded":"2023-09-23T04:45:24.788514+0000","last_fullsized":"2023-09-23T04:45:24.788514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796228+0000","last_change":"2023-09-23T04:44:24.833845+0000","last_active":"2023-09-23T04:45:24.796228+0000","last_peered":"2023-09-23T04:45:24.796228+0000","last_clean":"2023-09-23T04:45:24.796228+0000","last_became_active":"2023-09-23T04:44:24.814306+0000","last_became_peered":"2023-09-23T04:44:24.814306+0000","last_unstale":"2023-09-23T04:45:24.796228+0000","last_undegraded":"2023-09-23T04:45:24.796228+0000","last_fullsized":"2023-09-23T04:45:24.796228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:59.493234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793226+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:45:25.793226+0000","last_peered":"2023-09-23T04:45:25.793226+0000","last_clean":"2023-09-23T04:45:25.793226+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:45:25.793226+0000","last_undegraded":"2023-09-23T04:45:25.793226+0000","last_fullsized":"2023-09-23T04:45:25.793226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793328+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:45:24.793328+0000","last_peered":"2023-09-23T04:45:24.793328+0000","last_clean":"2023-09-23T04:45:24.793328+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:45:24.793328+0000","last_undegraded":"2023-09-23T04:45:24.793328+0000","last_fullsized":"2023-09-23T04:45:24.793328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.599369+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:45:56.599369+0000","last_peered":"2023-09-23T04:45:56.599369+0000","last_clean":"2023-09-23T04:45:56.599369+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:45:56.599369+0000","last_undegraded":"2023-09-23T04:45:56.599369+0000","last_fullsized":"2023-09-23T04:45:56.599369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791648+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:45:24.791648+0000","last_peered":"2023-09-23T04:45:24.791648+0000","last_clean":"2023-09-23T04:45:24.791648+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:45:24.791648+0000","last_undegraded":"2023-09-23T04:45:24.791648+0000","last_fullsized":"2023-09-23T04:45:24.791648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.948889+0000","last_change":"2023-09-23T04:45:25.948889+0000","last_active":"2023-09-23T04:45:25.948889+0000","last_peered":"2023-09-23T04:45:25.948889+0000","last_clean":"2023-09-23T04:45:25.948889+0000","last_became_active":"2023-09-23T04:45:25.948362+0000","last_became_peered":"2023-09-23T04:45:25.948362+0000","last_unstale":"2023-09-23T04:45:25.948889+0000","last_undegraded":"2023-09-23T04:45:25.948889+0000","last_fullsized":"2023-09-23T04:45:25.948889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:44:03.401943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793563+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:45:24.793563+0000","last_peered":"2023-09-23T04:45:24.793563+0000","last_clean":"2023-09-23T04:45:24.793563+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:45:24.793563+0000","last_undegraded":"2023-09-23T04:45:24.793563+0000","last_fullsized":"2023-09-23T04:45:24.793563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791588+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:45:24.791588+0000","last_peered":"2023-09-23T04:45:24.791588+0000","last_clean":"2023-09-23T04:45:24.791588+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:45:24.791588+0000","last_undegraded":"2023-09-23T04:45:24.791588+0000","last_fullsized":"2023-09-23T04:45:24.791588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791617+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:45:24.791617+0000","last_peered":"2023-09-23T04:45:24.791617+0000","last_clean":"2023-09-23T04:45:24.791617+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:45:24.791617+0000","last_undegraded":"2023-09-23T04:45:24.791617+0000","last_fullsized":"2023-09-23T04:45:24.791617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794471+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:45:25.794471+0000","last_peered":"2023-09-23T04:45:25.794471+0000","last_clean":"2023-09-23T04:45:25.794471+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:45:25.794471+0000","last_undegraded":"2023-09-23T04:45:25.794471+0000","last_fullsized":"2023-09-23T04:45:25.794471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791988+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:45:24.791988+0000","last_peered":"2023-09-23T04:45:24.791988+0000","last_clean":"2023-09-23T04:45:24.791988+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:45:24.791988+0000","last_undegraded":"2023-09-23T04:45:24.791988+0000","last_fullsized":"2023-09-23T04:45:24.791988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795607+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:45:25.795607+0000","last_peered":"2023-09-23T04:45:25.795607+0000","last_clean":"2023-09-23T04:45:25.795607+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:45:25.795607+0000","last_undegraded":"2023-09-23T04:45:25.795607+0000","last_fullsized":"2023-09-23T04:45:25.795607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793156+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:45:25.793156+0000","last_peered":"2023-09-23T04:45:25.793156+0000","last_clean":"2023-09-23T04:45:25.793156+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:45:25.793156+0000","last_undegraded":"2023-09-23T04:45:25.793156+0000","last_fullsized":"2023-09-23T04:45:25.793156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792180+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:45:25.792180+0000","last_peered":"2023-09-23T04:45:25.792180+0000","last_clean":"2023-09-23T04:45:25.792180+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:45:25.792180+0000","last_undegraded":"2023-09-23T04:45:25.792180+0000","last_fullsized":"2023-09-23T04:45:25.792180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790787+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:45:25.790787+0000","last_peered":"2023-09-23T04:45:25.790787+0000","last_clean":"2023-09-23T04:45:25.790787+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:45:25.790787+0000","last_undegraded":"2023-09-23T04:45:25.790787+0000","last_fullsized":"2023-09-23T04:45:25.790787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794389+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:45:25.794389+0000","last_peered":"2023-09-23T04:45:25.794389+0000","last_clean":"2023-09-23T04:45:25.794389+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:45:25.794389+0000","last_undegraded":"2023-09-23T04:45:25.794389+0000","last_fullsized":"2023-09-23T04:45:25.794389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794787+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:45:25.794787+0000","last_peered":"2023-09-23T04:45:25.794787+0000","last_clean":"2023-09-23T04:45:25.794787+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:45:25.794787+0000","last_undegraded":"2023-09-23T04:45:25.794787+0000","last_fullsized":"2023-09-23T04:45:25.794787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793808+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:45:25.793808+0000","last_peered":"2023-09-23T04:45:25.793808+0000","last_clean":"2023-09-23T04:45:25.793808+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:45:25.793808+0000","last_undegraded":"2023-09-23T04:45:25.793808+0000","last_fullsized":"2023-09-23T04:45:25.793808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947899+0000","last_change":"2023-09-23T04:44:24.812717+0000","last_active":"2023-09-23T04:45:25.947899+0000","last_peered":"2023-09-23T04:45:25.947899+0000","last_clean":"2023-09-23T04:45:25.947899+0000","last_became_active":"2023-09-23T04:44:24.812307+0000","last_became_peered":"2023-09-23T04:44:24.812307+0000","last_unstale":"2023-09-23T04:45:25.947899+0000","last_undegraded":"2023-09-23T04:45:25.947899+0000","last_fullsized":"2023-09-23T04:45:25.947899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:43:34.537044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796039+0000","last_change":"2023-09-23T04:44:24.999461+0000","last_active":"2023-09-23T04:45:24.796039+0000","last_peered":"2023-09-23T04:45:24.796039+0000","last_clean":"2023-09-23T04:45:24.796039+0000","last_became_active":"2023-09-23T04:44:24.999082+0000","last_became_peered":"2023-09-23T04:44:24.999082+0000","last_unstale":"2023-09-23T04:45:24.796039+0000","last_undegraded":"2023-09-23T04:45:24.796039+0000","last_fullsized":"2023-09-23T04:45:24.796039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:20.840200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797000+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:45:24.797000+0000","last_peered":"2023-09-23T04:45:24.797000+0000","last_clean":"2023-09-23T04:45:24.797000+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:45:24.797000+0000","last_undegraded":"2023-09-23T04:45:24.797000+0000","last_fullsized":"2023-09-23T04:45:24.797000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794123+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:45:24.794123+0000","last_peered":"2023-09-23T04:45:24.794123+0000","last_clean":"2023-09-23T04:45:24.794123+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:45:24.794123+0000","last_undegraded":"2023-09-23T04:45:24.794123+0000","last_fullsized":"2023-09-23T04:45:24.794123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787947+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:45:24.787947+0000","last_peered":"2023-09-23T04:45:24.787947+0000","last_clean":"2023-09-23T04:45:24.787947+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:45:24.787947+0000","last_undegraded":"2023-09-23T04:45:24.787947+0000","last_fullsized":"2023-09-23T04:45:24.787947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793154+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:45:24.793154+0000","last_peered":"2023-09-23T04:45:24.793154+0000","last_clean":"2023-09-23T04:45:24.793154+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:45:24.793154+0000","last_undegraded":"2023-09-23T04:45:24.793154+0000","last_fullsized":"2023-09-23T04:45:24.793154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68806,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1027,"num_read_kb":588,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":509022,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.71899999999999997,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":0.748},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92500000000000004,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.238,"15min":1.238},"last":0.78100000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80400000000000005,"5min":1.125,"15min":1.125},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.753}]},{"osd":6,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.80300000000000005,"15min":0.73099999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":10.571999999999999,"5min":10.571999999999999,"15min":10.571999999999999},"last":0.999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":10.398999999999999,"5min":10.398999999999999,"15min":10.398999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.92700000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86199999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88300000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":0.88900000000000001,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":9.734,"5min":9.734,"15min":9.734},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.278,"5min":0.89000000000000001,"15min":0.82599999999999996},"min":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":10.288,"5min":10.288,"15min":10.288},"last":0.97899999999999998}]},{"osd":10,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98599999999999999,"5min":1.272,"15min":1.272},"last":1.024},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.71,"15min":1.71},"last":0.94099999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1157081,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86599999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.85699999999999998,"15min":0.90700000000000003},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54500000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.62,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67800000000000005,"15min":0.71299999999999997},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.77900000000000003,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.75,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":8.0960000000000001,"15min":8.0960000000000001},"last":0.497}]},{"osd":4,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.88,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.752,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93700000000000006,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94699999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":0.94299999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0269999999999999,"15min":1.0029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":5.2880000000000003,"5min":5.3760000000000003,"15min":5.3760000000000003},"last":0.77300000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78800000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":512438,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.53700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94899999999999995,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89100000000000001,"5min":1.778,"15min":1.778},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84899999999999998,"15min":0.76000000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3080000000000001,"5min":13.364000000000001,"15min":13.364000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.78800000000000003,"15min":0.748},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73399999999999999,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.82199999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1180000000000001,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.8690000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.88100000000000001,"15min":0.90200000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0029999999999999,"5min":1.913,"15min":1.913},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.94699999999999995,"15min":0.96699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.93000000000000005,"15min":0.95799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0469999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.046,"15min":1.012},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":7.0350000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.155,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.97099999999999997,"15min":0.98199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":3.8330000000000002,"15min":3.8330000000000002},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.141,"5min":1.877,"15min":1.877},"last":0.623}]},{"osd":11,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.036,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.748}]}]},{"osd":8,"up_from":58,"seq":249108103219,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508698,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.86199999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47199999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.59399999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.57099999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.68100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.81699999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.284,"15min":1.284},"last":0.86799999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.82499999999999996,"15min":0.77300000000000002},"min":{"1min":0.70499999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.85099999999999998,"15min":0.82699999999999996},"min":{"1min":0.72699999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.60299999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.042,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.80900000000000005,"15min":0.77300000000000002},"min":{"1min":0.626,"5min":0.495,"15min":0.495},"max":{"1min":1.042,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.56899999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.98299999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.89000000000000001},"min":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.997,"5min":1.089,"15min":1.089},"last":0.80500000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1105343,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67000000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46200000000000002,"15min":0.44600000000000001},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52600000000000002,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86499999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97099999999999997,"5min":1,"15min":1},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.5570000000000004,"5min":6.5570000000000004,"15min":6.5570000000000004},"last":1.131}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89200000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.753,"15min":0.747},"min":{"1min":0.54200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.745},"min":{"1min":0.63600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98399999999999999,"5min":1.125,"15min":1.125},"last":0.755}]},{"osd":10,"last update":"Sat Sep 23 04:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71799999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":503846,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.59999999999999998,"15min":0.51900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0880000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76100000000000001,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85199999999999998,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.68000000000000005},"min":{"1min":0.55700000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85899999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78700000000000003,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0580000000000001,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.874,"5min":2.218,"15min":2.218},"last":0.436}]},{"osd":4,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96699999999999997,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99199999999999999,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.76000000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96799999999999997,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.80000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 23 04:45:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.079,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.871,"15min":0.82299999999999995},"min":{"1min":0.749,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0820000000000001,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93500000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.90500000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.093,"15min":0.98299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0409999999999999,"5min":15.34,"15min":15.34},"last":0.877},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.99099999999999999,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.242,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0069999999999999,"15min":1.0089999999999999},"min":{"1min":0.80700000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1259999999999999,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.167,"15min":1.1970000000000001},"min":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2230000000000001,"5min":6.8470000000000004,"15min":6.8470000000000004},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":1.2030000000000001,"15min":1.2490000000000001},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0920000000000001,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":565609,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.55400000000000005,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.44800000000000001,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.56799999999999995},"min":{"1min":0.58799999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.63800000000000001,"15min":0.58999999999999997},"min":{"1min":0.53000000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.497,"15min":0.45400000000000001},"min":{"1min":0.375,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.61299999999999999},"min":{"1min":0.47299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":1.087,"15min":1.087},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68400000000000005,"15min":0.63300000000000001},"min":{"1min":0.66100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.70999999999999996,"15min":0.67000000000000004},"min":{"1min":0.57999999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80400000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59599999999999997,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.65500000000000003,"15min":0.59899999999999998},"min":{"1min":0.50600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:45:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72299999999999998,"15min":0.68200000000000005},"min":{"1min":0.621,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.89900000000000002}]},{"osd":9,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.84499999999999997,"15min":0.81399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92000000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:45:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.03,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.875},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.026,"5min":1.232,"15min":1.232},"last":0.86499999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.73599999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.0609999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.83699999999999997,"15min":0.82899999999999996},"min":{"1min":0.71099999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":505302,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.64600000000000002,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89500000000000002,"5min":1.345,"15min":1.345},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.50900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.113,"5min":1.306,"15min":1.306},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71899999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68100000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.012,"15min":1.012},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68700000000000006,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.625},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.57999999999999996},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97799999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.751,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88500000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.746}]},{"osd":8,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.96099999999999997},"min":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1060000000000001,"5min":1.871,"15min":1.871},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.91200000000000003,"15min":0.93400000000000005},"min":{"1min":0.69399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.877}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88700000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.3,"15min":1.3},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.70799999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.88300000000000001,"15min":0.872},"min":{"1min":0.76800000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.78600000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":566521,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.71799999999999997,"15min":0.622},"min":{"1min":0.376,"5min":0.309,"15min":0.309},"max":{"1min":14.076000000000001,"5min":14.076000000000001,"15min":14.076000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.998,"15min":0.998},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.54900000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91800000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86799999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.48499999999999999,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52100000000000002,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.78700000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.90700000000000003}]},{"osd":7,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.745}]},{"osd":9,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":0.89700000000000002,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":14.143000000000001,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.88}]},{"osd":10,"last update":"Sat Sep 23 04:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85699999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79600000000000004,"15min":0.79500000000000004},"min":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.94399999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":503223,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.50800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32000000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.501,"15min":0.42699999999999999},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.59599999999999997},"min":{"1min":0.50600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89500000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.51100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.60099999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.60599999999999998},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.66500000000000004,"15min":0.61699999999999999},"min":{"1min":0.56299999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.70199999999999996,"15min":0.64600000000000002},"min":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89400000000000002,"5min":0.999,"15min":0.999},"last":0.97999999999999998}]},{"osd":6,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.76000000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.873},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.98099999999999998,"15min":0.93300000000000005},"min":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":10.496,"5min":10.496,"15min":10.496},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.359,"5min":0.98499999999999999,"15min":0.94299999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":12.574999999999999,"5min":12.574999999999999,"15min":12.574999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.97199999999999998,"5min":1.569,"15min":1.569},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.89600000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.95599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508879,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.34699999999999998},"min":{"1min":0.309,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.439,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.622,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.749,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.86399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66700000000000004,"15min":0.58299999999999996},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.879}]},{"osd":6,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73399999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63100000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69299999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78900000000000003}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.80300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1100939,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.52000000000000002},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.441},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":1.246,"15min":1.246},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79800000000000004,"5min":1.115,"15min":1.115},"last":0.503},{"interface":"front","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":1.071,"15min":1.071},"last":0.65200000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.377},"min":{"1min":0.314,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61599999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.72899999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.438,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.212,"15min":1.212},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":1.042,"15min":1.042},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.88200000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84399999999999997,"5min":1.23,"15min":1.23},"last":0.56899999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.53000000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.625},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.90900000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":1.292,"15min":1.292},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":1.105,"15min":1.105},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89600000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:46:01.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T04:46:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:01 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2935241686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:01 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3885840977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:01 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4057530180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:02.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 -- 172.21.15.5:0/4025513866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc100ea0 msgr2=0x7facfc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 --2- 172.21.15.5:0/4025513866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc100ea0 0x7facfc1012c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7facf0009990 tx=0x7facf0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 -- 172.21.15.5:0/4025513866 shutdown_connections 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 --2- 172.21.15.5:0/4025513866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 0x7facfc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 --2- 172.21.15.5:0/4025513866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc100ea0 0x7facfc1012c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 --2- 172.21.15.5:0/4025513866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc102080 0x7facfc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 -- 172.21.15.5:0/4025513866 >> 172.21.15.5:0/4025513866 conn(0x7facfc0fc480 msgr2=0x7facfc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 -- 172.21.15.5:0/4025513866 shutdown_connections 2023-09-23T04:46:02.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.187+0000 7fad02bff700 1 -- 172.21.15.5:0/4025513866 wait complete. 2023-09-23T04:46:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.188+0000 7fad02bff700 1 Processor -- start 2023-09-23T04:46:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 -- start start 2023-09-23T04:46:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 0x7facfc198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 0x7facfc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 0x7facfc1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1a0930 con 0x7facfc102080 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1a0a70 con 0x7facfc100ea0 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad02bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc1a0c00 con 0x7facfc102a40 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7facfbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 0x7facfc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7facfbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 0x7facfc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41018/0 (socket says 172.21.15.5:41018) 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7facfbfff700 1 -- 172.21.15.5:0/3548158269 learned_addr learned my addr 172.21.15.5:0/3548158269 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad0099b700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 0x7facfc198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.189+0000 7fad0119c700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 0x7facfc1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 -- 172.21.15.5:0/3548158269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 msgr2=0x7facfc1a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 0x7facfc1a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 -- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 msgr2=0x7facfc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 0x7facfc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 -- 172.21.15.5:0/3548158269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00095f0 con 0x7facfc100ea0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad0099b700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 0x7facfc198db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facf00098e0 tx=0x7facf0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001d070 con 0x7facfc100ea0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0021410 con 0x7facfc100ea0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc1a0e80 con 0x7facfc100ea0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0005720 con 0x7facfc100ea0 2023-09-23T04:46:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.190+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc1a11f0 con 0x7facfc100ea0 2023-09-23T04:46:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.191+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc193550 con 0x7facfc100ea0 2023-09-23T04:46:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.194+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7facf0021580 con 0x7facfc100ea0 2023-09-23T04:46:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.195+0000 7facf9ffb700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 0x7facec0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.195+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7facf00cec40 con 0x7facfc100ea0 2023-09-23T04:46:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.195+0000 7facfbfff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 0x7facec0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.196+0000 7facfbfff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 0x7facec0a3a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7face8005950 tx=0x7face8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.197+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7facf0097e70 con 0x7facfc100ea0 2023-09-23T04:46:02.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.406+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facfc02d2e0 con 0x7facec0a15a0 2023-09-23T04:46:02.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.419+0000 7facf9ffb700 1 -- 172.21.15.5:0/3548158269 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427809 (secure 0 0 0) 0x7facfc02d2e0 con 0x7facec0a15a0 2023-09-23T04:46:02.426 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 msgr2=0x7facec0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 0x7facec0a3a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7face8005950 tx=0x7face8009260 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 msgr2=0x7facfc198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 0x7facfc198db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facf00098e0 tx=0x7facf0009580 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 shutdown_connections 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7facec0a15a0 0x7facec0a3a60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7facfc102a40 0x7facfc1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facfc100ea0 0x7facfc198db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 --2- 172.21.15.5:0/3548158269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc102080 0x7facfc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 >> 172.21.15.5:0/3548158269 conn(0x7facfc0fc480 msgr2=0x7facfc107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 shutdown_connections 2023-09-23T04:46:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.424+0000 7fad02bff700 1 -- 172.21.15.5:0/3548158269 wait complete. 2023-09-23T04:46:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:46:02.444 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2023-09-23T04:46:01.665346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659420,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1163,"num_read_kb":707,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181458,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336544,"kb_used_data":12284,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605280,"statfs":{"total":1151940427776,"available":1151595806720,"internally_reserved":0,"allocated":12578816,"data_stored":8046901,"data_compressed":392895,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796321+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:45:25.796321+0000","last_peered":"2023-09-23T04:45:25.796321+0000","last_clean":"2023-09-23T04:45:25.796321+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:45:25.796321+0000","last_undegraded":"2023-09-23T04:45:25.796321+0000","last_fullsized":"2023-09-23T04:45:25.796321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791073+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:45:25.791073+0000","last_peered":"2023-09-23T04:45:25.791073+0000","last_clean":"2023-09-23T04:45:25.791073+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:45:25.791073+0000","last_undegraded":"2023-09-23T04:45:25.791073+0000","last_fullsized":"2023-09-23T04:45:25.791073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790918+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:45:25.790918+0000","last_peered":"2023-09-23T04:45:25.790918+0000","last_clean":"2023-09-23T04:45:25.790918+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:45:25.790918+0000","last_undegraded":"2023-09-23T04:45:25.790918+0000","last_fullsized":"2023-09-23T04:45:25.790918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791433+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:45:25.791433+0000","last_peered":"2023-09-23T04:45:25.791433+0000","last_clean":"2023-09-23T04:45:25.791433+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:45:25.791433+0000","last_undegraded":"2023-09-23T04:45:25.791433+0000","last_fullsized":"2023-09-23T04:45:25.791433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796437+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:45:24.796437+0000","last_peered":"2023-09-23T04:45:24.796437+0000","last_clean":"2023-09-23T04:45:24.796437+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:45:24.796437+0000","last_undegraded":"2023-09-23T04:45:24.796437+0000","last_fullsized":"2023-09-23T04:45:24.796437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788454+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:45:24.788454+0000","last_peered":"2023-09-23T04:45:24.788454+0000","last_clean":"2023-09-23T04:45:24.788454+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:45:24.788454+0000","last_undegraded":"2023-09-23T04:45:24.788454+0000","last_fullsized":"2023-09-23T04:45:24.788454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791116+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:45:24.791116+0000","last_peered":"2023-09-23T04:45:24.791116+0000","last_clean":"2023-09-23T04:45:24.791116+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:45:24.791116+0000","last_undegraded":"2023-09-23T04:45:24.791116+0000","last_fullsized":"2023-09-23T04:45:24.791116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796359+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:45:24.796359+0000","last_peered":"2023-09-23T04:45:24.796359+0000","last_clean":"2023-09-23T04:45:24.796359+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:45:24.796359+0000","last_undegraded":"2023-09-23T04:45:24.796359+0000","last_fullsized":"2023-09-23T04:45:24.796359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794018+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:45:24.794018+0000","last_peered":"2023-09-23T04:45:24.794018+0000","last_clean":"2023-09-23T04:45:24.794018+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:45:24.794018+0000","last_undegraded":"2023-09-23T04:45:24.794018+0000","last_fullsized":"2023-09-23T04:45:24.794018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.789806+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:45:24.789806+0000","last_peered":"2023-09-23T04:45:24.789806+0000","last_clean":"2023-09-23T04:45:24.789806+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:45:24.789806+0000","last_undegraded":"2023-09-23T04:45:24.789806+0000","last_fullsized":"2023-09-23T04:45:24.789806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791821+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:45:24.791821+0000","last_peered":"2023-09-23T04:45:24.791821+0000","last_clean":"2023-09-23T04:45:24.791821+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:45:24.791821+0000","last_undegraded":"2023-09-23T04:45:24.791821+0000","last_fullsized":"2023-09-23T04:45:24.791821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790570+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:45:24.790570+0000","last_peered":"2023-09-23T04:45:24.790570+0000","last_clean":"2023-09-23T04:45:24.790570+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:45:24.790570+0000","last_undegraded":"2023-09-23T04:45:24.790570+0000","last_fullsized":"2023-09-23T04:45:24.790570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788085+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:45:24.788085+0000","last_peered":"2023-09-23T04:45:24.788085+0000","last_clean":"2023-09-23T04:45:24.788085+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:45:24.788085+0000","last_undegraded":"2023-09-23T04:45:24.788085+0000","last_fullsized":"2023-09-23T04:45:24.788085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793195+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:45:24.793195+0000","last_peered":"2023-09-23T04:45:24.793195+0000","last_clean":"2023-09-23T04:45:24.793195+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:45:24.793195+0000","last_undegraded":"2023-09-23T04:45:24.793195+0000","last_fullsized":"2023-09-23T04:45:24.793195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793816+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:45:24.793816+0000","last_peered":"2023-09-23T04:45:24.793816+0000","last_clean":"2023-09-23T04:45:24.793816+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:45:24.793816+0000","last_undegraded":"2023-09-23T04:45:24.793816+0000","last_fullsized":"2023-09-23T04:45:24.793816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792675+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:45:24.792675+0000","last_peered":"2023-09-23T04:45:24.792675+0000","last_clean":"2023-09-23T04:45:24.792675+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:45:24.792675+0000","last_undegraded":"2023-09-23T04:45:24.792675+0000","last_fullsized":"2023-09-23T04:45:24.792675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793222+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:45:24.793222+0000","last_peered":"2023-09-23T04:45:24.793222+0000","last_clean":"2023-09-23T04:45:24.793222+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:45:24.793222+0000","last_undegraded":"2023-09-23T04:45:24.793222+0000","last_fullsized":"2023-09-23T04:45:24.793222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793658+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:45:24.793658+0000","last_peered":"2023-09-23T04:45:24.793658+0000","last_clean":"2023-09-23T04:45:24.793658+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:45:24.793658+0000","last_undegraded":"2023-09-23T04:45:24.793658+0000","last_fullsized":"2023-09-23T04:45:24.793658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.962541+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:45:56.962541+0000","last_peered":"2023-09-23T04:45:56.962541+0000","last_clean":"2023-09-23T04:45:56.962541+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:45:56.962541+0000","last_undegraded":"2023-09-23T04:45:56.962541+0000","last_fullsized":"2023-09-23T04:45:56.962541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947303+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:45:25.947303+0000","last_peered":"2023-09-23T04:45:25.947303+0000","last_clean":"2023-09-23T04:45:25.947303+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:45:25.947303+0000","last_undegraded":"2023-09-23T04:45:25.947303+0000","last_fullsized":"2023-09-23T04:45:25.947303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787715+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:45:24.787715+0000","last_peered":"2023-09-23T04:45:24.787715+0000","last_clean":"2023-09-23T04:45:24.787715+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:45:24.787715+0000","last_undegraded":"2023-09-23T04:45:24.787715+0000","last_fullsized":"2023-09-23T04:45:24.787715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947680+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:45:25.947680+0000","last_peered":"2023-09-23T04:45:25.947680+0000","last_clean":"2023-09-23T04:45:25.947680+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:45:25.947680+0000","last_undegraded":"2023-09-23T04:45:25.947680+0000","last_fullsized":"2023-09-23T04:45:25.947680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791138+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:45:25.791138+0000","last_peered":"2023-09-23T04:45:25.791138+0000","last_clean":"2023-09-23T04:45:25.791138+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:45:25.791138+0000","last_undegraded":"2023-09-23T04:45:25.791138+0000","last_fullsized":"2023-09-23T04:45:25.791138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788666+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:45:24.788666+0000","last_peered":"2023-09-23T04:45:24.788666+0000","last_clean":"2023-09-23T04:45:24.788666+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:45:24.788666+0000","last_undegraded":"2023-09-23T04:45:24.788666+0000","last_fullsized":"2023-09-23T04:45:24.788666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796632+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:45:24.796632+0000","last_peered":"2023-09-23T04:45:24.796632+0000","last_clean":"2023-09-23T04:45:24.796632+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:45:24.796632+0000","last_undegraded":"2023-09-23T04:45:24.796632+0000","last_fullsized":"2023-09-23T04:45:24.796632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793988+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:45:24.793988+0000","last_peered":"2023-09-23T04:45:24.793988+0000","last_clean":"2023-09-23T04:45:24.793988+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:45:24.793988+0000","last_undegraded":"2023-09-23T04:45:24.793988+0000","last_fullsized":"2023-09-23T04:45:24.793988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788201+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:45:24.788201+0000","last_peered":"2023-09-23T04:45:24.788201+0000","last_clean":"2023-09-23T04:45:24.788201+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:45:24.788201+0000","last_undegraded":"2023-09-23T04:45:24.788201+0000","last_fullsized":"2023-09-23T04:45:24.788201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796586+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:45:24.796586+0000","last_peered":"2023-09-23T04:45:24.796586+0000","last_clean":"2023-09-23T04:45:24.796586+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:45:24.796586+0000","last_undegraded":"2023-09-23T04:45:24.796586+0000","last_fullsized":"2023-09-23T04:45:24.796586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796924+0000","last_change":"2023-09-23T04:45:25.796924+0000","last_active":"2023-09-23T04:45:25.796924+0000","last_peered":"2023-09-23T04:45:25.796924+0000","last_clean":"2023-09-23T04:45:25.796924+0000","last_became_active":"2023-09-23T04:45:25.795995+0000","last_became_peered":"2023-09-23T04:45:25.795995+0000","last_unstale":"2023-09-23T04:45:25.796924+0000","last_undegraded":"2023-09-23T04:45:25.796924+0000","last_fullsized":"2023-09-23T04:45:25.796924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:43.679364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,2],"acting":[5,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793038+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:45:24.793038+0000","last_peered":"2023-09-23T04:45:24.793038+0000","last_clean":"2023-09-23T04:45:24.793038+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:45:24.793038+0000","last_undegraded":"2023-09-23T04:45:24.793038+0000","last_fullsized":"2023-09-23T04:45:24.793038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.602592+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:45:58.602592+0000","last_peered":"2023-09-23T04:45:58.602592+0000","last_clean":"2023-09-23T04:45:58.602592+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:45:58.602592+0000","last_undegraded":"2023-09-23T04:45:58.602592+0000","last_fullsized":"2023-09-23T04:45:58.602592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796301+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:45:24.796301+0000","last_peered":"2023-09-23T04:45:24.796301+0000","last_clean":"2023-09-23T04:45:24.796301+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:45:24.796301+0000","last_undegraded":"2023-09-23T04:45:24.796301+0000","last_fullsized":"2023-09-23T04:45:24.796301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.799726+0000","last_change":"2023-09-23T04:45:25.799726+0000","last_active":"2023-09-23T04:45:25.799726+0000","last_peered":"2023-09-23T04:45:25.799726+0000","last_clean":"2023-09-23T04:45:25.799726+0000","last_became_active":"2023-09-23T04:45:25.799273+0000","last_became_peered":"2023-09-23T04:45:25.799273+0000","last_unstale":"2023-09-23T04:45:25.799726+0000","last_undegraded":"2023-09-23T04:45:25.799726+0000","last_fullsized":"2023-09-23T04:45:25.799726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:19.330354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788266+0000","last_change":"2023-09-23T04:44:24.819153+0000","last_active":"2023-09-23T04:45:24.788266+0000","last_peered":"2023-09-23T04:45:24.788266+0000","last_clean":"2023-09-23T04:45:24.788266+0000","last_became_active":"2023-09-23T04:44:24.809881+0000","last_became_peered":"2023-09-23T04:44:24.809881+0000","last_unstale":"2023-09-23T04:45:24.788266+0000","last_undegraded":"2023-09-23T04:45:24.788266+0000","last_fullsized":"2023-09-23T04:45:24.788266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:38.014881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.798084+0000","last_change":"2023-09-23T04:45:25.798084+0000","last_active":"2023-09-23T04:45:25.798084+0000","last_peered":"2023-09-23T04:45:25.798084+0000","last_clean":"2023-09-23T04:45:25.798084+0000","last_became_active":"2023-09-23T04:45:25.797177+0000","last_became_peered":"2023-09-23T04:45:25.797177+0000","last_unstale":"2023-09-23T04:45:25.798084+0000","last_undegraded":"2023-09-23T04:45:25.798084+0000","last_fullsized":"2023-09-23T04:45:25.798084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:15.059834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,7],"acting":[5,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796126+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:45:25.796126+0000","last_peered":"2023-09-23T04:45:25.796126+0000","last_clean":"2023-09-23T04:45:25.796126+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:45:25.796126+0000","last_undegraded":"2023-09-23T04:45:25.796126+0000","last_fullsized":"2023-09-23T04:45:25.796126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787803+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:45:24.787803+0000","last_peered":"2023-09-23T04:45:24.787803+0000","last_clean":"2023-09-23T04:45:24.787803+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:45:24.787803+0000","last_undegraded":"2023-09-23T04:45:24.787803+0000","last_fullsized":"2023-09-23T04:45:24.787803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800083+0000","last_change":"2023-09-23T04:45:25.798999+0000","last_active":"2023-09-23T04:45:25.800083+0000","last_peered":"2023-09-23T04:45:25.800083+0000","last_clean":"2023-09-23T04:45:25.800083+0000","last_became_active":"2023-09-23T04:45:25.798608+0000","last_became_peered":"2023-09-23T04:45:25.798608+0000","last_unstale":"2023-09-23T04:45:25.800083+0000","last_undegraded":"2023-09-23T04:45:25.800083+0000","last_fullsized":"2023-09-23T04:45:25.800083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:31:20.866274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791022+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:45:25.791022+0000","last_peered":"2023-09-23T04:45:25.791022+0000","last_clean":"2023-09-23T04:45:25.791022+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:45:25.791022+0000","last_undegraded":"2023-09-23T04:45:25.791022+0000","last_fullsized":"2023-09-23T04:45:25.791022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792901+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:45:24.792901+0000","last_peered":"2023-09-23T04:45:24.792901+0000","last_clean":"2023-09-23T04:45:24.792901+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:45:24.792901+0000","last_undegraded":"2023-09-23T04:45:24.792901+0000","last_fullsized":"2023-09-23T04:45:24.792901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791370+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:45:24.791370+0000","last_peered":"2023-09-23T04:45:24.791370+0000","last_clean":"2023-09-23T04:45:24.791370+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:45:24.791370+0000","last_undegraded":"2023-09-23T04:45:24.791370+0000","last_fullsized":"2023-09-23T04:45:24.791370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796834+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:45:24.796834+0000","last_peered":"2023-09-23T04:45:24.796834+0000","last_clean":"2023-09-23T04:45:24.796834+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:45:24.796834+0000","last_undegraded":"2023-09-23T04:45:24.796834+0000","last_fullsized":"2023-09-23T04:45:24.796834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":223,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601638+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:45:58.601638+0000","last_peered":"2023-09-23T04:45:58.601638+0000","last_clean":"2023-09-23T04:45:58.601638+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:45:58.601638+0000","last_undegraded":"2023-09-23T04:45:58.601638+0000","last_fullsized":"2023-09-23T04:45:58.601638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793929+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:45:24.793929+0000","last_peered":"2023-09-23T04:45:24.793929+0000","last_clean":"2023-09-23T04:45:24.793929+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:45:24.793929+0000","last_undegraded":"2023-09-23T04:45:24.793929+0000","last_fullsized":"2023-09-23T04:45:24.793929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792058+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:45:24.792058+0000","last_peered":"2023-09-23T04:45:24.792058+0000","last_clean":"2023-09-23T04:45:24.792058+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:45:24.792058+0000","last_undegraded":"2023-09-23T04:45:24.792058+0000","last_fullsized":"2023-09-23T04:45:24.792058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791812+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:45:24.791812+0000","last_peered":"2023-09-23T04:45:24.791812+0000","last_clean":"2023-09-23T04:45:24.791812+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:45:24.791812+0000","last_undegraded":"2023-09-23T04:45:24.791812+0000","last_fullsized":"2023-09-23T04:45:24.791812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796210+0000","last_change":"2023-09-23T04:45:25.796210+0000","last_active":"2023-09-23T04:45:25.796210+0000","last_peered":"2023-09-23T04:45:25.796210+0000","last_clean":"2023-09-23T04:45:25.796210+0000","last_became_active":"2023-09-23T04:45:25.795713+0000","last_became_peered":"2023-09-23T04:45:25.795713+0000","last_unstale":"2023-09-23T04:45:25.796210+0000","last_undegraded":"2023-09-23T04:45:25.796210+0000","last_fullsized":"2023-09-23T04:45:25.796210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:02.588278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788864+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:45:24.788864+0000","last_peered":"2023-09-23T04:45:24.788864+0000","last_clean":"2023-09-23T04:45:24.788864+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:45:24.788864+0000","last_undegraded":"2023-09-23T04:45:24.788864+0000","last_fullsized":"2023-09-23T04:45:24.788864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792638+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:45:24.792638+0000","last_peered":"2023-09-23T04:45:24.792638+0000","last_clean":"2023-09-23T04:45:24.792638+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:45:24.792638+0000","last_undegraded":"2023-09-23T04:45:24.792638+0000","last_fullsized":"2023-09-23T04:45:24.792638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793884+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:45:24.793884+0000","last_peered":"2023-09-23T04:45:24.793884+0000","last_clean":"2023-09-23T04:45:24.793884+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:45:24.793884+0000","last_undegraded":"2023-09-23T04:45:24.793884+0000","last_fullsized":"2023-09-23T04:45:24.793884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791564+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:45:24.791564+0000","last_peered":"2023-09-23T04:45:24.791564+0000","last_clean":"2023-09-23T04:45:24.791564+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:45:24.791564+0000","last_undegraded":"2023-09-23T04:45:24.791564+0000","last_fullsized":"2023-09-23T04:45:24.791564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":267,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601841+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:45:58.601841+0000","last_peered":"2023-09-23T04:45:58.601841+0000","last_clean":"2023-09-23T04:45:58.601841+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:45:58.601841+0000","last_undegraded":"2023-09-23T04:45:58.601841+0000","last_fullsized":"2023-09-23T04:45:58.601841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61006,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":163,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796923+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:45:24.796923+0000","last_peered":"2023-09-23T04:45:24.796923+0000","last_clean":"2023-09-23T04:45:24.796923+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:45:24.796923+0000","last_undegraded":"2023-09-23T04:45:24.796923+0000","last_fullsized":"2023-09-23T04:45:24.796923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793813+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:45:24.793813+0000","last_peered":"2023-09-23T04:45:24.793813+0000","last_clean":"2023-09-23T04:45:24.793813+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:45:24.793813+0000","last_undegraded":"2023-09-23T04:45:24.793813+0000","last_fullsized":"2023-09-23T04:45:24.793813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792648+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:45:24.792648+0000","last_peered":"2023-09-23T04:45:24.792648+0000","last_clean":"2023-09-23T04:45:24.792648+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:45:24.792648+0000","last_undegraded":"2023-09-23T04:45:24.792648+0000","last_fullsized":"2023-09-23T04:45:24.792648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788136+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:45:24.788136+0000","last_peered":"2023-09-23T04:45:24.788136+0000","last_clean":"2023-09-23T04:45:24.788136+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:45:24.788136+0000","last_undegraded":"2023-09-23T04:45:24.788136+0000","last_fullsized":"2023-09-23T04:45:24.788136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800896+0000","last_change":"2023-09-23T04:45:25.800896+0000","last_active":"2023-09-23T04:45:25.800896+0000","last_peered":"2023-09-23T04:45:25.800896+0000","last_clean":"2023-09-23T04:45:25.800896+0000","last_became_active":"2023-09-23T04:45:25.800479+0000","last_became_peered":"2023-09-23T04:45:25.800479+0000","last_unstale":"2023-09-23T04:45:25.800896+0000","last_undegraded":"2023-09-23T04:45:25.800896+0000","last_fullsized":"2023-09-23T04:45:25.800896+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:31.746725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794928+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:45:24.794928+0000","last_peered":"2023-09-23T04:45:24.794928+0000","last_clean":"2023-09-23T04:45:24.794928+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:45:24.794928+0000","last_undegraded":"2023-09-23T04:45:24.794928+0000","last_fullsized":"2023-09-23T04:45:24.794928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791908+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:45:24.791908+0000","last_peered":"2023-09-23T04:45:24.791908+0000","last_clean":"2023-09-23T04:45:24.791908+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:45:24.791908+0000","last_undegraded":"2023-09-23T04:45:24.791908+0000","last_fullsized":"2023-09-23T04:45:24.791908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791302+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:45:24.791302+0000","last_peered":"2023-09-23T04:45:24.791302+0000","last_clean":"2023-09-23T04:45:24.791302+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:45:24.791302+0000","last_undegraded":"2023-09-23T04:45:24.791302+0000","last_fullsized":"2023-09-23T04:45:24.791302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.635371+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:45:59.635371+0000","last_peered":"2023-09-23T04:45:59.635371+0000","last_clean":"2023-09-23T04:45:59.635371+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:45:59.635371+0000","last_undegraded":"2023-09-23T04:45:59.635371+0000","last_fullsized":"2023-09-23T04:45:59.635371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.797255+0000","last_change":"2023-09-23T04:45:25.797255+0000","last_active":"2023-09-23T04:45:25.797255+0000","last_peered":"2023-09-23T04:45:25.797255+0000","last_clean":"2023-09-23T04:45:25.797255+0000","last_became_active":"2023-09-23T04:45:25.796320+0000","last_became_peered":"2023-09-23T04:45:25.796320+0000","last_unstale":"2023-09-23T04:45:25.797255+0000","last_undegraded":"2023-09-23T04:45:25.797255+0000","last_fullsized":"2023-09-23T04:45:25.797255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:22:13.933638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788018+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:45:24.788018+0000","last_peered":"2023-09-23T04:45:24.788018+0000","last_clean":"2023-09-23T04:45:24.788018+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:45:24.788018+0000","last_undegraded":"2023-09-23T04:45:24.788018+0000","last_fullsized":"2023-09-23T04:45:24.788018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796424+0000","last_change":"2023-09-23T04:44:24.830212+0000","last_active":"2023-09-23T04:45:24.796424+0000","last_peered":"2023-09-23T04:45:24.796424+0000","last_clean":"2023-09-23T04:45:24.796424+0000","last_became_active":"2023-09-23T04:44:24.809351+0000","last_became_peered":"2023-09-23T04:44:24.809351+0000","last_unstale":"2023-09-23T04:45:24.796424+0000","last_undegraded":"2023-09-23T04:45:24.796424+0000","last_fullsized":"2023-09-23T04:45:24.796424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:07:32.566651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794396+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:45:24.794396+0000","last_peered":"2023-09-23T04:45:24.794396+0000","last_clean":"2023-09-23T04:45:24.794396+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:45:24.794396+0000","last_undegraded":"2023-09-23T04:45:24.794396+0000","last_fullsized":"2023-09-23T04:45:24.794396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794287+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:45:24.794287+0000","last_peered":"2023-09-23T04:45:24.794287+0000","last_clean":"2023-09-23T04:45:24.794287+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:45:24.794287+0000","last_undegraded":"2023-09-23T04:45:24.794287+0000","last_fullsized":"2023-09-23T04:45:24.794287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796031+0000","last_change":"2023-09-23T04:44:24.833679+0000","last_active":"2023-09-23T04:45:25.796031+0000","last_peered":"2023-09-23T04:45:25.796031+0000","last_clean":"2023-09-23T04:45:25.796031+0000","last_became_active":"2023-09-23T04:44:24.814917+0000","last_became_peered":"2023-09-23T04:44:24.814917+0000","last_unstale":"2023-09-23T04:45:25.796031+0000","last_undegraded":"2023-09-23T04:45:25.796031+0000","last_fullsized":"2023-09-23T04:45:25.796031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:18:28.868416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794076+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:45:24.794076+0000","last_peered":"2023-09-23T04:45:24.794076+0000","last_clean":"2023-09-23T04:45:24.794076+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:45:24.794076+0000","last_undegraded":"2023-09-23T04:45:24.794076+0000","last_fullsized":"2023-09-23T04:45:24.794076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794710+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:45:25.794710+0000","last_peered":"2023-09-23T04:45:25.794710+0000","last_clean":"2023-09-23T04:45:25.794710+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:45:25.794710+0000","last_undegraded":"2023-09-23T04:45:25.794710+0000","last_fullsized":"2023-09-23T04:45:25.794710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789444+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:45:25.789444+0000","last_peered":"2023-09-23T04:45:25.789444+0000","last_clean":"2023-09-23T04:45:25.789444+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:45:25.789444+0000","last_undegraded":"2023-09-23T04:45:25.789444+0000","last_fullsized":"2023-09-23T04:45:25.789444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796417+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:45:25.796417+0000","last_peered":"2023-09-23T04:45:25.796417+0000","last_clean":"2023-09-23T04:45:25.796417+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:45:25.796417+0000","last_undegraded":"2023-09-23T04:45:25.796417+0000","last_fullsized":"2023-09-23T04:45:25.796417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794131+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:45:25.794131+0000","last_peered":"2023-09-23T04:45:25.794131+0000","last_clean":"2023-09-23T04:45:25.794131+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:45:25.794131+0000","last_undegraded":"2023-09-23T04:45:25.794131+0000","last_fullsized":"2023-09-23T04:45:25.794131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.965773+0000","last_change":"2023-09-23T04:44:24.821431+0000","last_active":"2023-09-23T04:45:58.965773+0000","last_peered":"2023-09-23T04:45:58.965773+0000","last_clean":"2023-09-23T04:45:58.965773+0000","last_became_active":"2023-09-23T04:44:24.812602+0000","last_became_peered":"2023-09-23T04:44:24.812602+0000","last_unstale":"2023-09-23T04:45:58.965773+0000","last_undegraded":"2023-09-23T04:45:58.965773+0000","last_fullsized":"2023-09-23T04:45:58.965773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:48:01.507032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796261+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:45:25.796261+0000","last_peered":"2023-09-23T04:45:25.796261+0000","last_clean":"2023-09-23T04:45:25.796261+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:45:25.796261+0000","last_undegraded":"2023-09-23T04:45:25.796261+0000","last_fullsized":"2023-09-23T04:45:25.796261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794308+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:45:25.794308+0000","last_peered":"2023-09-23T04:45:25.794308+0000","last_clean":"2023-09-23T04:45:25.794308+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:45:25.794308+0000","last_undegraded":"2023-09-23T04:45:25.794308+0000","last_fullsized":"2023-09-23T04:45:25.794308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.802136+0000","last_change":"2023-09-23T04:45:25.801027+0000","last_active":"2023-09-23T04:45:25.802136+0000","last_peered":"2023-09-23T04:45:25.802136+0000","last_clean":"2023-09-23T04:45:25.802136+0000","last_became_active":"2023-09-23T04:45:25.800573+0000","last_became_peered":"2023-09-23T04:45:25.800573+0000","last_unstale":"2023-09-23T04:45:25.802136+0000","last_undegraded":"2023-09-23T04:45:25.802136+0000","last_fullsized":"2023-09-23T04:45:25.802136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:03:10.223001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.634476+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:45:59.634476+0000","last_peered":"2023-09-23T04:45:59.634476+0000","last_clean":"2023-09-23T04:45:59.634476+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:45:59.634476+0000","last_undegraded":"2023-09-23T04:45:59.634476+0000","last_fullsized":"2023-09-23T04:45:59.634476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793344+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:45:25.793344+0000","last_peered":"2023-09-23T04:45:25.793344+0000","last_clean":"2023-09-23T04:45:25.793344+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:45:25.793344+0000","last_undegraded":"2023-09-23T04:45:25.793344+0000","last_fullsized":"2023-09-23T04:45:25.793344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947473+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:45:25.947473+0000","last_peered":"2023-09-23T04:45:25.947473+0000","last_clean":"2023-09-23T04:45:25.947473+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:45:25.947473+0000","last_undegraded":"2023-09-23T04:45:25.947473+0000","last_fullsized":"2023-09-23T04:45:25.947473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791581+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:45:25.791581+0000","last_peered":"2023-09-23T04:45:25.791581+0000","last_clean":"2023-09-23T04:45:25.791581+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:45:25.791581+0000","last_undegraded":"2023-09-23T04:45:25.791581+0000","last_fullsized":"2023-09-23T04:45:25.791581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792473+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:45:25.792473+0000","last_peered":"2023-09-23T04:45:25.792473+0000","last_clean":"2023-09-23T04:45:25.792473+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:45:25.792473+0000","last_undegraded":"2023-09-23T04:45:25.792473+0000","last_fullsized":"2023-09-23T04:45:25.792473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792278+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:45:24.792278+0000","last_peered":"2023-09-23T04:45:24.792278+0000","last_clean":"2023-09-23T04:45:24.792278+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:45:24.792278+0000","last_undegraded":"2023-09-23T04:45:24.792278+0000","last_fullsized":"2023-09-23T04:45:24.792278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793672+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:45:25.793672+0000","last_peered":"2023-09-23T04:45:25.793672+0000","last_clean":"2023-09-23T04:45:25.793672+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:45:25.793672+0000","last_undegraded":"2023-09-23T04:45:25.793672+0000","last_fullsized":"2023-09-23T04:45:25.793672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792657+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:45:25.792657+0000","last_peered":"2023-09-23T04:45:25.792657+0000","last_clean":"2023-09-23T04:45:25.792657+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:45:25.792657+0000","last_undegraded":"2023-09-23T04:45:25.792657+0000","last_fullsized":"2023-09-23T04:45:25.792657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794395+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:45:25.794395+0000","last_peered":"2023-09-23T04:45:25.794395+0000","last_clean":"2023-09-23T04:45:25.794395+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:45:25.794395+0000","last_undegraded":"2023-09-23T04:45:25.794395+0000","last_fullsized":"2023-09-23T04:45:25.794395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796099+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:45:25.796099+0000","last_peered":"2023-09-23T04:45:25.796099+0000","last_clean":"2023-09-23T04:45:25.796099+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:45:25.796099+0000","last_undegraded":"2023-09-23T04:45:25.796099+0000","last_fullsized":"2023-09-23T04:45:25.796099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794293+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:45:25.794293+0000","last_peered":"2023-09-23T04:45:25.794293+0000","last_clean":"2023-09-23T04:45:25.794293+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:45:25.794293+0000","last_undegraded":"2023-09-23T04:45:25.794293+0000","last_fullsized":"2023-09-23T04:45:25.794293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947273+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:45:25.947273+0000","last_peered":"2023-09-23T04:45:25.947273+0000","last_clean":"2023-09-23T04:45:25.947273+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:45:25.947273+0000","last_undegraded":"2023-09-23T04:45:25.947273+0000","last_fullsized":"2023-09-23T04:45:25.947273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789798+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:45:25.789798+0000","last_peered":"2023-09-23T04:45:25.789798+0000","last_clean":"2023-09-23T04:45:25.789798+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:45:25.789798+0000","last_undegraded":"2023-09-23T04:45:25.789798+0000","last_fullsized":"2023-09-23T04:45:25.789798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947990+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:45:25.947990+0000","last_peered":"2023-09-23T04:45:25.947990+0000","last_clean":"2023-09-23T04:45:25.947990+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:45:25.947990+0000","last_undegraded":"2023-09-23T04:45:25.947990+0000","last_fullsized":"2023-09-23T04:45:25.947990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792918+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:45:25.792918+0000","last_peered":"2023-09-23T04:45:25.792918+0000","last_clean":"2023-09-23T04:45:25.792918+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:45:25.792918+0000","last_undegraded":"2023-09-23T04:45:25.792918+0000","last_fullsized":"2023-09-23T04:45:25.792918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792003+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:45:25.792003+0000","last_peered":"2023-09-23T04:45:25.792003+0000","last_clean":"2023-09-23T04:45:25.792003+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:45:25.792003+0000","last_undegraded":"2023-09-23T04:45:25.792003+0000","last_fullsized":"2023-09-23T04:45:25.792003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790984+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:45:25.790984+0000","last_peered":"2023-09-23T04:45:25.790984+0000","last_clean":"2023-09-23T04:45:25.790984+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:45:25.790984+0000","last_undegraded":"2023-09-23T04:45:25.790984+0000","last_fullsized":"2023-09-23T04:45:25.790984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796715+0000","last_change":"2023-09-23T04:45:25.796715+0000","last_active":"2023-09-23T04:45:25.796715+0000","last_peered":"2023-09-23T04:45:25.796715+0000","last_clean":"2023-09-23T04:45:25.796715+0000","last_became_active":"2023-09-23T04:45:25.796226+0000","last_became_peered":"2023-09-23T04:45:25.796226+0000","last_unstale":"2023-09-23T04:45:25.796715+0000","last_undegraded":"2023-09-23T04:45:25.796715+0000","last_fullsized":"2023-09-23T04:45:25.796715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:29.286689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947031+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:45:25.947031+0000","last_peered":"2023-09-23T04:45:25.947031+0000","last_clean":"2023-09-23T04:45:25.947031+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:45:25.947031+0000","last_undegraded":"2023-09-23T04:45:25.947031+0000","last_fullsized":"2023-09-23T04:45:25.947031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794594+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:45:25.794594+0000","last_peered":"2023-09-23T04:45:25.794594+0000","last_clean":"2023-09-23T04:45:25.794594+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:45:25.794594+0000","last_undegraded":"2023-09-23T04:45:25.794594+0000","last_fullsized":"2023-09-23T04:45:25.794594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791471+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:45:24.791471+0000","last_peered":"2023-09-23T04:45:24.791471+0000","last_clean":"2023-09-23T04:45:24.791471+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:45:24.791471+0000","last_undegraded":"2023-09-23T04:45:24.791471+0000","last_fullsized":"2023-09-23T04:45:24.791471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947391+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:45:25.947391+0000","last_peered":"2023-09-23T04:45:25.947391+0000","last_clean":"2023-09-23T04:45:25.947391+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:45:25.947391+0000","last_undegraded":"2023-09-23T04:45:25.947391+0000","last_fullsized":"2023-09-23T04:45:25.947391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795516+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:45:25.795516+0000","last_peered":"2023-09-23T04:45:25.795516+0000","last_clean":"2023-09-23T04:45:25.795516+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:45:25.795516+0000","last_undegraded":"2023-09-23T04:45:25.795516+0000","last_fullsized":"2023-09-23T04:45:25.795516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790363+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:45:24.790363+0000","last_peered":"2023-09-23T04:45:24.790363+0000","last_clean":"2023-09-23T04:45:24.790363+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:45:24.790363+0000","last_undegraded":"2023-09-23T04:45:24.790363+0000","last_fullsized":"2023-09-23T04:45:24.790363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792496+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:45:25.792496+0000","last_peered":"2023-09-23T04:45:25.792496+0000","last_clean":"2023-09-23T04:45:25.792496+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:45:25.792496+0000","last_undegraded":"2023-09-23T04:45:25.792496+0000","last_fullsized":"2023-09-23T04:45:25.792496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797136+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:45:24.797136+0000","last_peered":"2023-09-23T04:45:24.797136+0000","last_clean":"2023-09-23T04:45:24.797136+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:45:24.797136+0000","last_undegraded":"2023-09-23T04:45:24.797136+0000","last_fullsized":"2023-09-23T04:45:24.797136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788514+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:45:24.788514+0000","last_peered":"2023-09-23T04:45:24.788514+0000","last_clean":"2023-09-23T04:45:24.788514+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:45:24.788514+0000","last_undegraded":"2023-09-23T04:45:24.788514+0000","last_fullsized":"2023-09-23T04:45:24.788514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796228+0000","last_change":"2023-09-23T04:44:24.833845+0000","last_active":"2023-09-23T04:45:24.796228+0000","last_peered":"2023-09-23T04:45:24.796228+0000","last_clean":"2023-09-23T04:45:24.796228+0000","last_became_active":"2023-09-23T04:44:24.814306+0000","last_became_peered":"2023-09-23T04:44:24.814306+0000","last_unstale":"2023-09-23T04:45:24.796228+0000","last_undegraded":"2023-09-23T04:45:24.796228+0000","last_fullsized":"2023-09-23T04:45:24.796228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:59.493234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793226+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:45:25.793226+0000","last_peered":"2023-09-23T04:45:25.793226+0000","last_clean":"2023-09-23T04:45:25.793226+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:45:25.793226+0000","last_undegraded":"2023-09-23T04:45:25.793226+0000","last_fullsized":"2023-09-23T04:45:25.793226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793328+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:45:24.793328+0000","last_peered":"2023-09-23T04:45:24.793328+0000","last_clean":"2023-09-23T04:45:24.793328+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:45:24.793328+0000","last_undegraded":"2023-09-23T04:45:24.793328+0000","last_fullsized":"2023-09-23T04:45:24.793328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.599369+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:45:56.599369+0000","last_peered":"2023-09-23T04:45:56.599369+0000","last_clean":"2023-09-23T04:45:56.599369+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:45:56.599369+0000","last_undegraded":"2023-09-23T04:45:56.599369+0000","last_fullsized":"2023-09-23T04:45:56.599369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791648+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:45:24.791648+0000","last_peered":"2023-09-23T04:45:24.791648+0000","last_clean":"2023-09-23T04:45:24.791648+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:45:24.791648+0000","last_undegraded":"2023-09-23T04:45:24.791648+0000","last_fullsized":"2023-09-23T04:45:24.791648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.948889+0000","last_change":"2023-09-23T04:45:25.948889+0000","last_active":"2023-09-23T04:45:25.948889+0000","last_peered":"2023-09-23T04:45:25.948889+0000","last_clean":"2023-09-23T04:45:25.948889+0000","last_became_active":"2023-09-23T04:45:25.948362+0000","last_became_peered":"2023-09-23T04:45:25.948362+0000","last_unstale":"2023-09-23T04:45:25.948889+0000","last_undegraded":"2023-09-23T04:45:25.948889+0000","last_fullsized":"2023-09-23T04:45:25.948889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:44:03.401943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793563+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:45:24.793563+0000","last_peered":"2023-09-23T04:45:24.793563+0000","last_clean":"2023-09-23T04:45:24.793563+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:45:24.793563+0000","last_undegraded":"2023-09-23T04:45:24.793563+0000","last_fullsized":"2023-09-23T04:45:24.793563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791588+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:45:24.791588+0000","last_peered":"2023-09-23T04:45:24.791588+0000","last_clean":"2023-09-23T04:45:24.791588+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:45:24.791588+0000","last_undegraded":"2023-09-23T04:45:24.791588+0000","last_fullsized":"2023-09-23T04:45:24.791588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791617+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:45:24.791617+0000","last_peered":"2023-09-23T04:45:24.791617+0000","last_clean":"2023-09-23T04:45:24.791617+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:45:24.791617+0000","last_undegraded":"2023-09-23T04:45:24.791617+0000","last_fullsized":"2023-09-23T04:45:24.791617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794471+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:45:25.794471+0000","last_peered":"2023-09-23T04:45:25.794471+0000","last_clean":"2023-09-23T04:45:25.794471+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:45:25.794471+0000","last_undegraded":"2023-09-23T04:45:25.794471+0000","last_fullsized":"2023-09-23T04:45:25.794471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791988+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:45:24.791988+0000","last_peered":"2023-09-23T04:45:24.791988+0000","last_clean":"2023-09-23T04:45:24.791988+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:45:24.791988+0000","last_undegraded":"2023-09-23T04:45:24.791988+0000","last_fullsized":"2023-09-23T04:45:24.791988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795607+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:45:25.795607+0000","last_peered":"2023-09-23T04:45:25.795607+0000","last_clean":"2023-09-23T04:45:25.795607+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:45:25.795607+0000","last_undegraded":"2023-09-23T04:45:25.795607+0000","last_fullsized":"2023-09-23T04:45:25.795607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793156+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:45:25.793156+0000","last_peered":"2023-09-23T04:45:25.793156+0000","last_clean":"2023-09-23T04:45:25.793156+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:45:25.793156+0000","last_undegraded":"2023-09-23T04:45:25.793156+0000","last_fullsized":"2023-09-23T04:45:25.793156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792180+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:45:25.792180+0000","last_peered":"2023-09-23T04:45:25.792180+0000","last_clean":"2023-09-23T04:45:25.792180+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:45:25.792180+0000","last_undegraded":"2023-09-23T04:45:25.792180+0000","last_fullsized":"2023-09-23T04:45:25.792180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790787+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:45:25.790787+0000","last_peered":"2023-09-23T04:45:25.790787+0000","last_clean":"2023-09-23T04:45:25.790787+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:45:25.790787+0000","last_undegraded":"2023-09-23T04:45:25.790787+0000","last_fullsized":"2023-09-23T04:45:25.790787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794389+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:45:25.794389+0000","last_peered":"2023-09-23T04:45:25.794389+0000","last_clean":"2023-09-23T04:45:25.794389+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:45:25.794389+0000","last_undegraded":"2023-09-23T04:45:25.794389+0000","last_fullsized":"2023-09-23T04:45:25.794389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794787+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:45:25.794787+0000","last_peered":"2023-09-23T04:45:25.794787+0000","last_clean":"2023-09-23T04:45:25.794787+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:45:25.794787+0000","last_undegraded":"2023-09-23T04:45:25.794787+0000","last_fullsized":"2023-09-23T04:45:25.794787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793808+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:45:25.793808+0000","last_peered":"2023-09-23T04:45:25.793808+0000","last_clean":"2023-09-23T04:45:25.793808+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:45:25.793808+0000","last_undegraded":"2023-09-23T04:45:25.793808+0000","last_fullsized":"2023-09-23T04:45:25.793808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947899+0000","last_change":"2023-09-23T04:44:24.812717+0000","last_active":"2023-09-23T04:45:25.947899+0000","last_peered":"2023-09-23T04:45:25.947899+0000","last_clean":"2023-09-23T04:45:25.947899+0000","last_became_active":"2023-09-23T04:44:24.812307+0000","last_became_peered":"2023-09-23T04:44:24.812307+0000","last_unstale":"2023-09-23T04:45:25.947899+0000","last_undegraded":"2023-09-23T04:45:25.947899+0000","last_fullsized":"2023-09-23T04:45:25.947899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:43:34.537044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796039+0000","last_change":"2023-09-23T04:44:24.999461+0000","last_active":"2023-09-23T04:45:24.796039+0000","last_peered":"2023-09-23T04:45:24.796039+0000","last_clean":"2023-09-23T04:45:24.796039+0000","last_became_active":"2023-09-23T04:44:24.999082+0000","last_became_peered":"2023-09-23T04:44:24.999082+0000","last_unstale":"2023-09-23T04:45:24.796039+0000","last_undegraded":"2023-09-23T04:45:24.796039+0000","last_fullsized":"2023-09-23T04:45:24.796039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:20.840200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797000+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:45:24.797000+0000","last_peered":"2023-09-23T04:45:24.797000+0000","last_clean":"2023-09-23T04:45:24.797000+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:45:24.797000+0000","last_undegraded":"2023-09-23T04:45:24.797000+0000","last_fullsized":"2023-09-23T04:45:24.797000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794123+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:45:24.794123+0000","last_peered":"2023-09-23T04:45:24.794123+0000","last_clean":"2023-09-23T04:45:24.794123+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:45:24.794123+0000","last_undegraded":"2023-09-23T04:45:24.794123+0000","last_fullsized":"2023-09-23T04:45:24.794123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787947+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:45:24.787947+0000","last_peered":"2023-09-23T04:45:24.787947+0000","last_clean":"2023-09-23T04:45:24.787947+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:45:24.787947+0000","last_undegraded":"2023-09-23T04:45:24.787947+0000","last_fullsized":"2023-09-23T04:45:24.787947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793154+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:45:24.793154+0000","last_peered":"2023-09-23T04:45:24.793154+0000","last_clean":"2023-09-23T04:45:24.793154+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:45:24.793154+0000","last_undegraded":"2023-09-23T04:45:24.793154+0000","last_fullsized":"2023-09-23T04:45:24.793154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68806,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1027,"num_read_kb":588,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":509022,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.71899999999999997,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":0.748},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92500000000000004,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.238,"15min":1.238},"last":0.78100000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80400000000000005,"5min":1.125,"15min":1.125},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.753}]},{"osd":6,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.80300000000000005,"15min":0.73099999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":10.571999999999999,"5min":10.571999999999999,"15min":10.571999999999999},"last":0.999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":10.398999999999999,"5min":10.398999999999999,"15min":10.398999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.92700000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86199999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88300000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":0.88900000000000001,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":9.734,"5min":9.734,"15min":9.734},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.278,"5min":0.89000000000000001,"15min":0.82599999999999996},"min":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":10.288,"5min":10.288,"15min":10.288},"last":0.97899999999999998}]},{"osd":10,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98599999999999999,"5min":1.272,"15min":1.272},"last":1.024},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.71,"15min":1.71},"last":0.94099999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1157081,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86599999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.85699999999999998,"15min":0.90700000000000003},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54500000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.62,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67800000000000005,"15min":0.71299999999999997},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.77900000000000003,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.75,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":8.0960000000000001,"15min":8.0960000000000001},"last":0.497}]},{"osd":4,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.88,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.752,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93700000000000006,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94699999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":0.94299999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0269999999999999,"15min":1.0029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":5.2880000000000003,"5min":5.3760000000000003,"15min":5.3760000000000003},"last":0.77300000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78800000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":512438,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.53700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94899999999999995,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89100000000000001,"5min":1.778,"15min":1.778},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84899999999999998,"15min":0.76000000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3080000000000001,"5min":13.364000000000001,"15min":13.364000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.78800000000000003,"15min":0.748},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73399999999999999,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.82199999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1180000000000001,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.8690000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.88100000000000001,"15min":0.90200000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0029999999999999,"5min":1.913,"15min":1.913},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.94699999999999995,"15min":0.96699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.93000000000000005,"15min":0.95799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0469999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.046,"15min":1.012},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":7.0350000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.155,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.97099999999999997,"15min":0.98199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":3.8330000000000002,"15min":3.8330000000000002},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.141,"5min":1.877,"15min":1.877},"last":0.623}]},{"osd":11,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.036,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.748}]}]},{"osd":8,"up_from":58,"seq":249108103219,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508698,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.86199999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47199999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.59399999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.57099999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.68100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.81699999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.284,"15min":1.284},"last":0.86799999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.82499999999999996,"15min":0.77300000000000002},"min":{"1min":0.70499999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.85099999999999998,"15min":0.82699999999999996},"min":{"1min":0.72699999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.60299999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.042,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.80900000000000005,"15min":0.77300000000000002},"min":{"1min":0.626,"5min":0.495,"15min":0.495},"max":{"1min":1.042,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.56899999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.98299999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.89000000000000001},"min":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.997,"5min":1.089,"15min":1.089},"last":0.80500000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1105343,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67000000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46200000000000002,"15min":0.44600000000000001},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52600000000000002,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86499999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97099999999999997,"5min":1,"15min":1},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.5570000000000004,"5min":6.5570000000000004,"15min":6.5570000000000004},"last":1.131}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89200000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.753,"15min":0.747},"min":{"1min":0.54200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.745},"min":{"1min":0.63600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98399999999999999,"5min":1.125,"15min":1.125},"last":0.755}]},{"osd":10,"last update":"Sat Sep 23 04:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71799999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":503846,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.59999999999999998,"15min":0.51900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0880000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76100000000000001,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85199999999999998,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.68000000000000005},"min":{"1min":0.55700000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85899999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78700000000000003,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0580000000000001,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.874,"5min":2.218,"15min":2.218},"last":0.436}]},{"osd":4,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96699999999999997,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99199999999999999,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.76000000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96799999999999997,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.80000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 23 04:45:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.079,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.871,"15min":0.82299999999999995},"min":{"1min":0.749,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0820000000000001,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93500000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.90500000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.093,"15min":0.98299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0409999999999999,"5min":15.34,"15min":15.34},"last":0.877},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.99099999999999999,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.242,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0069999999999999,"15min":1.0089999999999999},"min":{"1min":0.80700000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1259999999999999,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.167,"15min":1.1970000000000001},"min":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2230000000000001,"5min":6.8470000000000004,"15min":6.8470000000000004},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":1.2030000000000001,"15min":1.2490000000000001},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0920000000000001,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":565609,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.55400000000000005,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.44800000000000001,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.56799999999999995},"min":{"1min":0.58799999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.63800000000000001,"15min":0.58999999999999997},"min":{"1min":0.53000000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.497,"15min":0.45400000000000001},"min":{"1min":0.375,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.61299999999999999},"min":{"1min":0.47299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":1.087,"15min":1.087},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68400000000000005,"15min":0.63300000000000001},"min":{"1min":0.66100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.70999999999999996,"15min":0.67000000000000004},"min":{"1min":0.57999999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80400000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59599999999999997,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.65500000000000003,"15min":0.59899999999999998},"min":{"1min":0.50600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:45:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72299999999999998,"15min":0.68200000000000005},"min":{"1min":0.621,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.89900000000000002}]},{"osd":9,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.84499999999999997,"15min":0.81399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92000000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:45:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.03,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.875},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.026,"5min":1.232,"15min":1.232},"last":0.86499999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.73599999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.0609999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.83699999999999997,"15min":0.82899999999999996},"min":{"1min":0.71099999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":505302,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.64600000000000002,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89500000000000002,"5min":1.345,"15min":1.345},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.50900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.113,"5min":1.306,"15min":1.306},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71899999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68100000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.012,"15min":1.012},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68700000000000006,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.625},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.57999999999999996},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97799999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.751,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88500000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.746}]},{"osd":8,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.96099999999999997},"min":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1060000000000001,"5min":1.871,"15min":1.871},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.91200000000000003,"15min":0.93400000000000005},"min":{"1min":0.69399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.877}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88700000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.3,"15min":1.3},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.70799999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.88300000000000001,"15min":0.872},"min":{"1min":0.76800000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.78600000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":566521,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.71799999999999997,"15min":0.622},"min":{"1min":0.376,"5min":0.309,"15min":0.309},"max":{"1min":14.076000000000001,"5min":14.076000000000001,"15min":14.076000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.998,"15min":0.998},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.54900000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91800000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86799999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.48499999999999999,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52100000000000002,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.78700000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.90700000000000003}]},{"osd":7,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.745}]},{"osd":9,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":0.89700000000000002,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":14.143000000000001,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.88}]},{"osd":10,"last update":"Sat Sep 23 04:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85699999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79600000000000004,"15min":0.79500000000000004},"min":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.94399999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":503223,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.50800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32000000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.501,"15min":0.42699999999999999},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.59599999999999997},"min":{"1min":0.50600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89500000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.51100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.60099999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.60599999999999998},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.66500000000000004,"15min":0.61699999999999999},"min":{"1min":0.56299999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.70199999999999996,"15min":0.64600000000000002},"min":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89400000000000002,"5min":0.999,"15min":0.999},"last":0.97999999999999998}]},{"osd":6,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.76000000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.873},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.98099999999999998,"15min":0.93300000000000005},"min":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":10.496,"5min":10.496,"15min":10.496},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.359,"5min":0.98499999999999999,"15min":0.94299999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":12.574999999999999,"5min":12.574999999999999,"15min":12.574999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.97199999999999998,"5min":1.569,"15min":1.569},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.89600000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.95599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508879,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.34699999999999998},"min":{"1min":0.309,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.439,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.622,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.749,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.86399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66700000000000004,"15min":0.58299999999999996},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.879}]},{"osd":6,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73399999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63100000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69299999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78900000000000003}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.80300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1100939,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.52000000000000002},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.441},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":1.246,"15min":1.246},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79800000000000004,"5min":1.115,"15min":1.115},"last":0.503},{"interface":"front","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":1.071,"15min":1.071},"last":0.65200000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.377},"min":{"1min":0.314,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61599999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.72899999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.438,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.212,"15min":1.212},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":1.042,"15min":1.042},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.88200000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84399999999999997,"5min":1.23,"15min":1.23},"last":0.56899999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.53000000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.625},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.90900000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":1.292,"15min":1.292},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":1.105,"15min":1.105},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89600000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:46:02.454 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-23T04:46:02.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T04:46:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 -- 172.21.15.5:0/1631734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 msgr2=0x7f7b90103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 --2- 172.21.15.5:0/1631734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90103d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009990 tx=0x7f7b8c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 -- 172.21.15.5:0/1631734107 shutdown_connections 2023-09-23T04:46:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 --2- 172.21.15.5:0/1631734107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b90106be0 0x7f7b90109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 --2- 172.21.15.5:0/1631734107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90103d70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 --2- 172.21.15.5:0/1631734107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b901042b0 0x7f7b901066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 -- 172.21.15.5:0/1631734107 >> 172.21.15.5:0/1631734107 conn(0x7f7b900fb350 msgr2=0x7f7b900fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:02.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 -- 172.21.15.5:0/1631734107 shutdown_connections 2023-09-23T04:46:02.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.689+0000 7f7b97dff700 1 -- 172.21.15.5:0/1631734107 wait complete. 2023-09-23T04:46:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 Processor -- start 2023-09-23T04:46:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 -- start start 2023-09-23T04:46:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 0x7f7b901970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b90106be0 0x7f7b9019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019e6f0 con 0x7f7b90101980 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019e830 con 0x7f7b901042b0 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.691+0000 7f7b97dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9019e9c0 con 0x7f7b90106be0 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41040/0 (socket says 172.21.15.5:41040) 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 -- 172.21.15.5:0/263556143 learned_addr learned my addr 172.21.15.5:0/263556143 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b9639c700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b90106be0 0x7f7b9019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b9539a700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 0x7f7b901970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 -- 172.21.15.5:0/263556143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b90106be0 msgr2=0x7f7b9019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b90106be0 0x7f7b9019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 -- 172.21.15.5:0/263556143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 msgr2=0x7f7b901970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 0x7f7b901970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 -- 172.21.15.5:0/263556143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0095f0 con 0x7f7b90101980 2023-09-23T04:46:02.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b9539a700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 0x7f7b901970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.692+0000 7f7b95b9b700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0059d0 tx=0x7f7b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.693+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c01d070 con 0x7f7b90101980 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.693+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9019ec40 con 0x7f7b90101980 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.693+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8c00fc40 con 0x7f7b90101980 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.693+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0173f0 con 0x7f7b90101980 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.693+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9019f0c0 con 0x7f7b90101980 2023-09-23T04:46:02.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.694+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b900fced0 con 0x7f7b90101980 2023-09-23T04:46:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.696+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f7b8c022470 con 0x7f7b90101980 2023-09-23T04:46:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.698+0000 7f7b86ffd700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 0x7f7b7c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.698+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f7b8c0ce7e0 con 0x7f7b90101980 2023-09-23T04:46:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.698+0000 7f7b9539a700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 0x7f7b7c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.699+0000 7f7b9539a700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 0x7f7b7c0a3a60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009cb0 tx=0x7f7b80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.703+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7b8c05cec0 con 0x7f7b90101980 2023-09-23T04:46:02.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:02 smithi005 ceph-mon[97638]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-23T04:46:02.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:02 smithi118 ceph-mon[134297]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-23T04:46:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.912+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b9002d0f0 con 0x7f7b7c0a15a0 2023-09-23T04:46:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.925+0000 7f7b86ffd700 1 -- 172.21.15.5:0/263556143 <== mgr.14150 v2:172.21.15.5:6800/4010853589 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427809 (secure 0 0 0) 0x7f7b9002d0f0 con 0x7f7b7c0a15a0 2023-09-23T04:46:02.931 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 msgr2=0x7f7b7c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 0x7f7b7c0a3a60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009cb0 tx=0x7f7b80009260 comp rx=0 tx=0).stop 2023-09-23T04:46:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 msgr2=0x7f7b90196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0059d0 tx=0x7f7b8c004730 comp rx=0 tx=0).stop 2023-09-23T04:46:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 shutdown_connections 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f7b7c0a15a0 0x7f7b7c0a3a60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7b90106be0 0x7f7b9019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b90101980 0x7f7b90196b70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 --2- 172.21.15.5:0/263556143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b901042b0 0x7f7b901970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 >> 172.21.15.5:0/263556143 conn(0x7f7b900fb350 msgr2=0x7f7b901025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 shutdown_connections 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:02.931+0000 7f7b97dff700 1 -- 172.21.15.5:0/263556143 wait complete. 2023-09-23T04:46:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-09-23T04:46:02.951 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2023-09-23T04:46:01.665346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659420,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1163,"num_read_kb":707,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181458,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336544,"kb_used_data":12284,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605280,"statfs":{"total":1151940427776,"available":1151595806720,"internally_reserved":0,"allocated":12578816,"data_stored":8046901,"data_compressed":392895,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796321+0000","last_change":"2023-09-23T04:43:47.591136+0000","last_active":"2023-09-23T04:45:25.796321+0000","last_peered":"2023-09-23T04:45:25.796321+0000","last_clean":"2023-09-23T04:45:25.796321+0000","last_became_active":"2023-09-23T04:43:47.590698+0000","last_became_peered":"2023-09-23T04:43:47.590698+0000","last_unstale":"2023-09-23T04:45:25.796321+0000","last_undegraded":"2023-09-23T04:45:25.796321+0000","last_fullsized":"2023-09-23T04:45:25.796321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:36:05.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791073+0000","last_change":"2023-09-23T04:43:47.590214+0000","last_active":"2023-09-23T04:45:25.791073+0000","last_peered":"2023-09-23T04:45:25.791073+0000","last_clean":"2023-09-23T04:45:25.791073+0000","last_became_active":"2023-09-23T04:43:47.589918+0000","last_became_peered":"2023-09-23T04:43:47.589918+0000","last_unstale":"2023-09-23T04:45:25.791073+0000","last_undegraded":"2023-09-23T04:45:25.791073+0000","last_fullsized":"2023-09-23T04:45:25.791073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:03:28.115448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790918+0000","last_change":"2023-09-23T04:43:47.592822+0000","last_active":"2023-09-23T04:45:25.790918+0000","last_peered":"2023-09-23T04:45:25.790918+0000","last_clean":"2023-09-23T04:45:25.790918+0000","last_became_active":"2023-09-23T04:43:47.592513+0000","last_became_peered":"2023-09-23T04:43:47.592513+0000","last_unstale":"2023-09-23T04:45:25.790918+0000","last_undegraded":"2023-09-23T04:45:25.790918+0000","last_fullsized":"2023-09-23T04:45:25.790918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:11.064997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791433+0000","last_change":"2023-09-23T04:43:47.584326+0000","last_active":"2023-09-23T04:45:25.791433+0000","last_peered":"2023-09-23T04:45:25.791433+0000","last_clean":"2023-09-23T04:45:25.791433+0000","last_became_active":"2023-09-23T04:43:47.583146+0000","last_became_peered":"2023-09-23T04:43:47.583146+0000","last_unstale":"2023-09-23T04:45:25.791433+0000","last_undegraded":"2023-09-23T04:45:25.791433+0000","last_fullsized":"2023-09-23T04:45:25.791433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:25.106568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796437+0000","last_change":"2023-09-23T04:43:47.589237+0000","last_active":"2023-09-23T04:45:24.796437+0000","last_peered":"2023-09-23T04:45:24.796437+0000","last_clean":"2023-09-23T04:45:24.796437+0000","last_became_active":"2023-09-23T04:43:47.588904+0000","last_became_peered":"2023-09-23T04:43:47.588904+0000","last_unstale":"2023-09-23T04:45:24.796437+0000","last_undegraded":"2023-09-23T04:45:24.796437+0000","last_fullsized":"2023-09-23T04:45:24.796437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:07.966585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788454+0000","last_change":"2023-09-23T04:43:47.588872+0000","last_active":"2023-09-23T04:45:24.788454+0000","last_peered":"2023-09-23T04:45:24.788454+0000","last_clean":"2023-09-23T04:45:24.788454+0000","last_became_active":"2023-09-23T04:43:47.588547+0000","last_became_peered":"2023-09-23T04:43:47.588547+0000","last_unstale":"2023-09-23T04:45:24.788454+0000","last_undegraded":"2023-09-23T04:45:24.788454+0000","last_fullsized":"2023-09-23T04:45:24.788454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:50:36.568269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791116+0000","last_change":"2023-09-23T04:43:47.586754+0000","last_active":"2023-09-23T04:45:24.791116+0000","last_peered":"2023-09-23T04:45:24.791116+0000","last_clean":"2023-09-23T04:45:24.791116+0000","last_became_active":"2023-09-23T04:43:47.586446+0000","last_became_peered":"2023-09-23T04:43:47.586446+0000","last_unstale":"2023-09-23T04:45:24.791116+0000","last_undegraded":"2023-09-23T04:45:24.791116+0000","last_fullsized":"2023-09-23T04:45:24.791116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:17.423164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796359+0000","last_change":"2023-09-23T04:43:47.584762+0000","last_active":"2023-09-23T04:45:24.796359+0000","last_peered":"2023-09-23T04:45:24.796359+0000","last_clean":"2023-09-23T04:45:24.796359+0000","last_became_active":"2023-09-23T04:43:47.584422+0000","last_became_peered":"2023-09-23T04:43:47.584422+0000","last_unstale":"2023-09-23T04:45:24.796359+0000","last_undegraded":"2023-09-23T04:45:24.796359+0000","last_fullsized":"2023-09-23T04:45:24.796359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:33:42.957722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794018+0000","last_change":"2023-09-23T04:43:47.590735+0000","last_active":"2023-09-23T04:45:24.794018+0000","last_peered":"2023-09-23T04:45:24.794018+0000","last_clean":"2023-09-23T04:45:24.794018+0000","last_became_active":"2023-09-23T04:43:47.590405+0000","last_became_peered":"2023-09-23T04:43:47.590405+0000","last_unstale":"2023-09-23T04:45:24.794018+0000","last_undegraded":"2023-09-23T04:45:24.794018+0000","last_fullsized":"2023-09-23T04:45:24.794018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:23:25.927713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.789806+0000","last_change":"2023-09-23T04:43:47.589742+0000","last_active":"2023-09-23T04:45:24.789806+0000","last_peered":"2023-09-23T04:45:24.789806+0000","last_clean":"2023-09-23T04:45:24.789806+0000","last_became_active":"2023-09-23T04:43:47.589441+0000","last_became_peered":"2023-09-23T04:43:47.589441+0000","last_unstale":"2023-09-23T04:45:24.789806+0000","last_undegraded":"2023-09-23T04:45:24.789806+0000","last_fullsized":"2023-09-23T04:45:24.789806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:41:00.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791821+0000","last_change":"2023-09-23T04:43:47.588163+0000","last_active":"2023-09-23T04:45:24.791821+0000","last_peered":"2023-09-23T04:45:24.791821+0000","last_clean":"2023-09-23T04:45:24.791821+0000","last_became_active":"2023-09-23T04:43:47.587433+0000","last_became_peered":"2023-09-23T04:43:47.587433+0000","last_unstale":"2023-09-23T04:45:24.791821+0000","last_undegraded":"2023-09-23T04:45:24.791821+0000","last_fullsized":"2023-09-23T04:45:24.791821+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:16.971177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790570+0000","last_change":"2023-09-23T04:43:47.586695+0000","last_active":"2023-09-23T04:45:24.790570+0000","last_peered":"2023-09-23T04:45:24.790570+0000","last_clean":"2023-09-23T04:45:24.790570+0000","last_became_active":"2023-09-23T04:43:47.586230+0000","last_became_peered":"2023-09-23T04:43:47.586230+0000","last_unstale":"2023-09-23T04:45:24.790570+0000","last_undegraded":"2023-09-23T04:45:24.790570+0000","last_fullsized":"2023-09-23T04:45:24.790570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:44:04.855501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788085+0000","last_change":"2023-09-23T04:43:47.582199+0000","last_active":"2023-09-23T04:45:24.788085+0000","last_peered":"2023-09-23T04:45:24.788085+0000","last_clean":"2023-09-23T04:45:24.788085+0000","last_became_active":"2023-09-23T04:43:47.581856+0000","last_became_peered":"2023-09-23T04:43:47.581856+0000","last_unstale":"2023-09-23T04:45:24.788085+0000","last_undegraded":"2023-09-23T04:45:24.788085+0000","last_fullsized":"2023-09-23T04:45:24.788085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:02:37.551126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.594906+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.592611+0000","last_became_peered":"2023-09-23T04:43:47.592611+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:08:39.237223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792688+0000","last_change":"2023-09-23T04:43:47.589451+0000","last_active":"2023-09-23T04:45:24.792688+0000","last_peered":"2023-09-23T04:45:24.792688+0000","last_clean":"2023-09-23T04:45:24.792688+0000","last_became_active":"2023-09-23T04:43:47.589149+0000","last_became_peered":"2023-09-23T04:43:47.589149+0000","last_unstale":"2023-09-23T04:45:24.792688+0000","last_undegraded":"2023-09-23T04:45:24.792688+0000","last_fullsized":"2023-09-23T04:45:24.792688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:21.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793195+0000","last_change":"2023-09-23T04:43:47.592608+0000","last_active":"2023-09-23T04:45:24.793195+0000","last_peered":"2023-09-23T04:45:24.793195+0000","last_clean":"2023-09-23T04:45:24.793195+0000","last_became_active":"2023-09-23T04:43:47.592324+0000","last_became_peered":"2023-09-23T04:43:47.592324+0000","last_unstale":"2023-09-23T04:45:24.793195+0000","last_undegraded":"2023-09-23T04:45:24.793195+0000","last_fullsized":"2023-09-23T04:45:24.793195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:07:50.338682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793816+0000","last_change":"2023-09-23T04:43:47.597009+0000","last_active":"2023-09-23T04:45:24.793816+0000","last_peered":"2023-09-23T04:45:24.793816+0000","last_clean":"2023-09-23T04:45:24.793816+0000","last_became_active":"2023-09-23T04:43:47.596742+0000","last_became_peered":"2023-09-23T04:43:47.596742+0000","last_unstale":"2023-09-23T04:45:24.793816+0000","last_undegraded":"2023-09-23T04:45:24.793816+0000","last_fullsized":"2023-09-23T04:45:24.793816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:59:47.618213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792675+0000","last_change":"2023-09-23T04:43:47.589857+0000","last_active":"2023-09-23T04:45:24.792675+0000","last_peered":"2023-09-23T04:45:24.792675+0000","last_clean":"2023-09-23T04:45:24.792675+0000","last_became_active":"2023-09-23T04:43:47.589465+0000","last_became_peered":"2023-09-23T04:43:47.589465+0000","last_unstale":"2023-09-23T04:45:24.792675+0000","last_undegraded":"2023-09-23T04:45:24.792675+0000","last_fullsized":"2023-09-23T04:45:24.792675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:35:36.091652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793222+0000","last_change":"2023-09-23T04:43:47.595680+0000","last_active":"2023-09-23T04:45:24.793222+0000","last_peered":"2023-09-23T04:45:24.793222+0000","last_clean":"2023-09-23T04:45:24.793222+0000","last_became_active":"2023-09-23T04:43:47.595452+0000","last_became_peered":"2023-09-23T04:43:47.595452+0000","last_unstale":"2023-09-23T04:45:24.793222+0000","last_undegraded":"2023-09-23T04:45:24.793222+0000","last_fullsized":"2023-09-23T04:45:24.793222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:28.503347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793658+0000","last_change":"2023-09-23T04:43:51.735503+0000","last_active":"2023-09-23T04:45:24.793658+0000","last_peered":"2023-09-23T04:45:24.793658+0000","last_clean":"2023-09-23T04:45:24.793658+0000","last_became_active":"2023-09-23T04:43:51.735077+0000","last_became_peered":"2023-09-23T04:43:51.735077+0000","last_unstale":"2023-09-23T04:45:24.793658+0000","last_undegraded":"2023-09-23T04:45:24.793658+0000","last_fullsized":"2023-09-23T04:45:24.793658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:36.891194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.962541+0000","last_change":"2023-09-23T04:43:46.593292+0000","last_active":"2023-09-23T04:45:56.962541+0000","last_peered":"2023-09-23T04:45:56.962541+0000","last_clean":"2023-09-23T04:45:56.962541+0000","last_became_active":"2023-09-23T04:43:46.592971+0000","last_became_peered":"2023-09-23T04:43:46.592971+0000","last_unstale":"2023-09-23T04:45:56.962541+0000","last_undegraded":"2023-09-23T04:45:56.962541+0000","last_fullsized":"2023-09-23T04:45:56.962541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:48:51.493837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947303+0000","last_change":"2023-09-23T04:43:47.597277+0000","last_active":"2023-09-23T04:45:25.947303+0000","last_peered":"2023-09-23T04:45:25.947303+0000","last_clean":"2023-09-23T04:45:25.947303+0000","last_became_active":"2023-09-23T04:43:47.596915+0000","last_became_peered":"2023-09-23T04:43:47.596915+0000","last_unstale":"2023-09-23T04:45:25.947303+0000","last_undegraded":"2023-09-23T04:45:25.947303+0000","last_fullsized":"2023-09-23T04:45:25.947303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:09:06.823952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787715+0000","last_change":"2023-09-23T04:43:51.738228+0000","last_active":"2023-09-23T04:45:24.787715+0000","last_peered":"2023-09-23T04:45:24.787715+0000","last_clean":"2023-09-23T04:45:24.787715+0000","last_became_active":"2023-09-23T04:43:51.737660+0000","last_became_peered":"2023-09-23T04:43:51.737660+0000","last_unstale":"2023-09-23T04:45:24.787715+0000","last_undegraded":"2023-09-23T04:45:24.787715+0000","last_fullsized":"2023-09-23T04:45:24.787715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:40:28.664769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947680+0000","last_change":"2023-09-23T04:43:46.601862+0000","last_active":"2023-09-23T04:45:25.947680+0000","last_peered":"2023-09-23T04:45:25.947680+0000","last_clean":"2023-09-23T04:45:25.947680+0000","last_became_active":"2023-09-23T04:43:46.601475+0000","last_became_peered":"2023-09-23T04:43:46.601475+0000","last_unstale":"2023-09-23T04:45:25.947680+0000","last_undegraded":"2023-09-23T04:45:25.947680+0000","last_fullsized":"2023-09-23T04:45:25.947680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:58:36.701877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791138+0000","last_change":"2023-09-23T04:43:47.597106+0000","last_active":"2023-09-23T04:45:25.791138+0000","last_peered":"2023-09-23T04:45:25.791138+0000","last_clean":"2023-09-23T04:45:25.791138+0000","last_became_active":"2023-09-23T04:43:47.596017+0000","last_became_peered":"2023-09-23T04:43:47.596017+0000","last_unstale":"2023-09-23T04:45:25.791138+0000","last_undegraded":"2023-09-23T04:45:25.791138+0000","last_fullsized":"2023-09-23T04:45:25.791138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:13.980098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788666+0000","last_change":"2023-09-23T04:43:51.739185+0000","last_active":"2023-09-23T04:45:24.788666+0000","last_peered":"2023-09-23T04:45:24.788666+0000","last_clean":"2023-09-23T04:45:24.788666+0000","last_became_active":"2023-09-23T04:43:51.738359+0000","last_became_peered":"2023-09-23T04:43:51.738359+0000","last_unstale":"2023-09-23T04:45:24.788666+0000","last_undegraded":"2023-09-23T04:45:24.788666+0000","last_fullsized":"2023-09-23T04:45:24.788666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:40:02.254520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796632+0000","last_change":"2023-09-23T04:43:46.590697+0000","last_active":"2023-09-23T04:45:24.796632+0000","last_peered":"2023-09-23T04:45:24.796632+0000","last_clean":"2023-09-23T04:45:24.796632+0000","last_became_active":"2023-09-23T04:43:46.590390+0000","last_became_peered":"2023-09-23T04:43:46.590390+0000","last_unstale":"2023-09-23T04:45:24.796632+0000","last_undegraded":"2023-09-23T04:45:24.796632+0000","last_fullsized":"2023-09-23T04:45:24.796632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:40:21.900844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793988+0000","last_change":"2023-09-23T04:43:47.583916+0000","last_active":"2023-09-23T04:45:24.793988+0000","last_peered":"2023-09-23T04:45:24.793988+0000","last_clean":"2023-09-23T04:45:24.793988+0000","last_became_active":"2023-09-23T04:43:47.583615+0000","last_became_peered":"2023-09-23T04:43:47.583615+0000","last_unstale":"2023-09-23T04:45:24.793988+0000","last_undegraded":"2023-09-23T04:45:24.793988+0000","last_fullsized":"2023-09-23T04:45:24.793988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:14:01.225460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788201+0000","last_change":"2023-09-23T04:43:51.727265+0000","last_active":"2023-09-23T04:45:24.788201+0000","last_peered":"2023-09-23T04:45:24.788201+0000","last_clean":"2023-09-23T04:45:24.788201+0000","last_became_active":"2023-09-23T04:43:51.726868+0000","last_became_peered":"2023-09-23T04:43:51.726868+0000","last_unstale":"2023-09-23T04:45:24.788201+0000","last_undegraded":"2023-09-23T04:45:24.788201+0000","last_fullsized":"2023-09-23T04:45:24.788201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:12.173224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796586+0000","last_change":"2023-09-23T04:43:46.586690+0000","last_active":"2023-09-23T04:45:24.796586+0000","last_peered":"2023-09-23T04:45:24.796586+0000","last_clean":"2023-09-23T04:45:24.796586+0000","last_became_active":"2023-09-23T04:43:46.586455+0000","last_became_peered":"2023-09-23T04:43:46.586455+0000","last_unstale":"2023-09-23T04:45:24.796586+0000","last_undegraded":"2023-09-23T04:45:24.796586+0000","last_fullsized":"2023-09-23T04:45:24.796586+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:32:21.342148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796924+0000","last_change":"2023-09-23T04:45:25.796924+0000","last_active":"2023-09-23T04:45:25.796924+0000","last_peered":"2023-09-23T04:45:25.796924+0000","last_clean":"2023-09-23T04:45:25.796924+0000","last_became_active":"2023-09-23T04:45:25.795995+0000","last_became_peered":"2023-09-23T04:45:25.795995+0000","last_unstale":"2023-09-23T04:45:25.796924+0000","last_undegraded":"2023-09-23T04:45:25.796924+0000","last_fullsized":"2023-09-23T04:45:25.796924+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:43.679364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,2],"acting":[5,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793038+0000","last_change":"2023-09-23T04:43:51.728357+0000","last_active":"2023-09-23T04:45:24.793038+0000","last_peered":"2023-09-23T04:45:24.793038+0000","last_clean":"2023-09-23T04:45:24.793038+0000","last_became_active":"2023-09-23T04:43:51.727907+0000","last_became_peered":"2023-09-23T04:43:51.727907+0000","last_unstale":"2023-09-23T04:45:24.793038+0000","last_undegraded":"2023-09-23T04:45:24.793038+0000","last_fullsized":"2023-09-23T04:45:24.793038+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:18:51.142090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.602592+0000","last_change":"2023-09-23T04:43:46.595038+0000","last_active":"2023-09-23T04:45:58.602592+0000","last_peered":"2023-09-23T04:45:58.602592+0000","last_clean":"2023-09-23T04:45:58.602592+0000","last_became_active":"2023-09-23T04:43:46.591622+0000","last_became_peered":"2023-09-23T04:43:46.591622+0000","last_unstale":"2023-09-23T04:45:58.602592+0000","last_undegraded":"2023-09-23T04:45:58.602592+0000","last_fullsized":"2023-09-23T04:45:58.602592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:47:54.566974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796301+0000","last_change":"2023-09-23T04:43:47.586489+0000","last_active":"2023-09-23T04:45:24.796301+0000","last_peered":"2023-09-23T04:45:24.796301+0000","last_clean":"2023-09-23T04:45:24.796301+0000","last_became_active":"2023-09-23T04:43:47.586192+0000","last_became_peered":"2023-09-23T04:43:47.586192+0000","last_unstale":"2023-09-23T04:45:24.796301+0000","last_undegraded":"2023-09-23T04:45:24.796301+0000","last_fullsized":"2023-09-23T04:45:24.796301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:44:34.525255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.799726+0000","last_change":"2023-09-23T04:45:25.799726+0000","last_active":"2023-09-23T04:45:25.799726+0000","last_peered":"2023-09-23T04:45:25.799726+0000","last_clean":"2023-09-23T04:45:25.799726+0000","last_became_active":"2023-09-23T04:45:25.799273+0000","last_became_peered":"2023-09-23T04:45:25.799273+0000","last_unstale":"2023-09-23T04:45:25.799726+0000","last_undegraded":"2023-09-23T04:45:25.799726+0000","last_fullsized":"2023-09-23T04:45:25.799726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:19.330354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,10,5],"acting":[2,6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788266+0000","last_change":"2023-09-23T04:44:24.819153+0000","last_active":"2023-09-23T04:45:24.788266+0000","last_peered":"2023-09-23T04:45:24.788266+0000","last_clean":"2023-09-23T04:45:24.788266+0000","last_became_active":"2023-09-23T04:44:24.809881+0000","last_became_peered":"2023-09-23T04:44:24.809881+0000","last_unstale":"2023-09-23T04:45:24.788266+0000","last_undegraded":"2023-09-23T04:45:24.788266+0000","last_fullsized":"2023-09-23T04:45:24.788266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:38.014881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.798084+0000","last_change":"2023-09-23T04:45:25.798084+0000","last_active":"2023-09-23T04:45:25.798084+0000","last_peered":"2023-09-23T04:45:25.798084+0000","last_clean":"2023-09-23T04:45:25.798084+0000","last_became_active":"2023-09-23T04:45:25.797177+0000","last_became_peered":"2023-09-23T04:45:25.797177+0000","last_unstale":"2023-09-23T04:45:25.798084+0000","last_undegraded":"2023-09-23T04:45:25.798084+0000","last_fullsized":"2023-09-23T04:45:25.798084+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:12:15.059834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,7],"acting":[5,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796126+0000","last_change":"2023-09-23T04:43:51.735158+0000","last_active":"2023-09-23T04:45:25.796126+0000","last_peered":"2023-09-23T04:45:25.796126+0000","last_clean":"2023-09-23T04:45:25.796126+0000","last_became_active":"2023-09-23T04:43:51.734679+0000","last_became_peered":"2023-09-23T04:43:51.734679+0000","last_unstale":"2023-09-23T04:45:25.796126+0000","last_undegraded":"2023-09-23T04:45:25.796126+0000","last_fullsized":"2023-09-23T04:45:25.796126+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:12.289774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787803+0000","last_change":"2023-09-23T04:43:46.596090+0000","last_active":"2023-09-23T04:45:24.787803+0000","last_peered":"2023-09-23T04:45:24.787803+0000","last_clean":"2023-09-23T04:45:24.787803+0000","last_became_active":"2023-09-23T04:43:46.595465+0000","last_became_peered":"2023-09-23T04:43:46.595465+0000","last_unstale":"2023-09-23T04:45:24.787803+0000","last_undegraded":"2023-09-23T04:45:24.787803+0000","last_fullsized":"2023-09-23T04:45:24.787803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:21.163650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800083+0000","last_change":"2023-09-23T04:45:25.798999+0000","last_active":"2023-09-23T04:45:25.800083+0000","last_peered":"2023-09-23T04:45:25.800083+0000","last_clean":"2023-09-23T04:45:25.800083+0000","last_became_active":"2023-09-23T04:45:25.798608+0000","last_became_peered":"2023-09-23T04:45:25.798608+0000","last_unstale":"2023-09-23T04:45:25.800083+0000","last_undegraded":"2023-09-23T04:45:25.800083+0000","last_fullsized":"2023-09-23T04:45:25.800083+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:31:20.866274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791022+0000","last_change":"2023-09-23T04:43:51.732108+0000","last_active":"2023-09-23T04:45:25.791022+0000","last_peered":"2023-09-23T04:45:25.791022+0000","last_clean":"2023-09-23T04:45:25.791022+0000","last_became_active":"2023-09-23T04:43:51.731681+0000","last_became_peered":"2023-09-23T04:43:51.731681+0000","last_unstale":"2023-09-23T04:45:25.791022+0000","last_undegraded":"2023-09-23T04:45:25.791022+0000","last_fullsized":"2023-09-23T04:45:25.791022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:59.132763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792901+0000","last_change":"2023-09-23T04:43:46.589723+0000","last_active":"2023-09-23T04:45:24.792901+0000","last_peered":"2023-09-23T04:45:24.792901+0000","last_clean":"2023-09-23T04:45:24.792901+0000","last_became_active":"2023-09-23T04:43:46.589441+0000","last_became_peered":"2023-09-23T04:43:46.589441+0000","last_unstale":"2023-09-23T04:45:24.792901+0000","last_undegraded":"2023-09-23T04:45:24.792901+0000","last_fullsized":"2023-09-23T04:45:24.792901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:50.100238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791370+0000","last_change":"2023-09-23T04:43:47.593527+0000","last_active":"2023-09-23T04:45:24.791370+0000","last_peered":"2023-09-23T04:45:24.791370+0000","last_clean":"2023-09-23T04:45:24.791370+0000","last_became_active":"2023-09-23T04:43:47.593157+0000","last_became_peered":"2023-09-23T04:43:47.593157+0000","last_unstale":"2023-09-23T04:45:24.791370+0000","last_undegraded":"2023-09-23T04:45:24.791370+0000","last_fullsized":"2023-09-23T04:45:24.791370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:54.443120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796834+0000","last_change":"2023-09-23T04:43:51.736112+0000","last_active":"2023-09-23T04:45:24.796834+0000","last_peered":"2023-09-23T04:45:24.796834+0000","last_clean":"2023-09-23T04:45:24.796834+0000","last_became_active":"2023-09-23T04:43:51.735708+0000","last_became_peered":"2023-09-23T04:43:51.735708+0000","last_unstale":"2023-09-23T04:45:24.796834+0000","last_undegraded":"2023-09-23T04:45:24.796834+0000","last_fullsized":"2023-09-23T04:45:24.796834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:00:14.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":223,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601638+0000","last_change":"2023-09-23T04:43:46.588341+0000","last_active":"2023-09-23T04:45:58.601638+0000","last_peered":"2023-09-23T04:45:58.601638+0000","last_clean":"2023-09-23T04:45:58.601638+0000","last_became_active":"2023-09-23T04:43:46.588050+0000","last_became_peered":"2023-09-23T04:43:46.588050+0000","last_unstale":"2023-09-23T04:45:58.601638+0000","last_undegraded":"2023-09-23T04:45:58.601638+0000","last_fullsized":"2023-09-23T04:45:58.601638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:37:52.161039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793929+0000","last_change":"2023-09-23T04:43:47.590662+0000","last_active":"2023-09-23T04:45:24.793929+0000","last_peered":"2023-09-23T04:45:24.793929+0000","last_clean":"2023-09-23T04:45:24.793929+0000","last_became_active":"2023-09-23T04:43:47.590343+0000","last_became_peered":"2023-09-23T04:43:47.590343+0000","last_unstale":"2023-09-23T04:45:24.793929+0000","last_undegraded":"2023-09-23T04:45:24.793929+0000","last_fullsized":"2023-09-23T04:45:24.793929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:10:12.171251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792058+0000","last_change":"2023-09-23T04:43:51.735910+0000","last_active":"2023-09-23T04:45:24.792058+0000","last_peered":"2023-09-23T04:45:24.792058+0000","last_clean":"2023-09-23T04:45:24.792058+0000","last_became_active":"2023-09-23T04:43:51.735440+0000","last_became_peered":"2023-09-23T04:43:51.735440+0000","last_unstale":"2023-09-23T04:45:24.792058+0000","last_undegraded":"2023-09-23T04:45:24.792058+0000","last_fullsized":"2023-09-23T04:45:24.792058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:09.090521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791812+0000","last_change":"2023-09-23T04:43:46.592653+0000","last_active":"2023-09-23T04:45:24.791812+0000","last_peered":"2023-09-23T04:45:24.791812+0000","last_clean":"2023-09-23T04:45:24.791812+0000","last_became_active":"2023-09-23T04:43:46.592346+0000","last_became_peered":"2023-09-23T04:43:46.592346+0000","last_unstale":"2023-09-23T04:45:24.791812+0000","last_undegraded":"2023-09-23T04:45:24.791812+0000","last_fullsized":"2023-09-23T04:45:24.791812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:56:40.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796210+0000","last_change":"2023-09-23T04:45:25.796210+0000","last_active":"2023-09-23T04:45:25.796210+0000","last_peered":"2023-09-23T04:45:25.796210+0000","last_clean":"2023-09-23T04:45:25.796210+0000","last_became_active":"2023-09-23T04:45:25.795713+0000","last_became_peered":"2023-09-23T04:45:25.795713+0000","last_unstale":"2023-09-23T04:45:25.796210+0000","last_undegraded":"2023-09-23T04:45:25.796210+0000","last_fullsized":"2023-09-23T04:45:25.796210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:28:02.588278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788864+0000","last_change":"2023-09-23T04:43:51.737188+0000","last_active":"2023-09-23T04:45:24.788864+0000","last_peered":"2023-09-23T04:45:24.788864+0000","last_clean":"2023-09-23T04:45:24.788864+0000","last_became_active":"2023-09-23T04:43:51.736764+0000","last_became_peered":"2023-09-23T04:43:51.736764+0000","last_unstale":"2023-09-23T04:45:24.788864+0000","last_undegraded":"2023-09-23T04:45:24.788864+0000","last_fullsized":"2023-09-23T04:45:24.788864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:05.535949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792638+0000","last_change":"2023-09-23T04:43:46.596793+0000","last_active":"2023-09-23T04:45:24.792638+0000","last_peered":"2023-09-23T04:45:24.792638+0000","last_clean":"2023-09-23T04:45:24.792638+0000","last_became_active":"2023-09-23T04:43:46.596540+0000","last_became_peered":"2023-09-23T04:43:46.596540+0000","last_unstale":"2023-09-23T04:45:24.792638+0000","last_undegraded":"2023-09-23T04:45:24.792638+0000","last_fullsized":"2023-09-23T04:45:24.792638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:59.740012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793884+0000","last_change":"2023-09-23T04:43:47.588612+0000","last_active":"2023-09-23T04:45:24.793884+0000","last_peered":"2023-09-23T04:45:24.793884+0000","last_clean":"2023-09-23T04:45:24.793884+0000","last_became_active":"2023-09-23T04:43:47.588339+0000","last_became_peered":"2023-09-23T04:43:47.588339+0000","last_unstale":"2023-09-23T04:45:24.793884+0000","last_undegraded":"2023-09-23T04:45:24.793884+0000","last_fullsized":"2023-09-23T04:45:24.793884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:55.971973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791564+0000","last_change":"2023-09-23T04:43:51.731745+0000","last_active":"2023-09-23T04:45:24.791564+0000","last_peered":"2023-09-23T04:45:24.791564+0000","last_clean":"2023-09-23T04:45:24.791564+0000","last_became_active":"2023-09-23T04:43:51.731272+0000","last_became_peered":"2023-09-23T04:43:51.731272+0000","last_unstale":"2023-09-23T04:45:24.791564+0000","last_undegraded":"2023-09-23T04:45:24.791564+0000","last_fullsized":"2023-09-23T04:45:24.791564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:22:07.007807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":267,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.601841+0000","last_change":"2023-09-23T04:43:46.588638+0000","last_active":"2023-09-23T04:45:58.601841+0000","last_peered":"2023-09-23T04:45:58.601841+0000","last_clean":"2023-09-23T04:45:58.601841+0000","last_became_active":"2023-09-23T04:43:46.588275+0000","last_became_peered":"2023-09-23T04:43:46.588275+0000","last_unstale":"2023-09-23T04:45:58.601841+0000","last_undegraded":"2023-09-23T04:45:58.601841+0000","last_fullsized":"2023-09-23T04:45:58.601841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:21:36.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61006,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":163,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796923+0000","last_change":"2023-09-23T04:43:47.584818+0000","last_active":"2023-09-23T04:45:24.796923+0000","last_peered":"2023-09-23T04:45:24.796923+0000","last_clean":"2023-09-23T04:45:24.796923+0000","last_became_active":"2023-09-23T04:43:47.584494+0000","last_became_peered":"2023-09-23T04:43:47.584494+0000","last_unstale":"2023-09-23T04:45:24.796923+0000","last_undegraded":"2023-09-23T04:45:24.796923+0000","last_fullsized":"2023-09-23T04:45:24.796923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:04.074999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793813+0000","last_change":"2023-09-23T04:43:51.729363+0000","last_active":"2023-09-23T04:45:24.793813+0000","last_peered":"2023-09-23T04:45:24.793813+0000","last_clean":"2023-09-23T04:45:24.793813+0000","last_became_active":"2023-09-23T04:43:51.728918+0000","last_became_peered":"2023-09-23T04:43:51.728918+0000","last_unstale":"2023-09-23T04:45:24.793813+0000","last_undegraded":"2023-09-23T04:45:24.793813+0000","last_fullsized":"2023-09-23T04:45:24.793813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:58:33.691683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792648+0000","last_change":"2023-09-23T04:43:46.598960+0000","last_active":"2023-09-23T04:45:24.792648+0000","last_peered":"2023-09-23T04:45:24.792648+0000","last_clean":"2023-09-23T04:45:24.792648+0000","last_became_active":"2023-09-23T04:43:46.598667+0000","last_became_peered":"2023-09-23T04:43:46.598667+0000","last_unstale":"2023-09-23T04:45:24.792648+0000","last_undegraded":"2023-09-23T04:45:24.792648+0000","last_fullsized":"2023-09-23T04:45:24.792648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:28:21.340811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788136+0000","last_change":"2023-09-23T04:43:47.593801+0000","last_active":"2023-09-23T04:45:24.788136+0000","last_peered":"2023-09-23T04:45:24.788136+0000","last_clean":"2023-09-23T04:45:24.788136+0000","last_became_active":"2023-09-23T04:43:47.593480+0000","last_became_peered":"2023-09-23T04:43:47.593480+0000","last_unstale":"2023-09-23T04:45:24.788136+0000","last_undegraded":"2023-09-23T04:45:24.788136+0000","last_fullsized":"2023-09-23T04:45:24.788136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:33:52.702727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.800896+0000","last_change":"2023-09-23T04:45:25.800896+0000","last_active":"2023-09-23T04:45:25.800896+0000","last_peered":"2023-09-23T04:45:25.800896+0000","last_clean":"2023-09-23T04:45:25.800896+0000","last_became_active":"2023-09-23T04:45:25.800479+0000","last_became_peered":"2023-09-23T04:45:25.800479+0000","last_unstale":"2023-09-23T04:45:25.800896+0000","last_undegraded":"2023-09-23T04:45:25.800896+0000","last_fullsized":"2023-09-23T04:45:25.800896+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:31.746725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794928+0000","last_change":"2023-09-23T04:43:46.595550+0000","last_active":"2023-09-23T04:45:24.794928+0000","last_peered":"2023-09-23T04:45:24.794928+0000","last_clean":"2023-09-23T04:45:24.794928+0000","last_became_active":"2023-09-23T04:43:46.595210+0000","last_became_peered":"2023-09-23T04:43:46.595210+0000","last_unstale":"2023-09-23T04:45:24.794928+0000","last_undegraded":"2023-09-23T04:45:24.794928+0000","last_fullsized":"2023-09-23T04:45:24.794928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:47.532355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791908+0000","last_change":"2023-09-23T04:43:47.587831+0000","last_active":"2023-09-23T04:45:24.791908+0000","last_peered":"2023-09-23T04:45:24.791908+0000","last_clean":"2023-09-23T04:45:24.791908+0000","last_became_active":"2023-09-23T04:43:47.587526+0000","last_became_peered":"2023-09-23T04:43:47.587526+0000","last_unstale":"2023-09-23T04:45:24.791908+0000","last_undegraded":"2023-09-23T04:45:24.791908+0000","last_fullsized":"2023-09-23T04:45:24.791908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:19.849186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791302+0000","last_change":"2023-09-23T04:43:51.736520+0000","last_active":"2023-09-23T04:45:24.791302+0000","last_peered":"2023-09-23T04:45:24.791302+0000","last_clean":"2023-09-23T04:45:24.791302+0000","last_became_active":"2023-09-23T04:43:51.736060+0000","last_became_peered":"2023-09-23T04:43:51.736060+0000","last_unstale":"2023-09-23T04:45:24.791302+0000","last_undegraded":"2023-09-23T04:45:24.791302+0000","last_fullsized":"2023-09-23T04:45:24.791302+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:18:46.084190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":134,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.635371+0000","last_change":"2023-09-23T04:43:46.594606+0000","last_active":"2023-09-23T04:45:59.635371+0000","last_peered":"2023-09-23T04:45:59.635371+0000","last_clean":"2023-09-23T04:45:59.635371+0000","last_became_active":"2023-09-23T04:43:46.594304+0000","last_became_peered":"2023-09-23T04:43:46.594304+0000","last_unstale":"2023-09-23T04:45:59.635371+0000","last_undegraded":"2023-09-23T04:45:59.635371+0000","last_fullsized":"2023-09-23T04:45:59.635371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:43:09.183569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.797255+0000","last_change":"2023-09-23T04:45:25.797255+0000","last_active":"2023-09-23T04:45:25.797255+0000","last_peered":"2023-09-23T04:45:25.797255+0000","last_clean":"2023-09-23T04:45:25.797255+0000","last_became_active":"2023-09-23T04:45:25.796320+0000","last_became_peered":"2023-09-23T04:45:25.796320+0000","last_unstale":"2023-09-23T04:45:25.797255+0000","last_undegraded":"2023-09-23T04:45:25.797255+0000","last_fullsized":"2023-09-23T04:45:25.797255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:22:13.933638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788018+0000","last_change":"2023-09-23T04:43:51.729390+0000","last_active":"2023-09-23T04:45:24.788018+0000","last_peered":"2023-09-23T04:45:24.788018+0000","last_clean":"2023-09-23T04:45:24.788018+0000","last_became_active":"2023-09-23T04:43:51.728996+0000","last_became_peered":"2023-09-23T04:43:51.728996+0000","last_unstale":"2023-09-23T04:45:24.788018+0000","last_undegraded":"2023-09-23T04:45:24.788018+0000","last_fullsized":"2023-09-23T04:45:24.788018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:34.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796424+0000","last_change":"2023-09-23T04:44:24.830212+0000","last_active":"2023-09-23T04:45:24.796424+0000","last_peered":"2023-09-23T04:45:24.796424+0000","last_clean":"2023-09-23T04:45:24.796424+0000","last_became_active":"2023-09-23T04:44:24.809351+0000","last_became_peered":"2023-09-23T04:44:24.809351+0000","last_unstale":"2023-09-23T04:45:24.796424+0000","last_undegraded":"2023-09-23T04:45:24.796424+0000","last_fullsized":"2023-09-23T04:45:24.796424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:07:32.566651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794396+0000","last_change":"2023-09-23T04:43:47.592139+0000","last_active":"2023-09-23T04:45:24.794396+0000","last_peered":"2023-09-23T04:45:24.794396+0000","last_clean":"2023-09-23T04:45:24.794396+0000","last_became_active":"2023-09-23T04:43:47.591749+0000","last_became_peered":"2023-09-23T04:43:47.591749+0000","last_unstale":"2023-09-23T04:45:24.794396+0000","last_undegraded":"2023-09-23T04:45:24.794396+0000","last_fullsized":"2023-09-23T04:45:24.794396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:43.506123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794287+0000","last_change":"2023-09-23T04:43:51.728822+0000","last_active":"2023-09-23T04:45:24.794287+0000","last_peered":"2023-09-23T04:45:24.794287+0000","last_clean":"2023-09-23T04:45:24.794287+0000","last_became_active":"2023-09-23T04:43:51.728404+0000","last_became_peered":"2023-09-23T04:43:51.728404+0000","last_unstale":"2023-09-23T04:45:24.794287+0000","last_undegraded":"2023-09-23T04:45:24.794287+0000","last_fullsized":"2023-09-23T04:45:24.794287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:01:07.590115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796031+0000","last_change":"2023-09-23T04:44:24.833679+0000","last_active":"2023-09-23T04:45:25.796031+0000","last_peered":"2023-09-23T04:45:25.796031+0000","last_clean":"2023-09-23T04:45:25.796031+0000","last_became_active":"2023-09-23T04:44:24.814917+0000","last_became_peered":"2023-09-23T04:44:24.814917+0000","last_unstale":"2023-09-23T04:45:25.796031+0000","last_undegraded":"2023-09-23T04:45:25.796031+0000","last_fullsized":"2023-09-23T04:45:25.796031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:18:28.868416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794076+0000","last_change":"2023-09-23T04:43:47.590612+0000","last_active":"2023-09-23T04:45:24.794076+0000","last_peered":"2023-09-23T04:45:24.794076+0000","last_clean":"2023-09-23T04:45:24.794076+0000","last_became_active":"2023-09-23T04:43:47.590282+0000","last_became_peered":"2023-09-23T04:43:47.590282+0000","last_unstale":"2023-09-23T04:45:24.794076+0000","last_undegraded":"2023-09-23T04:45:24.794076+0000","last_fullsized":"2023-09-23T04:45:24.794076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:27:11.597629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794710+0000","last_change":"2023-09-23T04:43:51.737872+0000","last_active":"2023-09-23T04:45:25.794710+0000","last_peered":"2023-09-23T04:45:25.794710+0000","last_clean":"2023-09-23T04:45:25.794710+0000","last_became_active":"2023-09-23T04:43:51.737428+0000","last_became_peered":"2023-09-23T04:43:51.737428+0000","last_unstale":"2023-09-23T04:45:25.794710+0000","last_undegraded":"2023-09-23T04:45:25.794710+0000","last_fullsized":"2023-09-23T04:45:25.794710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:57:39.044108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789444+0000","last_change":"2023-09-23T04:43:46.599851+0000","last_active":"2023-09-23T04:45:25.789444+0000","last_peered":"2023-09-23T04:45:25.789444+0000","last_clean":"2023-09-23T04:45:25.789444+0000","last_became_active":"2023-09-23T04:43:46.599459+0000","last_became_peered":"2023-09-23T04:43:46.599459+0000","last_unstale":"2023-09-23T04:45:25.789444+0000","last_undegraded":"2023-09-23T04:45:25.789444+0000","last_fullsized":"2023-09-23T04:45:25.789444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:51:27.996234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796417+0000","last_change":"2023-09-23T04:43:47.589259+0000","last_active":"2023-09-23T04:45:25.796417+0000","last_peered":"2023-09-23T04:45:25.796417+0000","last_clean":"2023-09-23T04:45:25.796417+0000","last_became_active":"2023-09-23T04:43:47.588949+0000","last_became_peered":"2023-09-23T04:43:47.588949+0000","last_unstale":"2023-09-23T04:45:25.796417+0000","last_undegraded":"2023-09-23T04:45:25.796417+0000","last_fullsized":"2023-09-23T04:45:25.796417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:14:53.535929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794131+0000","last_change":"2023-09-23T04:43:51.729050+0000","last_active":"2023-09-23T04:45:25.794131+0000","last_peered":"2023-09-23T04:45:25.794131+0000","last_clean":"2023-09-23T04:45:25.794131+0000","last_became_active":"2023-09-23T04:43:51.728588+0000","last_became_peered":"2023-09-23T04:43:51.728588+0000","last_unstale":"2023-09-23T04:45:25.794131+0000","last_undegraded":"2023-09-23T04:45:25.794131+0000","last_fullsized":"2023-09-23T04:45:25.794131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:10:35.826122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":115,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:58.965773+0000","last_change":"2023-09-23T04:44:24.821431+0000","last_active":"2023-09-23T04:45:58.965773+0000","last_peered":"2023-09-23T04:45:58.965773+0000","last_clean":"2023-09-23T04:45:58.965773+0000","last_became_active":"2023-09-23T04:44:24.812602+0000","last_became_peered":"2023-09-23T04:44:24.812602+0000","last_unstale":"2023-09-23T04:45:58.965773+0000","last_undegraded":"2023-09-23T04:45:58.965773+0000","last_fullsized":"2023-09-23T04:45:58.965773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:48:01.507032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":588,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,6,1],"acting":[11,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796261+0000","last_change":"2023-09-23T04:43:47.589784+0000","last_active":"2023-09-23T04:45:25.796261+0000","last_peered":"2023-09-23T04:45:25.796261+0000","last_clean":"2023-09-23T04:45:25.796261+0000","last_became_active":"2023-09-23T04:43:47.589464+0000","last_became_peered":"2023-09-23T04:43:47.589464+0000","last_unstale":"2023-09-23T04:45:25.796261+0000","last_undegraded":"2023-09-23T04:45:25.796261+0000","last_fullsized":"2023-09-23T04:45:25.796261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:14.569981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794308+0000","last_change":"2023-09-23T04:43:51.728437+0000","last_active":"2023-09-23T04:45:25.794308+0000","last_peered":"2023-09-23T04:45:25.794308+0000","last_clean":"2023-09-23T04:45:25.794308+0000","last_became_active":"2023-09-23T04:43:51.727941+0000","last_became_peered":"2023-09-23T04:43:51.727941+0000","last_unstale":"2023-09-23T04:45:25.794308+0000","last_undegraded":"2023-09-23T04:45:25.794308+0000","last_fullsized":"2023-09-23T04:45:25.794308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:03:19.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.802136+0000","last_change":"2023-09-23T04:45:25.801027+0000","last_active":"2023-09-23T04:45:25.802136+0000","last_peered":"2023-09-23T04:45:25.802136+0000","last_clean":"2023-09-23T04:45:25.802136+0000","last_became_active":"2023-09-23T04:45:25.800573+0000","last_became_peered":"2023-09-23T04:45:25.800573+0000","last_unstale":"2023-09-23T04:45:25.802136+0000","last_undegraded":"2023-09-23T04:45:25.802136+0000","last_fullsized":"2023-09-23T04:45:25.802136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:03:10.223001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:59.634476+0000","last_change":"2023-09-23T04:43:46.597975+0000","last_active":"2023-09-23T04:45:59.634476+0000","last_peered":"2023-09-23T04:45:59.634476+0000","last_clean":"2023-09-23T04:45:59.634476+0000","last_became_active":"2023-09-23T04:43:46.597633+0000","last_became_peered":"2023-09-23T04:43:46.597633+0000","last_unstale":"2023-09-23T04:45:59.634476+0000","last_undegraded":"2023-09-23T04:45:59.634476+0000","last_fullsized":"2023-09-23T04:45:59.634476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:21.362390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793344+0000","last_change":"2023-09-23T04:43:51.727005+0000","last_active":"2023-09-23T04:45:25.793344+0000","last_peered":"2023-09-23T04:45:25.793344+0000","last_clean":"2023-09-23T04:45:25.793344+0000","last_became_active":"2023-09-23T04:43:51.726537+0000","last_became_peered":"2023-09-23T04:43:51.726537+0000","last_unstale":"2023-09-23T04:45:25.793344+0000","last_undegraded":"2023-09-23T04:45:25.793344+0000","last_fullsized":"2023-09-23T04:45:25.793344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:29:52.881070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947473+0000","last_change":"2023-09-23T04:43:47.600414+0000","last_active":"2023-09-23T04:45:25.947473+0000","last_peered":"2023-09-23T04:45:25.947473+0000","last_clean":"2023-09-23T04:45:25.947473+0000","last_became_active":"2023-09-23T04:43:47.592225+0000","last_became_peered":"2023-09-23T04:43:47.592225+0000","last_unstale":"2023-09-23T04:45:25.947473+0000","last_undegraded":"2023-09-23T04:45:25.947473+0000","last_fullsized":"2023-09-23T04:45:25.947473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:37.056184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.791581+0000","last_change":"2023-09-23T04:43:46.595884+0000","last_active":"2023-09-23T04:45:25.791581+0000","last_peered":"2023-09-23T04:45:25.791581+0000","last_clean":"2023-09-23T04:45:25.791581+0000","last_became_active":"2023-09-23T04:43:46.595516+0000","last_became_peered":"2023-09-23T04:43:46.595516+0000","last_unstale":"2023-09-23T04:45:25.791581+0000","last_undegraded":"2023-09-23T04:45:25.791581+0000","last_fullsized":"2023-09-23T04:45:25.791581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:40.133939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":326,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792473+0000","last_change":"2023-09-23T04:42:41.880492+0000","last_active":"2023-09-23T04:45:25.792473+0000","last_peered":"2023-09-23T04:45:25.792473+0000","last_clean":"2023-09-23T04:45:25.792473+0000","last_became_active":"2023-09-23T04:42:41.880054+0000","last_became_peered":"2023-09-23T04:42:41.880054+0000","last_unstale":"2023-09-23T04:45:25.792473+0000","last_undegraded":"2023-09-23T04:45:25.792473+0000","last_fullsized":"2023-09-23T04:45:25.792473+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:39:38.199692+0000","last_clean_scrub_stamp":"2023-09-23T04:39:38.199692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:22:17.364140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.792278+0000","last_change":"2023-09-23T04:43:51.738321+0000","last_active":"2023-09-23T04:45:24.792278+0000","last_peered":"2023-09-23T04:45:24.792278+0000","last_clean":"2023-09-23T04:45:24.792278+0000","last_became_active":"2023-09-23T04:43:51.737873+0000","last_became_peered":"2023-09-23T04:43:51.737873+0000","last_unstale":"2023-09-23T04:45:24.792278+0000","last_undegraded":"2023-09-23T04:45:24.792278+0000","last_fullsized":"2023-09-23T04:45:24.792278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:59:04.761814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793672+0000","last_change":"2023-09-23T04:43:47.582929+0000","last_active":"2023-09-23T04:45:25.793672+0000","last_peered":"2023-09-23T04:45:25.793672+0000","last_clean":"2023-09-23T04:45:25.793672+0000","last_became_active":"2023-09-23T04:43:47.582640+0000","last_became_peered":"2023-09-23T04:43:47.582640+0000","last_unstale":"2023-09-23T04:45:25.793672+0000","last_undegraded":"2023-09-23T04:45:25.793672+0000","last_fullsized":"2023-09-23T04:45:25.793672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:33.880097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792657+0000","last_change":"2023-09-23T04:43:46.591938+0000","last_active":"2023-09-23T04:45:25.792657+0000","last_peered":"2023-09-23T04:45:25.792657+0000","last_clean":"2023-09-23T04:45:25.792657+0000","last_became_active":"2023-09-23T04:43:46.591590+0000","last_became_peered":"2023-09-23T04:43:46.591590+0000","last_unstale":"2023-09-23T04:45:25.792657+0000","last_undegraded":"2023-09-23T04:45:25.792657+0000","last_fullsized":"2023-09-23T04:45:25.792657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:36:18.665330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794395+0000","last_change":"2023-09-23T04:43:51.729644+0000","last_active":"2023-09-23T04:45:25.794395+0000","last_peered":"2023-09-23T04:45:25.794395+0000","last_clean":"2023-09-23T04:45:25.794395+0000","last_became_active":"2023-09-23T04:43:51.729199+0000","last_became_peered":"2023-09-23T04:43:51.729199+0000","last_unstale":"2023-09-23T04:45:25.794395+0000","last_undegraded":"2023-09-23T04:45:25.794395+0000","last_fullsized":"2023-09-23T04:45:25.794395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:48.426501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796099+0000","last_change":"2023-09-23T04:43:46.599630+0000","last_active":"2023-09-23T04:45:25.796099+0000","last_peered":"2023-09-23T04:45:25.796099+0000","last_clean":"2023-09-23T04:45:25.796099+0000","last_became_active":"2023-09-23T04:43:46.599238+0000","last_became_peered":"2023-09-23T04:43:46.599238+0000","last_unstale":"2023-09-23T04:45:25.796099+0000","last_undegraded":"2023-09-23T04:45:25.796099+0000","last_fullsized":"2023-09-23T04:45:25.796099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:32.087614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794293+0000","last_change":"2023-09-23T04:43:47.591338+0000","last_active":"2023-09-23T04:45:25.794293+0000","last_peered":"2023-09-23T04:45:25.794293+0000","last_clean":"2023-09-23T04:45:25.794293+0000","last_became_active":"2023-09-23T04:43:47.591034+0000","last_became_peered":"2023-09-23T04:43:47.591034+0000","last_unstale":"2023-09-23T04:45:25.794293+0000","last_undegraded":"2023-09-23T04:45:25.794293+0000","last_fullsized":"2023-09-23T04:45:25.794293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:16.295246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947273+0000","last_change":"2023-09-23T04:43:51.733165+0000","last_active":"2023-09-23T04:45:25.947273+0000","last_peered":"2023-09-23T04:45:25.947273+0000","last_clean":"2023-09-23T04:45:25.947273+0000","last_became_active":"2023-09-23T04:43:51.732614+0000","last_became_peered":"2023-09-23T04:43:51.732614+0000","last_unstale":"2023-09-23T04:45:25.947273+0000","last_undegraded":"2023-09-23T04:45:25.947273+0000","last_fullsized":"2023-09-23T04:45:25.947273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:12:13.710646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.789798+0000","last_change":"2023-09-23T04:43:46.594911+0000","last_active":"2023-09-23T04:45:25.789798+0000","last_peered":"2023-09-23T04:45:25.789798+0000","last_clean":"2023-09-23T04:45:25.789798+0000","last_became_active":"2023-09-23T04:43:46.594506+0000","last_became_peered":"2023-09-23T04:43:46.594506+0000","last_unstale":"2023-09-23T04:45:25.789798+0000","last_undegraded":"2023-09-23T04:45:25.789798+0000","last_fullsized":"2023-09-23T04:45:25.789798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:16.186480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947990+0000","last_change":"2023-09-23T04:43:47.596551+0000","last_active":"2023-09-23T04:45:25.947990+0000","last_peered":"2023-09-23T04:45:25.947990+0000","last_clean":"2023-09-23T04:45:25.947990+0000","last_became_active":"2023-09-23T04:43:47.595988+0000","last_became_peered":"2023-09-23T04:43:47.595988+0000","last_unstale":"2023-09-23T04:45:25.947990+0000","last_undegraded":"2023-09-23T04:45:25.947990+0000","last_fullsized":"2023-09-23T04:45:25.947990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:36.642819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792918+0000","last_change":"2023-09-23T04:43:51.724900+0000","last_active":"2023-09-23T04:45:25.792918+0000","last_peered":"2023-09-23T04:45:25.792918+0000","last_clean":"2023-09-23T04:45:25.792918+0000","last_became_active":"2023-09-23T04:43:51.724565+0000","last_became_peered":"2023-09-23T04:43:51.724565+0000","last_unstale":"2023-09-23T04:45:25.792918+0000","last_undegraded":"2023-09-23T04:45:25.792918+0000","last_fullsized":"2023-09-23T04:45:25.792918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:19:20.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792003+0000","last_change":"2023-09-23T04:43:46.600118+0000","last_active":"2023-09-23T04:45:25.792003+0000","last_peered":"2023-09-23T04:45:25.792003+0000","last_clean":"2023-09-23T04:45:25.792003+0000","last_became_active":"2023-09-23T04:43:46.599770+0000","last_became_peered":"2023-09-23T04:43:46.599770+0000","last_unstale":"2023-09-23T04:45:25.792003+0000","last_undegraded":"2023-09-23T04:45:25.792003+0000","last_fullsized":"2023-09-23T04:45:25.792003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:25:16.171459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790984+0000","last_change":"2023-09-23T04:43:47.591025+0000","last_active":"2023-09-23T04:45:25.790984+0000","last_peered":"2023-09-23T04:45:25.790984+0000","last_clean":"2023-09-23T04:45:25.790984+0000","last_became_active":"2023-09-23T04:43:47.590707+0000","last_became_peered":"2023-09-23T04:43:47.590707+0000","last_unstale":"2023-09-23T04:45:25.790984+0000","last_undegraded":"2023-09-23T04:45:25.790984+0000","last_fullsized":"2023-09-23T04:45:25.790984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:29:28.115577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.796715+0000","last_change":"2023-09-23T04:45:25.796715+0000","last_active":"2023-09-23T04:45:25.796715+0000","last_peered":"2023-09-23T04:45:25.796715+0000","last_clean":"2023-09-23T04:45:25.796715+0000","last_became_active":"2023-09-23T04:45:25.796226+0000","last_became_peered":"2023-09-23T04:45:25.796226+0000","last_unstale":"2023-09-23T04:45:25.796715+0000","last_undegraded":"2023-09-23T04:45:25.796715+0000","last_fullsized":"2023-09-23T04:45:25.796715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:29.286689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947031+0000","last_change":"2023-09-23T04:43:46.599435+0000","last_active":"2023-09-23T04:45:25.947031+0000","last_peered":"2023-09-23T04:45:25.947031+0000","last_clean":"2023-09-23T04:45:25.947031+0000","last_became_active":"2023-09-23T04:43:46.599132+0000","last_became_peered":"2023-09-23T04:43:46.599132+0000","last_unstale":"2023-09-23T04:45:25.947031+0000","last_undegraded":"2023-09-23T04:45:25.947031+0000","last_fullsized":"2023-09-23T04:45:25.947031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:19:37.308830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794594+0000","last_change":"2023-09-23T04:43:47.583278+0000","last_active":"2023-09-23T04:45:25.794594+0000","last_peered":"2023-09-23T04:45:25.794594+0000","last_clean":"2023-09-23T04:45:25.794594+0000","last_became_active":"2023-09-23T04:43:47.582987+0000","last_became_peered":"2023-09-23T04:43:47.582987+0000","last_unstale":"2023-09-23T04:45:25.794594+0000","last_undegraded":"2023-09-23T04:45:25.794594+0000","last_fullsized":"2023-09-23T04:45:25.794594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:17.713050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791471+0000","last_change":"2023-09-23T04:43:51.735694+0000","last_active":"2023-09-23T04:45:24.791471+0000","last_peered":"2023-09-23T04:45:24.791471+0000","last_clean":"2023-09-23T04:45:24.791471+0000","last_became_active":"2023-09-23T04:43:51.735184+0000","last_became_peered":"2023-09-23T04:43:51.735184+0000","last_unstale":"2023-09-23T04:45:24.791471+0000","last_undegraded":"2023-09-23T04:45:24.791471+0000","last_fullsized":"2023-09-23T04:45:24.791471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:47:27.018210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947391+0000","last_change":"2023-09-23T04:43:46.595572+0000","last_active":"2023-09-23T04:45:25.947391+0000","last_peered":"2023-09-23T04:45:25.947391+0000","last_clean":"2023-09-23T04:45:25.947391+0000","last_became_active":"2023-09-23T04:43:46.595230+0000","last_became_peered":"2023-09-23T04:43:46.595230+0000","last_unstale":"2023-09-23T04:45:25.947391+0000","last_undegraded":"2023-09-23T04:45:25.947391+0000","last_fullsized":"2023-09-23T04:45:25.947391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:27:55.846678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795516+0000","last_change":"2023-09-23T04:43:47.590603+0000","last_active":"2023-09-23T04:45:25.795516+0000","last_peered":"2023-09-23T04:45:25.795516+0000","last_clean":"2023-09-23T04:45:25.795516+0000","last_became_active":"2023-09-23T04:43:47.590241+0000","last_became_peered":"2023-09-23T04:43:47.590241+0000","last_unstale":"2023-09-23T04:45:25.795516+0000","last_undegraded":"2023-09-23T04:45:25.795516+0000","last_fullsized":"2023-09-23T04:45:25.795516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:55:10.458285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.790363+0000","last_change":"2023-09-23T04:43:51.735561+0000","last_active":"2023-09-23T04:45:24.790363+0000","last_peered":"2023-09-23T04:45:24.790363+0000","last_clean":"2023-09-23T04:45:24.790363+0000","last_became_active":"2023-09-23T04:43:51.735208+0000","last_became_peered":"2023-09-23T04:43:51.735208+0000","last_unstale":"2023-09-23T04:45:24.790363+0000","last_undegraded":"2023-09-23T04:45:24.790363+0000","last_fullsized":"2023-09-23T04:45:24.790363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:31:28.790910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792496+0000","last_change":"2023-09-23T04:43:46.598774+0000","last_active":"2023-09-23T04:45:25.792496+0000","last_peered":"2023-09-23T04:45:25.792496+0000","last_clean":"2023-09-23T04:45:25.792496+0000","last_became_active":"2023-09-23T04:43:46.598461+0000","last_became_peered":"2023-09-23T04:43:46.598461+0000","last_unstale":"2023-09-23T04:45:25.792496+0000","last_undegraded":"2023-09-23T04:45:25.792496+0000","last_fullsized":"2023-09-23T04:45:25.792496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:46:23.305942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797136+0000","last_change":"2023-09-23T04:43:47.600766+0000","last_active":"2023-09-23T04:45:24.797136+0000","last_peered":"2023-09-23T04:45:24.797136+0000","last_clean":"2023-09-23T04:45:24.797136+0000","last_became_active":"2023-09-23T04:43:47.600017+0000","last_became_peered":"2023-09-23T04:43:47.600017+0000","last_unstale":"2023-09-23T04:45:24.797136+0000","last_undegraded":"2023-09-23T04:45:24.797136+0000","last_fullsized":"2023-09-23T04:45:24.797136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:16:11.230799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.788514+0000","last_change":"2023-09-23T04:43:51.728229+0000","last_active":"2023-09-23T04:45:24.788514+0000","last_peered":"2023-09-23T04:45:24.788514+0000","last_clean":"2023-09-23T04:45:24.788514+0000","last_became_active":"2023-09-23T04:43:51.727785+0000","last_became_peered":"2023-09-23T04:43:51.727785+0000","last_unstale":"2023-09-23T04:45:24.788514+0000","last_undegraded":"2023-09-23T04:45:24.788514+0000","last_fullsized":"2023-09-23T04:45:24.788514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:39:58.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796228+0000","last_change":"2023-09-23T04:44:24.833845+0000","last_active":"2023-09-23T04:45:24.796228+0000","last_peered":"2023-09-23T04:45:24.796228+0000","last_clean":"2023-09-23T04:45:24.796228+0000","last_became_active":"2023-09-23T04:44:24.814306+0000","last_became_peered":"2023-09-23T04:44:24.814306+0000","last_unstale":"2023-09-23T04:45:24.796228+0000","last_undegraded":"2023-09-23T04:45:24.796228+0000","last_fullsized":"2023-09-23T04:45:24.796228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:59.493234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793226+0000","last_change":"2023-09-23T04:43:47.588968+0000","last_active":"2023-09-23T04:45:25.793226+0000","last_peered":"2023-09-23T04:45:25.793226+0000","last_clean":"2023-09-23T04:45:25.793226+0000","last_became_active":"2023-09-23T04:43:47.588715+0000","last_became_peered":"2023-09-23T04:43:47.588715+0000","last_unstale":"2023-09-23T04:45:25.793226+0000","last_undegraded":"2023-09-23T04:45:25.793226+0000","last_fullsized":"2023-09-23T04:45:25.793226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:19:33.749824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793328+0000","last_change":"2023-09-23T04:43:51.728299+0000","last_active":"2023-09-23T04:45:24.793328+0000","last_peered":"2023-09-23T04:45:24.793328+0000","last_clean":"2023-09-23T04:45:24.793328+0000","last_became_active":"2023-09-23T04:43:51.727900+0000","last_became_peered":"2023-09-23T04:43:51.727900+0000","last_unstale":"2023-09-23T04:45:24.793328+0000","last_undegraded":"2023-09-23T04:45:24.793328+0000","last_fullsized":"2023-09-23T04:45:24.793328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:02.344795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:56.599369+0000","last_change":"2023-09-23T04:43:46.597538+0000","last_active":"2023-09-23T04:45:56.599369+0000","last_peered":"2023-09-23T04:45:56.599369+0000","last_clean":"2023-09-23T04:45:56.599369+0000","last_became_active":"2023-09-23T04:43:46.597217+0000","last_became_peered":"2023-09-23T04:43:46.597217+0000","last_unstale":"2023-09-23T04:45:56.599369+0000","last_undegraded":"2023-09-23T04:45:56.599369+0000","last_fullsized":"2023-09-23T04:45:56.599369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:41:55.322431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791648+0000","last_change":"2023-09-23T04:43:47.588354+0000","last_active":"2023-09-23T04:45:24.791648+0000","last_peered":"2023-09-23T04:45:24.791648+0000","last_clean":"2023-09-23T04:45:24.791648+0000","last_became_active":"2023-09-23T04:43:47.587845+0000","last_became_peered":"2023-09-23T04:43:47.587845+0000","last_unstale":"2023-09-23T04:45:24.791648+0000","last_undegraded":"2023-09-23T04:45:24.791648+0000","last_fullsized":"2023-09-23T04:45:24.791648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:21:37.592304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.948889+0000","last_change":"2023-09-23T04:45:25.948889+0000","last_active":"2023-09-23T04:45:25.948889+0000","last_peered":"2023-09-23T04:45:25.948889+0000","last_clean":"2023-09-23T04:45:25.948889+0000","last_became_active":"2023-09-23T04:45:25.948362+0000","last_became_peered":"2023-09-23T04:45:25.948362+0000","last_unstale":"2023-09-23T04:45:25.948889+0000","last_undegraded":"2023-09-23T04:45:25.948889+0000","last_fullsized":"2023-09-23T04:45:25.948889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:44:03.401943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793563+0000","last_change":"2023-09-23T04:43:46.595665+0000","last_active":"2023-09-23T04:45:24.793563+0000","last_peered":"2023-09-23T04:45:24.793563+0000","last_clean":"2023-09-23T04:45:24.793563+0000","last_became_active":"2023-09-23T04:43:46.595362+0000","last_became_peered":"2023-09-23T04:43:46.595362+0000","last_unstale":"2023-09-23T04:45:24.793563+0000","last_undegraded":"2023-09-23T04:45:24.793563+0000","last_fullsized":"2023-09-23T04:45:24.793563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:22:20.671541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791588+0000","last_change":"2023-09-23T04:43:47.602882+0000","last_active":"2023-09-23T04:45:24.791588+0000","last_peered":"2023-09-23T04:45:24.791588+0000","last_clean":"2023-09-23T04:45:24.791588+0000","last_became_active":"2023-09-23T04:43:47.602640+0000","last_became_peered":"2023-09-23T04:43:47.602640+0000","last_unstale":"2023-09-23T04:45:24.791588+0000","last_undegraded":"2023-09-23T04:45:24.791588+0000","last_fullsized":"2023-09-23T04:45:24.791588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:44:49.894502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791617+0000","last_change":"2023-09-23T04:43:51.726705+0000","last_active":"2023-09-23T04:45:24.791617+0000","last_peered":"2023-09-23T04:45:24.791617+0000","last_clean":"2023-09-23T04:45:24.791617+0000","last_became_active":"2023-09-23T04:43:51.726294+0000","last_became_peered":"2023-09-23T04:43:51.726294+0000","last_unstale":"2023-09-23T04:45:24.791617+0000","last_undegraded":"2023-09-23T04:45:24.791617+0000","last_fullsized":"2023-09-23T04:45:24.791617+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:50.694782+0000","last_clean_scrub_stamp":"2023-09-23T04:43:50.694782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:20:01.788868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794471+0000","last_change":"2023-09-23T04:43:46.592219+0000","last_active":"2023-09-23T04:45:25.794471+0000","last_peered":"2023-09-23T04:45:25.794471+0000","last_clean":"2023-09-23T04:45:25.794471+0000","last_became_active":"2023-09-23T04:43:46.591779+0000","last_became_peered":"2023-09-23T04:43:46.591779+0000","last_unstale":"2023-09-23T04:45:25.794471+0000","last_undegraded":"2023-09-23T04:45:25.794471+0000","last_fullsized":"2023-09-23T04:45:25.794471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:45.560591+0000","last_clean_scrub_stamp":"2023-09-23T04:43:45.560591+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:13:34.837138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.791988+0000","last_change":"2023-09-23T04:43:47.581767+0000","last_active":"2023-09-23T04:45:24.791988+0000","last_peered":"2023-09-23T04:45:24.791988+0000","last_clean":"2023-09-23T04:45:24.791988+0000","last_became_active":"2023-09-23T04:43:47.581473+0000","last_became_peered":"2023-09-23T04:43:47.581473+0000","last_unstale":"2023-09-23T04:45:24.791988+0000","last_undegraded":"2023-09-23T04:45:24.791988+0000","last_fullsized":"2023-09-23T04:45:24.791988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:17.289180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.795607+0000","last_change":"2023-09-23T04:43:47.586570+0000","last_active":"2023-09-23T04:45:25.795607+0000","last_peered":"2023-09-23T04:45:25.795607+0000","last_clean":"2023-09-23T04:45:25.795607+0000","last_became_active":"2023-09-23T04:43:47.586234+0000","last_became_peered":"2023-09-23T04:43:47.586234+0000","last_unstale":"2023-09-23T04:45:25.795607+0000","last_undegraded":"2023-09-23T04:45:25.795607+0000","last_fullsized":"2023-09-23T04:45:25.795607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:33.652495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793156+0000","last_change":"2023-09-23T04:43:47.585626+0000","last_active":"2023-09-23T04:45:25.793156+0000","last_peered":"2023-09-23T04:45:25.793156+0000","last_clean":"2023-09-23T04:45:25.793156+0000","last_became_active":"2023-09-23T04:43:47.585320+0000","last_became_peered":"2023-09-23T04:43:47.585320+0000","last_unstale":"2023-09-23T04:45:25.793156+0000","last_undegraded":"2023-09-23T04:45:25.793156+0000","last_fullsized":"2023-09-23T04:45:25.793156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:46:30.446348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.792180+0000","last_change":"2023-09-23T04:43:47.595685+0000","last_active":"2023-09-23T04:45:25.792180+0000","last_peered":"2023-09-23T04:45:25.792180+0000","last_clean":"2023-09-23T04:45:25.792180+0000","last_became_active":"2023-09-23T04:43:47.595426+0000","last_became_peered":"2023-09-23T04:43:47.595426+0000","last_unstale":"2023-09-23T04:45:25.792180+0000","last_undegraded":"2023-09-23T04:45:25.792180+0000","last_fullsized":"2023-09-23T04:45:25.792180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:21:13.722746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.790787+0000","last_change":"2023-09-23T04:43:47.591691+0000","last_active":"2023-09-23T04:45:25.790787+0000","last_peered":"2023-09-23T04:45:25.790787+0000","last_clean":"2023-09-23T04:45:25.790787+0000","last_became_active":"2023-09-23T04:43:47.591362+0000","last_became_peered":"2023-09-23T04:43:47.591362+0000","last_unstale":"2023-09-23T04:45:25.790787+0000","last_undegraded":"2023-09-23T04:45:25.790787+0000","last_fullsized":"2023-09-23T04:45:25.790787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:39.231449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794389+0000","last_change":"2023-09-23T04:43:47.584223+0000","last_active":"2023-09-23T04:45:25.794389+0000","last_peered":"2023-09-23T04:45:25.794389+0000","last_clean":"2023-09-23T04:45:25.794389+0000","last_became_active":"2023-09-23T04:43:47.583834+0000","last_became_peered":"2023-09-23T04:43:47.583834+0000","last_unstale":"2023-09-23T04:45:25.794389+0000","last_undegraded":"2023-09-23T04:45:25.794389+0000","last_fullsized":"2023-09-23T04:45:25.794389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:49.169878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.794787+0000","last_change":"2023-09-23T04:43:47.589641+0000","last_active":"2023-09-23T04:45:25.794787+0000","last_peered":"2023-09-23T04:45:25.794787+0000","last_clean":"2023-09-23T04:45:25.794787+0000","last_became_active":"2023-09-23T04:43:47.589174+0000","last_became_peered":"2023-09-23T04:43:47.589174+0000","last_unstale":"2023-09-23T04:45:25.794787+0000","last_undegraded":"2023-09-23T04:45:25.794787+0000","last_fullsized":"2023-09-23T04:45:25.794787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:45.834989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.793808+0000","last_change":"2023-09-23T04:43:47.591130+0000","last_active":"2023-09-23T04:45:25.793808+0000","last_peered":"2023-09-23T04:45:25.793808+0000","last_clean":"2023-09-23T04:45:25.793808+0000","last_became_active":"2023-09-23T04:43:47.590832+0000","last_became_peered":"2023-09-23T04:43:47.590832+0000","last_unstale":"2023-09-23T04:45:25.793808+0000","last_undegraded":"2023-09-23T04:45:25.793808+0000","last_fullsized":"2023-09-23T04:45:25.793808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:06.981530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-23T04:45:25.947899+0000","last_change":"2023-09-23T04:44:24.812717+0000","last_active":"2023-09-23T04:45:25.947899+0000","last_peered":"2023-09-23T04:45:25.947899+0000","last_clean":"2023-09-23T04:45:25.947899+0000","last_became_active":"2023-09-23T04:44:24.812307+0000","last_became_peered":"2023-09-23T04:44:24.812307+0000","last_unstale":"2023-09-23T04:45:25.947899+0000","last_undegraded":"2023-09-23T04:45:25.947899+0000","last_fullsized":"2023-09-23T04:45:25.947899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:43:34.537044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.796039+0000","last_change":"2023-09-23T04:44:24.999461+0000","last_active":"2023-09-23T04:45:24.796039+0000","last_peered":"2023-09-23T04:45:24.796039+0000","last_clean":"2023-09-23T04:45:24.796039+0000","last_became_active":"2023-09-23T04:44:24.999082+0000","last_became_peered":"2023-09-23T04:44:24.999082+0000","last_unstale":"2023-09-23T04:45:24.796039+0000","last_undegraded":"2023-09-23T04:45:24.796039+0000","last_fullsized":"2023-09-23T04:45:24.796039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:21:20.840200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.797000+0000","last_change":"2023-09-23T04:43:47.597991+0000","last_active":"2023-09-23T04:45:24.797000+0000","last_peered":"2023-09-23T04:45:24.797000+0000","last_clean":"2023-09-23T04:45:24.797000+0000","last_became_active":"2023-09-23T04:43:47.597692+0000","last_became_peered":"2023-09-23T04:43:47.597692+0000","last_unstale":"2023-09-23T04:45:24.797000+0000","last_undegraded":"2023-09-23T04:45:24.797000+0000","last_fullsized":"2023-09-23T04:45:24.797000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:48:11.396171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.794123+0000","last_change":"2023-09-23T04:43:47.604156+0000","last_active":"2023-09-23T04:45:24.794123+0000","last_peered":"2023-09-23T04:45:24.794123+0000","last_clean":"2023-09-23T04:45:24.794123+0000","last_became_active":"2023-09-23T04:43:47.603863+0000","last_became_peered":"2023-09-23T04:43:47.603863+0000","last_unstale":"2023-09-23T04:45:24.794123+0000","last_undegraded":"2023-09-23T04:45:24.794123+0000","last_fullsized":"2023-09-23T04:45:24.794123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:04:34.543729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.787947+0000","last_change":"2023-09-23T04:43:47.594849+0000","last_active":"2023-09-23T04:45:24.787947+0000","last_peered":"2023-09-23T04:45:24.787947+0000","last_clean":"2023-09-23T04:45:24.787947+0000","last_became_active":"2023-09-23T04:43:47.594321+0000","last_became_peered":"2023-09-23T04:43:47.594321+0000","last_unstale":"2023-09-23T04:45:24.787947+0000","last_undegraded":"2023-09-23T04:45:24.787947+0000","last_fullsized":"2023-09-23T04:45:24.787947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T16:42:41.058619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-23T04:45:24.793154+0000","last_change":"2023-09-23T04:43:47.590792+0000","last_active":"2023-09-23T04:45:24.793154+0000","last_peered":"2023-09-23T04:45:24.793154+0000","last_clean":"2023-09-23T04:45:24.793154+0000","last_became_active":"2023-09-23T04:43:47.590207+0000","last_became_peered":"2023-09-23T04:43:47.590207+0000","last_unstale":"2023-09-23T04:45:24.793154+0000","last_undegraded":"2023-09-23T04:45:24.793154+0000","last_fullsized":"2023-09-23T04:45:24.793154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T04:43:46.565888+0000","last_clean_scrub_stamp":"2023-09-23T04:43:46.565888+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:56:13.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":248873,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68806,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1027,"num_read_kb":588,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":509022,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":0.71899999999999997,"15min":0.65100000000000002},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.625999999999999,"5min":10.625999999999999,"15min":10.625999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":0.748},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92500000000000004,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":1.238,"15min":1.238},"last":0.78100000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80400000000000005,"5min":1.125,"15min":1.125},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.753}]},{"osd":6,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":0.80300000000000005,"15min":0.73099999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":10.571999999999999,"5min":10.571999999999999,"15min":10.571999999999999},"last":0.999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":10.398999999999999,"5min":10.398999999999999,"15min":10.398999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96999999999999997,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.54000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.92700000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.95399999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71699999999999997,"15min":0.72499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86199999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68400000000000005,"15min":0.68200000000000005},"min":{"1min":0.48999999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.88300000000000001}]},{"osd":9,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":0.88900000000000001,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":9.734,"5min":9.734,"15min":9.734},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.278,"5min":0.89000000000000001,"15min":0.82599999999999996},"min":{"1min":0.64000000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":10.288,"5min":10.288,"15min":10.288},"last":0.97899999999999998}]},{"osd":10,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98599999999999999,"5min":1.272,"15min":1.272},"last":1.024},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94099999999999995,"5min":1.71,"15min":1.71},"last":0.94099999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1157081,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86599999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.85699999999999998,"15min":0.90700000000000003},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.442,"15min":0.44800000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54500000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.62,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.61199999999999999,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.67800000000000005,"15min":0.71299999999999997},"min":{"1min":0.38200000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.77900000000000003,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.77500000000000002,"15min":0.78300000000000003},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":3.8220000000000001,"15min":3.8220000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.75,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":8.0960000000000001,"15min":8.0960000000000001},"last":0.497}]},{"osd":4,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.316,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.88,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.752,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.036,"5min":5.5289999999999999,"15min":5.5289999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.51500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Sep 23 04:43:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.41999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93700000000000006,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94699999999999995,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.92400000000000004,"15min":0.94299999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0269999999999999,"15min":1.0029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":5.2880000000000003,"5min":5.3760000000000003,"15min":5.3760000000000003},"last":0.77300000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78800000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":512438,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.62,"15min":0.59899999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":6.4669999999999996,"15min":6.4669999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88,"5min":6.6310000000000002,"15min":6.6310000000000002},"last":0.53700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94899999999999995,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89100000000000001,"5min":1.778,"15min":1.778},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.79600000000000004,"15min":0.81599999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84899999999999998,"15min":0.76000000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3080000000000001,"5min":13.364000000000001,"15min":13.364000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.78800000000000003,"15min":0.748},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73399999999999999,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.82199999999999995},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1180000000000001,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90400000000000003,"15min":0.88700000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.8690000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.88100000000000001,"15min":0.90200000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0029999999999999,"5min":1.913,"15min":1.913},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.94699999999999995,"15min":0.96699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.236,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.93000000000000005,"15min":0.95799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0469999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.046,"15min":1.012},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":7.0350000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.155,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.97099999999999997,"15min":0.98199999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.272,"5min":3.8330000000000002,"15min":3.8330000000000002},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.95299999999999996,"15min":0.98799999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.141,"5min":1.877,"15min":1.877},"last":0.623}]},{"osd":11,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89500000000000002,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.036,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.748}]}]},{"osd":8,"up_from":58,"seq":249108103219,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508698,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.64900000000000002,"15min":0.61899999999999999},"min":{"1min":0.35799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":5.5069999999999997,"5min":5.5069999999999997,"15min":5.5069999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65000000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.456,"5min":3.456,"15min":3.456},"last":0.86199999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47199999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.45600000000000002,"15min":0.438},"min":{"1min":0.40200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64600000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.72099999999999997,"15min":0.68899999999999995},"min":{"1min":0.59399999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65800000000000003,"15min":0.64100000000000001},"min":{"1min":0.55700000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.57099999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.55900000000000005,"15min":0.54100000000000004},"min":{"1min":0.51000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.94599999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78600000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.75,"15min":0.71799999999999997},"min":{"1min":0.68100000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.60399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.81699999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56699999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.284,"15min":1.284},"last":0.86799999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":0.82499999999999996,"15min":0.77300000000000002},"min":{"1min":0.70499999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.85099999999999998,"15min":0.82699999999999996},"min":{"1min":0.72699999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.72999999999999998},"min":{"1min":0.60299999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.042,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.80900000000000005,"15min":0.77300000000000002},"min":{"1min":0.626,"5min":0.495,"15min":0.495},"max":{"1min":1.042,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.83599999999999997},"min":{"1min":0.56899999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.98299999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.88300000000000001,"15min":0.89000000000000001},"min":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.997,"5min":1.089,"15min":1.089},"last":0.80500000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1105343,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60799999999999998,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.432,"15min":0.42299999999999999},"min":{"1min":0.32300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67000000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.46200000000000002,"15min":0.44600000000000001},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86199999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61499999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90200000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.72599999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52600000000000002,"15min":0.54100000000000004},"min":{"1min":0.34499999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46200000000000002}]},{"osd":5,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.66700000000000004},"min":{"1min":0.45800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86499999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97099999999999997,"5min":1,"15min":1},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.46700000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.5570000000000004,"5min":6.5570000000000004,"15min":6.5570000000000004},"last":1.131}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89200000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74299999999999999}]},{"osd":9,"last update":"Sat Sep 23 04:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.753,"15min":0.747},"min":{"1min":0.54200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.628},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.745},"min":{"1min":0.63600000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.98399999999999999,"5min":1.125,"15min":1.125},"last":0.755}]},{"osd":10,"last update":"Sat Sep 23 04:45:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56599999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.68999999999999995,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72999999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83499999999999996,"15min":0.83699999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71799999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":503846,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.59999999999999998,"15min":0.51900000000000002},"min":{"1min":0.35099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84599999999999997,"5min":8.2750000000000004,"15min":8.2750000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.53300000000000003},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0880000000000001,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76100000000000001,"15min":0.71299999999999997},"min":{"1min":0.498,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":4.2480000000000002,"15min":4.2480000000000002},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85199999999999998,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71299999999999997,"15min":0.68000000000000005},"min":{"1min":0.55700000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85899999999999999,"5min":2.9460000000000002,"15min":2.9460000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78700000000000003,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":4.3819999999999997,"15min":4.3819999999999997},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.73599999999999999,"15min":0.69299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0580000000000001,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.874,"5min":2.218,"15min":2.218},"last":0.436}]},{"osd":4,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96699999999999997,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83799999999999997,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.99199999999999999,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.76000000000000001}]},{"osd":5,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96799999999999997,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82099999999999995,"15min":0.80000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98199999999999998,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Sat Sep 23 04:45:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.079,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.871,"15min":0.82299999999999995},"min":{"1min":0.749,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0820000000000001,"5min":4.3029999999999999,"15min":4.3029999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Sat Sep 23 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78800000000000003,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.92700000000000005,"15min":0.93500000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.056,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.90500000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.093,"15min":0.98299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0409999999999999,"5min":15.34,"15min":15.34},"last":0.877},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.94799999999999995,"15min":0.93200000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.99099999999999999,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.82399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.242,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.0069999999999999,"15min":1.0089999999999999},"min":{"1min":0.80700000000000005,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1259999999999999,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.167,"15min":1.1970000000000001},"min":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.2230000000000001,"5min":6.8470000000000004,"15min":6.8470000000000004},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":1.2030000000000001,"15min":1.2490000000000001},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0920000000000001,"5min":8.3249999999999993,"15min":8.3249999999999993},"last":0.86099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":565609,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.55400000000000005,"15min":0.57199999999999995},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.31},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.44800000000000001,"5min":0.34000000000000002,"15min":0.33400000000000002},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.54400000000000004},"min":{"1min":0.47799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.56799999999999995},"min":{"1min":0.58799999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.63800000000000001,"15min":0.58999999999999997},"min":{"1min":0.53000000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.497,"15min":0.45400000000000001},"min":{"1min":0.375,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66800000000000004,"15min":0.61299999999999999},"min":{"1min":0.47299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":1.087,"15min":1.087},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68400000000000005,"15min":0.63300000000000001},"min":{"1min":0.66100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76800000000000002}]},{"osd":6,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.441,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.70999999999999996,"15min":0.67000000000000004},"min":{"1min":0.57999999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80400000000000005}]},{"osd":7,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.59599999999999997,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.65500000000000003,"15min":0.59899999999999998},"min":{"1min":0.50600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":8,"last update":"Sat Sep 23 04:45:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.65700000000000003},"min":{"1min":0.46300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.72299999999999998,"15min":0.68200000000000005},"min":{"1min":0.621,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.89900000000000002}]},{"osd":9,"last update":"Sat Sep 23 04:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.78400000000000003,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.84499999999999997,"15min":0.81399999999999995},"min":{"1min":0.66600000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92000000000000004}]},{"osd":10,"last update":"Sat Sep 23 04:45:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.64700000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.03,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.875},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.026,"5min":1.232,"15min":1.232},"last":0.86499999999999999}]},{"osd":11,"last update":"Sat Sep 23 04:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.73599999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.0609999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.83699999999999997,"15min":0.82899999999999996},"min":{"1min":0.71099999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":505302,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 04:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.64600000000000002,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89500000000000002,"5min":1.345,"15min":1.345},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.50900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.113,"5min":1.306,"15min":1.306},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71899999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68100000000000005,"15min":0.68200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.012,"15min":1.012},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Sep 23 04:45:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68700000000000006,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.625},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66600000000000004,"15min":0.57999999999999996},"min":{"1min":0.51500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67200000000000004}]},{"osd":5,"last update":"Sat Sep 23 04:44:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.71999999999999997,"15min":0.66800000000000004},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97799999999999998,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.751,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88500000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Sep 23 04:45:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.02,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.746}]},{"osd":8,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.626,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91100000000000003,"15min":0.96099999999999997},"min":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1060000000000001,"5min":1.871,"15min":1.871},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.91200000000000003,"15min":0.93400000000000005},"min":{"1min":0.69399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.877}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.88700000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.3,"15min":1.3},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86199999999999999,"15min":0.85299999999999998},"min":{"1min":0.70799999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Sep 23 04:45:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.88300000000000001,"15min":0.872},"min":{"1min":0.76800000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.78600000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":566521,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":0.71799999999999997,"15min":0.622},"min":{"1min":0.376,"5min":0.309,"15min":0.309},"max":{"1min":14.076000000000001,"5min":14.076000000000001,"15min":14.076000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60699999999999998,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.998,"15min":0.998},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 23 04:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60499999999999998,"15min":0.57099999999999995},"min":{"1min":0.54900000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91800000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86799999999999999}]},{"osd":4,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.48499999999999999,"15min":0.44900000000000001},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.52100000000000002,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Sep 23 04:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.78700000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82299999999999995,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.90700000000000003}]},{"osd":7,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72899999999999998,"15min":0.72399999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Sep 23 04:44:58 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.46899999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.745}]},{"osd":9,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":0.89700000000000002,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":14.143000000000001,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.88}]},{"osd":10,"last update":"Sat Sep 23 04:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72899999999999998,"15min":0.70799999999999996},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85699999999999998}]},{"osd":11,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.499,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79600000000000004,"15min":0.79500000000000004},"min":{"1min":0.52400000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.94399999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":503223,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.50800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32000000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":6.0179999999999998,"5min":6.0179999999999998,"15min":6.0179999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.501,"15min":0.42699999999999999},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.59599999999999997},"min":{"1min":0.50600000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89500000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.51100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.60099999999999998},"min":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.99099999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.60599999999999998},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:45:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.66500000000000004,"15min":0.61699999999999999},"min":{"1min":0.56299999999999994,"5min":0.443,"15min":0.443},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.70199999999999996,"15min":0.64600000000000002},"min":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Sat Sep 23 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89400000000000002,"5min":0.999,"15min":0.999},"last":0.97999999999999998}]},{"osd":6,"last update":"Sat Sep 23 04:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.38,"15min":0.38},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.76000000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.873},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.59699999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72999999999999998}]},{"osd":8,"last update":"Sat Sep 23 04:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.98099999999999998,"15min":0.93300000000000005},"min":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":10.496,"5min":10.496,"15min":10.496},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.359,"5min":0.98499999999999999,"15min":0.94299999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":12.574999999999999,"5min":12.574999999999999,"15min":12.574999999999999},"last":0.76900000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.97199999999999998,"5min":1.569,"15min":1.569},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.89600000000000002}]},{"osd":11,"last update":"Sat Sep 23 04:45:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.90400000000000003,"15min":0.91600000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.85499999999999998,"15min":0.86299999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.95599999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":508879,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.501,"15min":0.48799999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.34699999999999998},"min":{"1min":0.309,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.59299999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.439,"5min":0.41499999999999998,"15min":0.377},"min":{"1min":0.28399999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.622,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.54600000000000004},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.749,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Sep 23 04:45:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73099999999999998}]},{"osd":5,"last update":"Sat Sep 23 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.55300000000000005},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.86399999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66700000000000004,"15min":0.58299999999999996},"min":{"1min":0.51300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.879}]},{"osd":6,"last update":"Sat Sep 23 04:45:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65900000000000003,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73399999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78600000000000003,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63100000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.622,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96599999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69299999999999995}]},{"osd":10,"last update":"Sat Sep 23 04:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95199999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78900000000000003}]},{"osd":11,"last update":"Sat Sep 23 04:45:49 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70799999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.80300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1100939,"data_compressed":59934,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57099999999999995,"15min":0.52000000000000002},"min":{"1min":0.41599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82499999999999996,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.441},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54100000000000004,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70399999999999996,"5min":1.246,"15min":1.246},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79800000000000004,"5min":1.115,"15min":1.115},"last":0.503},{"interface":"front","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.54200000000000004},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":1.071,"15min":1.071},"last":0.65200000000000002}]},{"osd":2,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.377},"min":{"1min":0.314,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61599999999999999,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.439,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.72899999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59699999999999998,"15min":0.54900000000000004},"min":{"1min":0.438,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.212,"15min":1.212},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.45900000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94299999999999995,"5min":1.042,"15min":1.042},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.54700000000000004},"min":{"1min":0.40699999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.88200000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84399999999999997,"5min":1.23,"15min":1.23},"last":0.56899999999999995}]},{"osd":6,"last update":"Sat Sep 23 04:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.53000000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0720000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.625},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.90900000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Sep 23 04:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81999999999999995,"5min":1.292,"15min":1.292},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Sat Sep 23 04:46:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":1.105,"15min":1.105},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89600000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66300000000000003}]},{"osd":9,"last update":"Sat Sep 23 04:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.78100000000000003,"15min":0.78500000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.77400000000000002}]},{"osd":10,"last update":"Sat Sep 23 04:45:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.60399999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.752},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91000000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34615,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36257,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T04:46:02.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:46:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:02 smithi069 ceph-mon[133931]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 -- 172.21.15.5:0/1855512632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 msgr2=0x7f93b81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1855512632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81024f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009990 tx=0x7f93ac009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 -- 172.21.15.5:0/1855512632 shutdown_connections 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1855512632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8102a30 0x7f93b8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1855512632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8100e90 0x7f93b81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1855512632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81024f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 -- 172.21.15.5:0/1855512632 >> 172.21.15.5:0/1855512632 conn(0x7f93b80fc450 msgr2=0x7f93b80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 -- 172.21.15.5:0/1855512632 shutdown_connections 2023-09-23T04:46:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.192+0000 7f93be3ff700 1 -- 172.21.15.5:0/1855512632 wait complete. 2023-09-23T04:46:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.193+0000 7f93be3ff700 1 Processor -- start 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.193+0000 7f93be3ff700 1 -- start start 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.193+0000 7f93be3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 0x7f93b8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93be3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93be3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8102a30 0x7f93b81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93be3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81a09f0 con 0x7f93b8102070 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93be3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81a0b30 con 0x7f93b8100e90 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93be3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81a0cc0 con 0x7f93b8102a30 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 0x7f93b8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41052/0 (socket says 172.21.15.5:41052) 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 -- 172.21.15.5:0/1777376826 learned_addr learned my addr 172.21.15.5:0/1777376826 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93bc99c700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8102a30 0x7f93b81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 -- 172.21.15.5:0/1777376826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8102a30 msgr2=0x7f93b81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8102a30 0x7f93b81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 -- 172.21.15.5:0/1777376826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 msgr2=0x7f93b8198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 0x7f93b8198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b77fe700 1 -- 172.21.15.5:0/1777376826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0095f0 con 0x7f93b8102070 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.194+0000 7f93b7fff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 0x7f93b8198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93b77fe700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f93ac004710 tx=0x7f93ac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac01d070 con 0x7f93b8102070 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac021410 con 0x7f93b8102070 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac017650 con 0x7f93b8102070 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b81a0f40 con 0x7f93b8102070 2023-09-23T04:46:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.195+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81a12b0 con 0x7f93b8102070 2023-09-23T04:46:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.196+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b810c750 con 0x7f93b8102070 2023-09-23T04:46:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.198+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f93ac021580 con 0x7f93b8102070 2023-09-23T04:46:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.199+0000 7f93b57fa700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 0x7f93a00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.199+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f93ac0ce7d0 con 0x7f93b8102070 2023-09-23T04:46:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.200+0000 7f93b7fff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 0x7f93a00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.200+0000 7f93b7fff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 0x7f93a00a3ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93b8101be0 tx=0x7f93a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.202+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f93ac097a00 con 0x7f93b8102070 2023-09-23T04:46:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.417+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93b81a1440 con 0x7f93b8102070 2023-09-23T04:46:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.419+0000 7f93b57fa700 1 -- 172.21.15.5:0/1777376826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7f93ac09b020 con 0x7f93b8102070 2023-09-23T04:46:03.420 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:03.421 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 msgr2=0x7f93a00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 0x7f93a00a3ae0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93b8101be0 tx=0x7f93a8009260 comp rx=0 tx=0).stop 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 msgr2=0x7f93b81993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f93ac004710 tx=0x7f93ac009580 comp rx=0 tx=0).stop 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 shutdown_connections 2023-09-23T04:46:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f93a00a1620 0x7f93a00a3ae0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f93b8102a30 0x7f93b81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93b8100e90 0x7f93b8198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 --2- 172.21.15.5:0/1777376826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8102070 0x7f93b81993b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 >> 172.21.15.5:0/1777376826 conn(0x7f93b80fc450 msgr2=0x7f93b80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 shutdown_connections 2023-09-23T04:46:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.422+0000 7f93be3ff700 1 -- 172.21.15.5:0/1777376826 wait complete. 2023-09-23T04:46:03.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T04:46:03.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.652+0000 7fd94adff700 1 -- 172.21.15.5:0/3500458182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 msgr2=0x7fd944102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.652+0000 7fd94adff700 1 --2- 172.21.15.5:0/3500458182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944102510 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd930009990 tx=0x7fd930009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 -- 172.21.15.5:0/3500458182 shutdown_connections 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 --2- 172.21.15.5:0/3500458182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944102a50 0x7fd944109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 --2- 172.21.15.5:0/3500458182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944100eb0 0x7fd9441012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 --2- 172.21.15.5:0/3500458182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944102510 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 -- 172.21.15.5:0/3500458182 >> 172.21.15.5:0/3500458182 conn(0x7fd9440fc470 msgr2=0x7fd9440fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 -- 172.21.15.5:0/3500458182 shutdown_connections 2023-09-23T04:46:03.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.653+0000 7fd94adff700 1 -- 172.21.15.5:0/3500458182 wait complete. 2023-09-23T04:46:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 Processor -- start 2023-09-23T04:46:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 -- start start 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944100eb0 0x7fd944196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944102a50 0x7fd94419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419e770 con 0x7fd944102090 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419e8e0 con 0x7fd944100eb0 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd94adff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419eaa0 con 0x7fd944102a50 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41066/0 (socket says 172.21.15.5:41066) 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd943fff700 1 -- 172.21.15.5:0/83378977 learned_addr learned my addr 172.21.15.5:0/83378977 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.654+0000 7fd948b9b700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944100eb0 0x7fd944196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd94939c700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944102a50 0x7fd94419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 -- 172.21.15.5:0/83378977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944102a50 msgr2=0x7fd94419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944102a50 0x7fd94419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 -- 172.21.15.5:0/83378977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944100eb0 msgr2=0x7fd944196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944100eb0 0x7fd944196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 -- 172.21.15.5:0/83378977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd938009520 con 0x7fd944102090 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd943fff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd93000b680 tx=0x7fd930005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93001d070 con 0x7fd944102090 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9300095f0 con 0x7fd944102090 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd930021410 con 0x7fd944102090 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.655+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930017650 con 0x7fd944102090 2023-09-23T04:46:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.656+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94419f010 con 0x7fd944102090 2023-09-23T04:46:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.657+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd944107eb0 con 0x7fd944102090 2023-09-23T04:46:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.659+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fd930021580 con 0x7fd944102090 2023-09-23T04:46:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.660+0000 7fd941ffb700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 0x7fd9340a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.660+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fd9300ceee0 con 0x7fd944102090 2023-09-23T04:46:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.660+0000 7fd948b9b700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 0x7fd9340a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.661+0000 7fd948b9b700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 0x7fd9340a3a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd944198100 tx=0x7fd938009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.662+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd930098110 con 0x7fd944102090 2023-09-23T04:46:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.872+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd94402d160 con 0x7fd944102090 2023-09-23T04:46:03.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.873+0000 7fd941ffb700 1 -- 172.21.15.5:0/83378977 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd930021860 con 0x7fd944102090 2023-09-23T04:46:03.873 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T04:46:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.875+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 msgr2=0x7fd9340a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.875+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 0x7fd9340a3a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd944198100 tx=0x7fd938009260 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.875+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 msgr2=0x7fd944197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.875+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd93000b680 tx=0x7fd930005740 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.875+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 shutdown_connections 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fd9340a15a0 0x7fd9340a3a60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd944102a50 0x7fd94419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd944100eb0 0x7fd944196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 --2- 172.21.15.5:0/83378977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd944102090 0x7fd944197170 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 >> 172.21.15.5:0/83378977 conn(0x7fd9440fc470 msgr2=0x7fd9440fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:03.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 shutdown_connections 2023-09-23T04:46:03.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:03.876+0000 7fd94adff700 1 -- 172.21.15.5:0/83378977 wait complete. 2023-09-23T04:46:03.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:03 smithi005 ceph-mon[97638]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:03 smithi005 ceph-mon[97638]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:03 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1777376826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:03.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T04:46:03.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:03 smithi118 ceph-mon[134297]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:03 smithi118 ceph-mon[134297]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:03 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1777376826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:03 smithi069 ceph-mon[133931]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:03 smithi069 ceph-mon[133931]: from='client.25478 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:03 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1777376826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:04.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 -- 172.21.15.5:0/1450136582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44073880 msgr2=0x7f3d44073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/1450136582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44073880 0x7f3d44073d00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009990 tx=0x7f3d34009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 -- 172.21.15.5:0/1450136582 shutdown_connections 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/1450136582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44108dc0 0x7f3d4410b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/1450136582 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 0x7f3d440732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/1450136582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44073880 0x7f3d44073d00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.105+0000 7f3d4bfff700 1 -- 172.21.15.5:0/1450136582 >> 172.21.15.5:0/1450136582 conn(0x7f3d440fc470 msgr2=0x7f3d440fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.106+0000 7f3d4bfff700 1 -- 172.21.15.5:0/1450136582 shutdown_connections 2023-09-23T04:46:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.106+0000 7f3d4bfff700 1 -- 172.21.15.5:0/1450136582 wait complete. 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 Processor -- start 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 -- start start 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 0x7f3d44199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44108dc0 0x7f3d441a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441a09c0 con 0x7f3d44108dc0 2023-09-23T04:46:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441a0b00 con 0x7f3d44073880 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441a0c90 con 0x7f3d44074e60 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d49d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d49d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59830/0 (socket says 172.21.15.5:59830) 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d49d9b700 1 -- 172.21.15.5:0/2144540346 learned_addr learned my addr 172.21.15.5:0/2144540346 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4a59c700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44108dc0 0x7f3d441a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.107+0000 7f3d4959a700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 0x7f3d44199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 -- 172.21.15.5:0/2144540346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 msgr2=0x7f3d44199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 0x7f3d44199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 -- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44108dc0 msgr2=0x7f3d441a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44108dc0 0x7f3d441a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 -- 172.21.15.5:0/2144540346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009c40 con 0x7f3d44073880 2023-09-23T04:46:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d49d9b700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000e9a0 tx=0x7f3d4000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40009790 con 0x7f3d44073880 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d340095f0 con 0x7f3d44073880 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d4000ce80 con 0x7f3d44073880 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40018680 con 0x7f3d44073880 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.108+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d441a1210 con 0x7f3d44073880 2023-09-23T04:46:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.109+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d44107470 con 0x7f3d44073880 2023-09-23T04:46:04.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.112+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f3d40010430 con 0x7f3d44073880 2023-09-23T04:46:04.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.113+0000 7f3d3affd700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 0x7f3d300a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.113+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f3d40014070 con 0x7f3d44073880 2023-09-23T04:46:04.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.113+0000 7f3d4959a700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 0x7f3d300a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.114+0000 7f3d4959a700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 0x7f3d300a3a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400b2e0 tx=0x7f3d34005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:04.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.115+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f3d40096a40 con 0x7f3d44073880 2023-09-23T04:46:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.328+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d4410ead0 con 0x7f3d44073880 2023-09-23T04:46:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.329+0000 7f3d3affd700 1 -- 172.21.15.5:0/2144540346 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d4009a060 con 0x7f3d44073880 2023-09-23T04:46:04.329 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:46:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 msgr2=0x7f3d300a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 0x7f3d300a3a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400b2e0 tx=0x7f3d34005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 msgr2=0x7f3d44198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000e9a0 tx=0x7f3d4000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 shutdown_connections 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f3d300a15a0 0x7f3d300a3a60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44108dc0 0x7f3d441a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d44073880 0x7f3d44198e40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 --2- 172.21.15.5:0/2144540346 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d44074e60 0x7f3d44199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.331+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 >> 172.21.15.5:0/2144540346 conn(0x7f3d440fc470 msgr2=0x7f3d440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.332+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 shutdown_connections 2023-09-23T04:46:04.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.332+0000 7f3d4bfff700 1 -- 172.21.15.5:0/2144540346 wait complete. 2023-09-23T04:46:04.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T04:46:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.557+0000 7f80c07ff700 1 -- 172.21.15.5:0/2533511885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8102a50 msgr2=0x7f80b8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.557+0000 7f80c07ff700 1 --2- 172.21.15.5:0/2533511885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8102a50 0x7f80b8109370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b0a0 tx=0x7f80b000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.557+0000 7f80c07ff700 1 -- 172.21.15.5:0/2533511885 shutdown_connections 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.557+0000 7f80c07ff700 1 --2- 172.21.15.5:0/2533511885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8102a50 0x7f80b8109370 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.558+0000 7f80c07ff700 1 --2- 172.21.15.5:0/2533511885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8100eb0 0x7f80b81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.558+0000 7f80c07ff700 1 --2- 172.21.15.5:0/2533511885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102090 0x7f80b8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.558+0000 7f80c07ff700 1 -- 172.21.15.5:0/2533511885 >> 172.21.15.5:0/2533511885 conn(0x7f80b80fc470 msgr2=0x7f80b80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.558+0000 7f80c07ff700 1 -- 172.21.15.5:0/2533511885 shutdown_connections 2023-09-23T04:46:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.558+0000 7f80c07ff700 1 -- 172.21.15.5:0/2533511885 wait complete. 2023-09-23T04:46:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 Processor -- start 2023-09-23T04:46:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 -- start start 2023-09-23T04:46:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 0x7f80b8196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 0x7f80b819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819e880 con 0x7f80b8102090 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819e9f0 con 0x7f80b8102a50 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80c07ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819ebb0 con 0x7f80b8100eb0 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80bdd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80be59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 0x7f80b8196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.559+0000 7f80bdd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41086/0 (socket says 172.21.15.5:41086) 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 -- 172.21.15.5:0/1752971869 learned_addr learned my addr 172.21.15.5:0/1752971869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bed9c700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 0x7f80b819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 -- 172.21.15.5:0/1752971869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 msgr2=0x7f80b8196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 0x7f80b8196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 -- 172.21.15.5:0/1752971869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 msgr2=0x7f80b819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 0x7f80b819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 -- 172.21.15.5:0/1752971869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009d00 con 0x7f80b8102090 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80be59b700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 0x7f80b8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bed9c700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 0x7f80b819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80bdd9a700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f80a800e970 tx=0x7f80a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:04.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.560+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8009790 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.561+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b819ee90 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.561+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a800c7e0 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.561+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8003c70 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.561+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b819f2b0 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.562+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8108470 con 0x7f80b8102090 2023-09-23T04:46:04.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.564+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f80a800cc50 con 0x7f80b8102090 2023-09-23T04:46:04.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.565+0000 7f80af7fe700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 0x7f80a40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.565+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f80a8014070 con 0x7f80b8102090 2023-09-23T04:46:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.565+0000 7f80be59b700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 0x7f80a40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.566+0000 7f80be59b700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 0x7f80a40a3a60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80b8198210 tx=0x7f80b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.567+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f80a80967e0 con 0x7f80b8102090 2023-09-23T04:46:04.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:04 smithi005 ceph-mon[97638]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:04 smithi005 ceph-mon[97638]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-23T04:46:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:04 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/83378977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:04 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2144540346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.779+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80b810c9e0 con 0x7f80b8102090 2023-09-23T04:46:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.779+0000 7f80af7fe700 1 -- 172.21.15.5:0/1752971869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80a8003dd0 con 0x7f80b8102090 2023-09-23T04:46:04.780 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T04:46:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 msgr2=0x7f80a40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 0x7f80a40a3a60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80b8198210 tx=0x7f80b4009260 comp rx=0 tx=0).stop 2023-09-23T04:46:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 msgr2=0x7f80b8197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f80a800e970 tx=0x7f80a800ec80 comp rx=0 tx=0).stop 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 shutdown_connections 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f80a40a15a0 0x7f80a40a3a60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80b8102a50 0x7f80b819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80b8100eb0 0x7f80b8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 --2- 172.21.15.5:0/1752971869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b8102090 0x7f80b8197280 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 >> 172.21.15.5:0/1752971869 conn(0x7f80b80fc470 msgr2=0x7f80b81055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:04.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 shutdown_connections 2023-09-23T04:46:04.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:04.782+0000 7f80c07ff700 1 -- 172.21.15.5:0/1752971869 wait complete. 2023-09-23T04:46:04.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T04:46:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:04 smithi118 ceph-mon[134297]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:04 smithi118 ceph-mon[134297]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-23T04:46:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/83378977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:04.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:04 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2144540346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:04 smithi069 ceph-mon[133931]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-23T04:46:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:04 smithi069 ceph-mon[133931]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-23T04:46:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:04 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/83378977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T04:46:04.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:04 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2144540346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 -- 172.21.15.5:0/1377995739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8100f10 msgr2=0x7fbde8101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1377995739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8100f10 0x7fbde8101330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009990 tx=0x7fbdd8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 -- 172.21.15.5:0/1377995739 shutdown_connections 2023-09-23T04:46:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1377995739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 0x7fbde8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1377995739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8100f10 0x7fbde8101330 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1377995739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8102060 0x7fbde81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 -- 172.21.15.5:0/1377995739 >> 172.21.15.5:0/1377995739 conn(0x7fbde80fc480 msgr2=0x7fbde80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:05.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 -- 172.21.15.5:0/1377995739 shutdown_connections 2023-09-23T04:46:05.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.013+0000 7fbded5ff700 1 -- 172.21.15.5:0/1377995739 wait complete. 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.014+0000 7fbded5ff700 1 Processor -- start 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.014+0000 7fbded5ff700 1 -- start start 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8100f10 0x7fbde8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 0x7fbde819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde819e760 con 0x7fbde8102060 2023-09-23T04:46:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde819e8d0 con 0x7fbde8102a20 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbded5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde819ea90 con 0x7fbde8100f10 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41108/0 (socket says 172.21.15.5:41108) 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 -- 172.21.15.5:0/1181180463 learned_addr learned my addr 172.21.15.5:0/1181180463 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde77fe700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 0x7fbde819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde6ffd700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8100f10 0x7fbde8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 -- 172.21.15.5:0/1181180463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8100f10 msgr2=0x7fbde8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8100f10 0x7fbde8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 -- 172.21.15.5:0/1181180463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 msgr2=0x7fbde819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 0x7fbde819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.015+0000 7fbde67fc700 1 -- 172.21.15.5:0/1181180463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc009520 con 0x7fbde8102060 2023-09-23T04:46:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbde67fc700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00eb30 tx=0x7fbddc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc009880 con 0x7fbde8102060 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbddc0044e0 con 0x7fbde8102060 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc003d80 con 0x7fbde8102060 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80095f0 con 0x7fbde8102060 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.016+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde819f0f0 con 0x7fbde8102060 2023-09-23T04:46:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.017+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde810c760 con 0x7fbde8102060 2023-09-23T04:46:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.019+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fbddc003f20 con 0x7fbde8102060 2023-09-23T04:46:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.021+0000 7fbdcffff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 0x7fbdd00a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.021+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fbddc014070 con 0x7fbde8102060 2023-09-23T04:46:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.022+0000 7fbde6ffd700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 0x7fbdd00a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.022+0000 7fbde6ffd700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 0x7fbdd00a3ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b2e0 tx=0x7fbdd8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.025+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbddc0969b0 con 0x7fbde8102060 2023-09-23T04:46:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.242+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbde8107de0 con 0x7fbde8102060 2023-09-23T04:46:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.243+0000 7fbdcffff700 1 -- 172.21.15.5:0/1181180463 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbddc099fd0 con 0x7fbde8102060 2023-09-23T04:46:05.243 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T04:46:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 msgr2=0x7fbdd00a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 0x7fbdd00a3ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b2e0 tx=0x7fbdd8005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 msgr2=0x7fbde8197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00eb30 tx=0x7fbddc00ee40 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 shutdown_connections 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fbdd00a1620 0x7fbdd00a3ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbde8102a20 0x7fbde819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbde8100f10 0x7fbde8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 --2- 172.21.15.5:0/1181180463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde8102060 0x7fbde8197160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.245+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 >> 172.21.15.5:0/1181180463 conn(0x7fbde80fc480 msgr2=0x7fbde80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.246+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 shutdown_connections 2023-09-23T04:46:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.246+0000 7fbded5ff700 1 -- 172.21.15.5:0/1181180463 wait complete. 2023-09-23T04:46:05.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:46:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 -- 172.21.15.5:0/3397032832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c108db0 msgr2=0x7fe21c10b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 --2- 172.21.15.5:0/3397032832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c108db0 0x7fe21c10b240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe21400b0a0 tx=0x7fe21400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 -- 172.21.15.5:0/3397032832 shutdown_connections 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 --2- 172.21.15.5:0/3397032832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c108db0 0x7fe21c10b240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 --2- 172.21.15.5:0/3397032832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c074e60 0x7fe21c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 --2- 172.21.15.5:0/3397032832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.476+0000 7fe220fff700 1 -- 172.21.15.5:0/3397032832 >> 172.21.15.5:0/3397032832 conn(0x7fe21c0fc480 msgr2=0x7fe21c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.477+0000 7fe220fff700 1 -- 172.21.15.5:0/3397032832 shutdown_connections 2023-09-23T04:46:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.477+0000 7fe220fff700 1 -- 172.21.15.5:0/3397032832 wait complete. 2023-09-23T04:46:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.477+0000 7fe220fff700 1 Processor -- start 2023-09-23T04:46:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 -- start start 2023-09-23T04:46:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c074e60 0x7fe21c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c108db0 0x7fe21c1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a0a60 con 0x7fe21c074e60 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a0ba0 con 0x7fe21c073880 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe220fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1a0d30 con 0x7fe21c108db0 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe21a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe21a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59874/0 (socket says 172.21.15.5:59874) 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe21a59c700 1 -- 172.21.15.5:0/3379509832 learned_addr learned my addr 172.21.15.5:0/3379509832 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.478+0000 7fe219d9b700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c074e60 0x7fe21c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21ad9d700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c108db0 0x7fe21c1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 -- 172.21.15.5:0/3379509832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c108db0 msgr2=0x7fe21c1a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c108db0 0x7fe21c1a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 -- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c074e60 msgr2=0x7fe21c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c074e60 0x7fe21c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 -- 172.21.15.5:0/3379509832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe214009d00 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe21a59c700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00e9a0 tx=0x7fe20c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c009790 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20c00cbd0 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c1a0fb0 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c018680 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.479+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c1a1430 con 0x7fe21c073880 2023-09-23T04:46:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.480+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c107460 con 0x7fe21c073880 2023-09-23T04:46:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.482+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7fe20c00cd40 con 0x7fe21c073880 2023-09-23T04:46:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.484+0000 7fe20b7fe700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 0x7fe2040a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.484+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fe20c014070 con 0x7fe21c073880 2023-09-23T04:46:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.484+0000 7fe219d9b700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 0x7fe2040a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.485+0000 7fe219d9b700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 0x7fe2040a3a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe21c074b80 tx=0x7fe210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.486+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe20c0967b0 con 0x7fe21c073880 2023-09-23T04:46:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.756+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe21c10ebc0 con 0x7fe21c073880 2023-09-23T04:46:05.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.757+0000 7fe20b7fe700 1 -- 172.21.15.5:0/3379509832 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7fe20c099dd0 con 0x7fe21c073880 2023-09-23T04:46:05.757 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:05.757 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 msgr2=0x7fe2040a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 0x7fe2040a3a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe21c074b80 tx=0x7fe210009260 comp rx=0 tx=0).stop 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 msgr2=0x7fe21c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00e9a0 tx=0x7fe20c00ed60 comp rx=0 tx=0).stop 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 shutdown_connections 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fe2040a15a0 0x7fe2040a3a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe21c108db0 0x7fe21c1a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe21c073880 0x7fe21c198ee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 --2- 172.21.15.5:0/3379509832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21c074e60 0x7fe21c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.760+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 >> 172.21.15.5:0/3379509832 conn(0x7fe21c0fc480 msgr2=0x7fe21c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.761+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 shutdown_connections 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:05.761+0000 7fe220fff700 1 -- 172.21.15.5:0/3379509832 wait complete. 2023-09-23T04:46:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:46:05.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1752971869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:05 smithi005 ceph-mon[97638]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:05.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:05 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1181180463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T04:46:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1752971869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:05 smithi118 ceph-mon[134297]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:05.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:05 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1181180463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1752971869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T04:46:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:05 smithi069 ceph-mon[133931]: from='mgr.14150 172.21.15.5:0/3830523259' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:05 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1181180463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T04:46:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 -- 172.21.15.5:0/1417452909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c100e70 msgr2=0x7f788c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 --2- 172.21.15.5:0/1417452909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c100e70 0x7f788c101290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7880009990 tx=0x7f7880009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 -- 172.21.15.5:0/1417452909 shutdown_connections 2023-09-23T04:46:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 --2- 172.21.15.5:0/1417452909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c102a10 0x7f788c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 --2- 172.21.15.5:0/1417452909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c100e70 0x7f788c101290 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 --2- 172.21.15.5:0/1417452909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102050 0x7f788c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 -- 172.21.15.5:0/1417452909 >> 172.21.15.5:0/1417452909 conn(0x7f788c0fc410 msgr2=0x7f788c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 -- 172.21.15.5:0/1417452909 shutdown_connections 2023-09-23T04:46:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.016+0000 7f7892bff700 1 -- 172.21.15.5:0/1417452909 wait complete. 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.017+0000 7f7892bff700 1 Processor -- start 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.017+0000 7f7892bff700 1 -- start start 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.017+0000 7f7892bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c100e70 0x7f788c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.017+0000 7f7892bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f7892bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102a10 0x7f788c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f7892bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19e760 con 0x7f788c102050 2023-09-23T04:46:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f7892bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19e8d0 con 0x7f788c100e70 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f7892bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c19ea90 con 0x7f788c102a10 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41148/0 (socket says 172.21.15.5:41148) 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 -- 172.21.15.5:0/956014395 learned_addr learned my addr 172.21.15.5:0/956014395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f789119c700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102a10 0x7f788c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f789099b700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c100e70 0x7f788c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 -- 172.21.15.5:0/956014395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102a10 msgr2=0x7f788c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102a10 0x7f788c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 -- 172.21.15.5:0/956014395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c100e70 msgr2=0x7f788c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c100e70 0x7f788c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.018+0000 7f788bfff700 1 -- 172.21.15.5:0/956014395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800095f0 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f788bfff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f787800e970 tx=0x7f787800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878009790 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787800c7e0 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c19ed70 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878003c70 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.019+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c19f1d0 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.020+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c107ea0 con 0x7f788c102050 2023-09-23T04:46:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.022+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f787800cc50 con 0x7f788c102050 2023-09-23T04:46:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.024+0000 7f7889ffb700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 0x7f787c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.024+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f7878014070 con 0x7f788c102050 2023-09-23T04:46:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.024+0000 7f789099b700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 0x7f787c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.025+0000 7f789099b700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 0x7f787c0a3a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f7880005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.028+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f78780966e0 con 0x7f788c102050 2023-09-23T04:46:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.259+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f788c10c730 con 0x7f788c102050 2023-09-23T04:46:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.261+0000 7f7889ffb700 1 -- 172.21.15.5:0/956014395 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24774 (secure 0 0 0) 0x7f7878003dd0 con 0x7f788c102050 2023-09-23T04:46:06.261 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T04:45:25.785139+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T04:46:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 msgr2=0x7f787c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 0x7f787c0a3a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f7880005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 msgr2=0x7f788c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f787800e970 tx=0x7f787800ec80 comp rx=0 tx=0).stop 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 shutdown_connections 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f787c0a15a0 0x7f787c0a3a60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f788c102a10 0x7f788c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f788c100e70 0x7f788c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 --2- 172.21.15.5:0/956014395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f788c102050 0x7f788c197160 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 >> 172.21.15.5:0/956014395 conn(0x7f788c0fc410 msgr2=0x7f788c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 shutdown_connections 2023-09-23T04:46:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.263+0000 7f7892bff700 1 -- 172.21.15.5:0/956014395 wait complete. 2023-09-23T04:46:06.284 DEBUG:tasks.fwd_scrub:Yielding 2023-09-23T04:46:06.284 INFO:teuthology.run_tasks:Running task exec... 2023-09-23T04:46:06.295 INFO:teuthology.task.exec:Executing custom commands... 2023-09-23T04:46:06.295 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T04:46:06.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-23T04:46:06.296 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-23T04:46:06.296 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:46:06.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:46:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 -- 172.21.15.5:0/57949798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c0ff760 msgr2=0x7f888c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 --2- 172.21.15.5:0/57949798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c0ff760 0x7f888c101b90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8888009990 tx=0x7f8888009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 -- 172.21.15.5:0/57949798 shutdown_connections 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 --2- 172.21.15.5:0/57949798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c104aa0 0x7f888c104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 --2- 172.21.15.5:0/57949798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c0ff760 0x7f888c101b90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 --2- 172.21.15.5:0/57949798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 -- 172.21.15.5:0/57949798 >> 172.21.15.5:0/57949798 conn(0x7f888c0fb3a0 msgr2=0x7f888c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 -- 172.21.15.5:0/57949798 shutdown_connections 2023-09-23T04:46:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.520+0000 7f8893bff700 1 -- 172.21.15.5:0/57949798 wait complete. 2023-09-23T04:46:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.521+0000 7f8893bff700 1 Processor -- start 2023-09-23T04:46:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.521+0000 7f8893bff700 1 -- start start 2023-09-23T04:46:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c104aa0 0x7f888c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1a4d10 con 0x7f888c0ff760 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1a4e50 con 0x7f888c104aa0 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f8893bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c1a4fe0 con 0x7f888c1020d0 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f889199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f889199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41154/0 (socket says 172.21.15.5:41154) 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f889199b700 1 -- 172.21.15.5:0/2122742903 learned_addr learned my addr 172.21.15.5:0/2122742903 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.522+0000 7f889119a700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889219c700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c104aa0 0x7f888c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 -- 172.21.15.5:0/2122742903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 msgr2=0x7f888c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 -- 172.21.15.5:0/2122742903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c104aa0 msgr2=0x7f888c1a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c104aa0 0x7f888c1a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 -- 172.21.15.5:0/2122742903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88880095f0 con 0x7f888c0ff760 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889119a700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.523+0000 7f889199b700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8888004740 tx=0x7f8888009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.526+0000 7f8882ffd700 1 -- 172.21.15.5:0/2122742903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888801d070 con 0x7f888c0ff760 2023-09-23T04:46:06.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.526+0000 7f8882ffd700 1 -- 172.21.15.5:0/2122742903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8888004830 con 0x7f888c0ff760 2023-09-23T04:46:06.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.526+0000 7f8882ffd700 1 -- 172.21.15.5:0/2122742903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880053e0 con 0x7f888c0ff760 2023-09-23T04:46:06.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.527+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c1a5260 con 0x7f888c0ff760 2023-09-23T04:46:06.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.527+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c1a55d0 con 0x7f888c0ff760 2023-09-23T04:46:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f8882ffd700 1 -- 172.21.15.5:0/2122742903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f8888005560 con 0x7f888c0ff760 2023-09-23T04:46:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 -- 172.21.15.5:0/532380439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 msgr2=0x7f0a1810a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 --2- 172.21.15.5:0/532380439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a1810a250 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009990 tx=0x7f0a0c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 -- 172.21.15.5:0/532380439 shutdown_connections 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 --2- 172.21.15.5:0/532380439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810d160 0x7f0a1810d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 --2- 172.21.15.5:0/532380439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a1810a250 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 --2- 172.21.15.5:0/532380439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810a790 0x7f0a1810cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 -- 172.21.15.5:0/532380439 >> 172.21.15.5:0/532380439 conn(0x7f0a1806dc30 msgr2=0x7f0a18070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 -- 172.21.15.5:0/532380439 shutdown_connections 2023-09-23T04:46:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.533+0000 7f0a1edff700 1 -- 172.21.15.5:0/532380439 wait complete. 2023-09-23T04:46:06.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.534+0000 7f0a1edff700 1 Processor -- start 2023-09-23T04:46:06.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.534+0000 7f8882ffd700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 0x7f88780a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 -- start start 2023-09-23T04:46:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f8882ffd700 1 -- 172.21.15.5:0/2122742903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f88880ce3e0 con 0x7f888c0ff760 2023-09-23T04:46:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f88a7469b80 1 Processor -- start 2023-09-23T04:46:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a181a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f889119a700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 0x7f88780a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810a790 0x7f0a181a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 0x7f0a181acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181ad4c0 con 0x7f0a18107e20 2023-09-23T04:46:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181ad600 con 0x7f0a1810d160 2023-09-23T04:46:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1edff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181ad790 con 0x7f0a1810a790 2023-09-23T04:46:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f88a7469b80 1 -- start start 2023-09-23T04:46:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1cb9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a181a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1cb9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a181a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41166/0 (socket says 172.21.15.5:41166) 2023-09-23T04:46:06.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1cb9b700 1 -- 172.21.15.5:0/3499791883 learned_addr learned my addr 172.21.15.5:0/3499791883 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a1d39c700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 0x7f0a181acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f0a17fff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810a790 0x7f0a181a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.535+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 0x562dcc267a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc2692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f88a7469b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1cf4e0 con 0x562dcc267620 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f88a7469b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1c0ca0 con 0x562dcc267f80 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f88a7469b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1be5c0 con 0x562dcc268e20 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 -- 172.21.15.5:0/3499791883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810a790 msgr2=0x7f0a181a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810a790 0x7f0a181a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 -- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 msgr2=0x7f0a181a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a181a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 -- 172.21.15.5:0/3499791883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0095f0 con 0x7f0a1810d160 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 0x562dcc267a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59926/0 (socket says 172.21.15.5:59926) 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 -- 172.21.15.5:0/3454505480 learned_addr learned my addr 172.21.15.5:0/3454505480 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889219c700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc2692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 -- 172.21.15.5:0/3454505480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 msgr2=0x562dcc2692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc2692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 -- 172.21.15.5:0/3454505480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 msgr2=0x562dcc267a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 0x562dcc267a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f0a1d39c700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 0x7f0a181acef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000ab40 tx=0x7f0a1000ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889119a700 1 -- 172.21.15.5:0/3454505480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcc21c950 con 0x562dcc267f80 2023-09-23T04:46:06.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889199b700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 0x562dcc267a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:06.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.536+0000 7f889219c700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc2692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000f070 con 0x7f0a1810d160 2023-09-23T04:46:06.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a10007a70 con 0x7f0a1810d160 2023-09-23T04:46:06.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10005280 con 0x7f0a1810d160 2023-09-23T04:46:06.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a181ada70 con 0x7f0a1810d160 2023-09-23T04:46:06.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f889119a700 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f887c00f470 tx=0x7f887c00f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fcb3b0aeff676ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181adf30 con 0x7f0a1810d160 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f889119a700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 0x7f88780a3ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f887c00fad0 tx=0x7f887c013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f8877fff700 1 -- 172.21.15.5:0/3454505480 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00ec50 con 0x562dcc267f80 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f8877fff700 1 -- 172.21.15.5:0/3454505480 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f887c015070 con 0x562dcc267f80 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.537+0000 7f8877fff700 1 -- 172.21.15.5:0/3454505480 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0038e0 con 0x562dcc267f80 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 -- 172.21.15.5:0/3454505480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 msgr2=0x562dcc268840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f887c00f470 tx=0x7f887c00f780 comp rx=0 tx=0).stop 2023-09-23T04:46:06.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1806f7b0 con 0x7f0a1810d160 2023-09-23T04:46:06.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 -- 172.21.15.5:0/3454505480 shutdown_connections 2023-09-23T04:46:06.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc2692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267620 0x562dcc267a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 --2- 172.21.15.5:0/3454505480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc268840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 -- 172.21.15.5:0/3454505480 >> 172.21.15.5:0/3454505480 conn(0x562dcc1db2e0 msgr2=0x562dcc262500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 -- 172.21.15.5:0/3454505480 shutdown_connections 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.538+0000 7f88a7469b80 1 -- 172.21.15.5:0/3454505480 wait complete. 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.539+0000 7f88a7469b80 1 -- start start 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.539+0000 7f88a7469b80 1 Processor -- start 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 0x562dcc271410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc278990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1b9eb0 con 0x562dcc268e20 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc156790 con 0x562dcc267f80 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc156430 con 0x562dcc267620 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f88a7469b80 10 client.0 _ll_register_callbacks cb 0x562dcbfb3880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889119a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 0x562dcc271410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889119a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59930/0 (socket says 172.21.15.5:59930) 2023-09-23T04:46:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889119a700 1 -- 172.21.15.5:0/1945251001 learned_addr learned my addr 172.21.15.5:0/1945251001 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f0a10004ad0 con 0x7f0a1810d160 2023-09-23T04:46:06.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889219c700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc278990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 -- 172.21.15.5:0/1945251001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 msgr2=0x562dcc271950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 -- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 msgr2=0x562dcc278990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc278990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889199b700 1 -- 172.21.15.5:0/1945251001 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8884011230 con 0x562dcc267620 2023-09-23T04:46:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.540+0000 7f889119a700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f889199b700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 0x562dcc271410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88880cbe60 tx=0x7f888809ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d078ac34042fd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88757fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:46:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888809b330 con 0x562dcc267620 2023-09-23T04:46:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f888809f480 con 0x562dcc267620 2023-09-23T04:46:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562dcc25a2d0 con 0x562dcc267620 2023-09-23T04:46:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88880a3630 con 0x562dcc267620 2023-09-23T04:46:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88a7469b80 10 client.25588 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f887c011ce0 con 0x562dcc267620 2023-09-23T04:46:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.541+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562dcc25a2d0 con 0x562dcc267620 2023-09-23T04:46:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f88880a31a0 con 0x562dcc267620 2023-09-23T04:46:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 10 client.25588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:06.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 20 client.25588 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:06.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc27ceb0 0x562dcc27f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 4 client.25588 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:06.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dcc1b2e90 con 0x562dcc27ceb0 2023-09-23T04:46:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f0a15ffb700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 0x7f0a080a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f889119a700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc27ceb0 0x562dcc27f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f0a100cc6c0 con 0x7f0a1810d160 2023-09-23T04:46:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.542+0000 7f0a1cb9b700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 0x7f0a080a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f889119a700 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc27ceb0 0x562dcc27f2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c2b4252c77fdc9 server_cookie=a3d78f512aa895aa in_seq=0 out_seq=0 2023-09-23T04:46:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f88757fa700 10 client.25588 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f0a1cb9b700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 0x7f0a080a3ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b2e0 tx=0x7f0a0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f88757fa700 1 -- 172.21.15.5:0/1945251001 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562dcc1b2e90 con 0x562dcc27ceb0 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f88757fa700 10 client.25588 handle_command_reply: tid=0 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.543+0000 7f88a7469b80 1 client.25588 shutdown 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 msgr2=0x562dcc271410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 0x562dcc271410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88880cbe60 tx=0x7f888809ba30 comp rx=0 tx=0).stop 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 shutdown_connections 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc27ceb0 0x562dcc27f2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc278990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267620 0x562dcc271410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1945251001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267f80 0x562dcc271950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 >> 172.21.15.5:0/1945251001 conn(0x562dcc1db2e0 msgr2=0x562dcc262310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88757fa700 0 client.25588 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 shutdown_connections 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 1 -- 172.21.15.5:0/1945251001 wait complete. 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.544+0000 7f88a7469b80 20 client.25588 trim_cache size 0 max 16384 2023-09-23T04:46:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.546+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f0a10095950 con 0x7f0a1810d160 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.562+0000 7f88a7469b80 1 Processor -- start 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.562+0000 7f88a7469b80 1 -- start start 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 0x562dcc278950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc271700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc03a9a0 con 0x562dcc268e20 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc1b2e90 con 0x562dcc267620 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f88a7469b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc2686d0 con 0x562dcc267f80 2023-09-23T04:46:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc271700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc271700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41198/0 (socket says 172.21.15.5:41198) 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 0x562dcc278950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889219c700 1 -- 172.21.15.5:0/1680065355 learned_addr learned my addr 172.21.15.5:0/1680065355 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 -- 172.21.15.5:0/1680065355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 msgr2=0x562dcc2711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 -- 172.21.15.5:0/1680065355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 msgr2=0x562dcc271700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc271700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.563+0000 7f889119a700 1 -- 172.21.15.5:0/1680065355 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcc261250 con 0x562dcc267f80 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.564+0000 7f889119a700 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 0x562dcc278950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562dcc27fbb0 tx=0x7f887c0119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4831ff7df432a513 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.564+0000 7f8880bff700 1 -- 172.21.15.5:0/1680065355 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c02d050 con 0x562dcc267f80 2023-09-23T04:46:06.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.564+0000 7f8880bff700 1 -- 172.21.15.5:0/1680065355 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f887c02e040 con 0x562dcc267f80 2023-09-23T04:46:06.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.564+0000 7f8880bff700 1 -- 172.21.15.5:0/1680065355 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c034980 con 0x562dcc267f80 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 -- 172.21.15.5:0/1680065355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 msgr2=0x562dcc278950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 0x562dcc278950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x562dcc27fbb0 tx=0x7f887c0119c0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 -- 172.21.15.5:0/1680065355 shutdown_connections 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc268e20 0x562dcc271700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 --2- 172.21.15.5:0/1680065355 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc267f80 0x562dcc278950 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 -- 172.21.15.5:0/1680065355 >> 172.21.15.5:0/1680065355 conn(0x562dcc1db2e0 msgr2=0x562dcc26bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 -- 172.21.15.5:0/1680065355 shutdown_connections 2023-09-23T04:46:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.565+0000 7f88a7469b80 1 -- 172.21.15.5:0/1680065355 wait complete. 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.566+0000 7f88a7469b80 1 -- start start 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.566+0000 7f88a7469b80 1 Processor -- start 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 0x562dcc277720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc277c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc2d0ca0 con 0x562dcc267f80 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc2d0de0 con 0x562dcc267620 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcc2d0f20 con 0x562dcc268e20 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f88a7469b80 10 client.0 _ll_register_callbacks cb 0x7f888809ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 0x562dcc277720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59952/0 (socket says 172.21.15.5:59952) 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 -- 172.21.15.5:0/681685735 learned_addr learned my addr 172.21.15.5:0/681685735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889219c700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc277c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 -- 172.21.15.5:0/681685735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 msgr2=0x562dcc277c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc277c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 -- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 msgr2=0x562dcc277720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 0x562dcc277720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889199b700 1 -- 172.21.15.5:0/681685735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dcc21c950 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.567+0000 7f889119a700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 0x562dcc277720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f889199b700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88880cbb40 tx=0x7f88880ce360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b569afcf0586861c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88757fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888097120 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f888809b330 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f887c015070 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88880a3630 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88a7469b80 10 client.25547 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562dcc25e500 con 0x562dcc267620 2023-09-23T04:46:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.568+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f887c015070 con 0x562dcc267620 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f88880b3bc0 con 0x562dcc267620 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 20 client.25547 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc2d2480 0x562dcc2d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 4 client.25547 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562dcc1b2bb0 con 0x562dcc2d2480 2023-09-23T04:46:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.569+0000 7f889119a700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc2d2480 0x562dcc2d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.570+0000 7f88757fa700 10 client.25547 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.570+0000 7f889119a700 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc2d2480 0x562dcc2d4870 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e219c7b90cab2089 server_cookie=79561fba2524ea1 in_seq=0 out_seq=0 2023-09-23T04:46:06.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88757fa700 1 -- 172.21.15.5:0/681685735 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562dcc1b2bb0 con 0x562dcc2d2480 2023-09-23T04:46:06.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88757fa700 10 client.25547 handle_command_reply: tid=0 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 client.25547 shutdown 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 msgr2=0x562dcc2771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88880cbb40 tx=0x7f88880ce360 comp rx=0 tx=0).stop 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 shutdown_connections 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562dcc2d2480 0x562dcc2d4870 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562dcc268e20 0x562dcc277c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562dcc267620 0x562dcc2771e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 --2- 172.21.15.5:0/681685735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dcc267f80 0x562dcc277720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88757fa700 0 client.25547 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.571+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 >> 172.21.15.5:0/681685735 conn(0x562dcc1db2e0 msgr2=0x562dcc2711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.572+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 shutdown_connections 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.572+0000 7f88a7469b80 1 -- 172.21.15.5:0/681685735 wait complete. 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.572+0000 7f88a7469b80 20 client.25547 trim_cache size 0 max 16384 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:46:06.576 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "fa1d376c-5ef9-479f-acf8-b1edfd47d953", 2023-09-23T04:46:06.577 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:46:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:46:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 msgr2=0x7f88780a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 0x7f88780a3ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f887c00fad0 tx=0x7f887c013000 comp rx=0 tx=0).stop 2023-09-23T04:46:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 msgr2=0x7f888c19d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8888004740 tx=0x7f8888009580 comp rx=0 tx=0).stop 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 shutdown_connections 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f88780a1620 0x7f88780a3ae0 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f887c00fad0 tx=0x7f887c013000 comp rx=0 tx=0).stop 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f888c104aa0 0x7f888c1a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f888c0ff760 0x7f888c19d190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 --2- 172.21.15.5:0/2122742903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f888c1020d0 0x7f888c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 >> 172.21.15.5:0/2122742903 conn(0x7f888c0fb3a0 msgr2=0x7f888c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 shutdown_connections 2023-09-23T04:46:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.573+0000 7f8893bff700 1 -- 172.21.15.5:0/2122742903 wait complete. 2023-09-23T04:46:06.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:06 smithi005 ceph-mon[97638]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-23T04:46:06.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:06 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3379509832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:06.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:06 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/956014395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:06.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:46:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 -- 172.21.15.5:0/1694467171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 msgr2=0x7f49a8109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 --2- 172.21.15.5:0/1694467171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a8109040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009f90 tx=0x7f49a000b330 comp rx=0 tx=0).stop 2023-09-23T04:46:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 -- 172.21.15.5:0/1694467171 shutdown_connections 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 --2- 172.21.15.5:0/1694467171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a8109040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 --2- 172.21.15.5:0/1694467171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a8101970 0x7f49a8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 --2- 172.21.15.5:0/1694467171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a81042a0 0x7f49a8106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 -- 172.21.15.5:0/1694467171 >> 172.21.15.5:0/1694467171 conn(0x7f49a80fb3a0 msgr2=0x7f49a80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 -- 172.21.15.5:0/1694467171 shutdown_connections 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.824+0000 7f49af7ff700 1 -- 172.21.15.5:0/1694467171 wait complete. 2023-09-23T04:46:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 Processor -- start 2023-09-23T04:46:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 -- start start 2023-09-23T04:46:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 0x7f49a8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a819e830 con 0x7f49a8101970 2023-09-23T04:46:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a819e9a0 con 0x7f49a8106bd0 2023-09-23T04:46:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.826+0000 7f49af7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a819eb60 con 0x7f49a81042a0 2023-09-23T04:46:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 0x7f49a8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41224/0 (socket says 172.21.15.5:41224) 2023-09-23T04:46:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49ad59b700 1 -- 172.21.15.5:0/4062853445 learned_addr learned my addr 172.21.15.5:0/4062853445 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49add9c700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 -- 172.21.15.5:0/4062853445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 msgr2=0x7f49a819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 -- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 msgr2=0x7f49a8196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49acd9a700 1 -- 172.21.15.5:0/4062853445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980095f0 con 0x7f49a81042a0 2023-09-23T04:46:06.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.827+0000 7f49ad59b700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.828+0000 7f49acd9a700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 0x7f49a8197230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f499800eb90 tx=0x7f499800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.828+0000 7f499e7fc700 1 -- 172.21.15.5:0/4062853445 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998009970 con 0x7f49a81042a0 2023-09-23T04:46:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.828+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a0009c40 con 0x7f49a81042a0 2023-09-23T04:46:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.828+0000 7f499e7fc700 1 -- 172.21.15.5:0/4062853445 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49980044e0 con 0x7f49a81042a0 2023-09-23T04:46:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.828+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a819f1c0 con 0x7f49a81042a0 2023-09-23T04:46:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.829+0000 7f499e7fc700 1 -- 172.21.15.5:0/4062853445 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998003e40 con 0x7f49a81042a0 2023-09-23T04:46:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.832+0000 7f499e7fc700 1 -- 172.21.15.5:0/4062853445 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 18) v1 ==== 136191+0+0 (secure 0 0 0) 0x7f4998003fa0 con 0x7f49a81042a0 2023-09-23T04:46:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.833+0000 7f499e7fc700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 0x7f49940a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.833+0000 7f49c3154b80 1 Processor -- start 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.833+0000 7f499e7fc700 1 -- 172.21.15.5:0/4062853445 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f4998014070 con 0x7f49a81042a0 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.833+0000 7f49ad59b700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 0x7f49940a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.833+0000 7f49c3154b80 1 -- start start 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806351740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806352150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588062b8460 con 0x558806351cd0 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588062a9c20 con 0x5588063504d0 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49c3154b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588062a7540 con 0x558806350e30 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59962/0 (socket says 172.21.15.5:59962) 2023-09-23T04:46:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49ad59b700 1 -- 172.21.15.5:0/3957748597 learned_addr learned my addr 172.21.15.5:0/3957748597 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806351740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.834+0000 7f49ad59b700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 0x7f49940a3a60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49a81981c0 tx=0x7f49a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49add9c700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806352150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 -- 172.21.15.5:0/3957748597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 msgr2=0x558806351740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806351740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 -- 172.21.15.5:0/3957748597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 msgr2=0x558806352150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806352150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 -- 172.21.15.5:0/3957748597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588063058d0 con 0x5588063504d0 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806351740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49a40043b0 tx=0x7f49a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64003b5f069fcbe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49937fe700 1 -- 172.21.15.5:0/3957748597 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4003d10 con 0x5588063504d0 2023-09-23T04:46:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49937fe700 1 -- 172.21.15.5:0/3957748597 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49a4003e70 con 0x5588063504d0 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.835+0000 7f49937fe700 1 -- 172.21.15.5:0/3957748597 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4026840 con 0x5588063504d0 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.836+0000 7f49c3154b80 1 -- 172.21.15.5:0/3957748597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 msgr2=0x5588063508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.836+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49a40043b0 tx=0x7f49a4014000 comp rx=0 tx=0).stop 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 -- 172.21.15.5:0/3957748597 shutdown_connections 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806352150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x5588063508f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3957748597 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806351740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 -- 172.21.15.5:0/3957748597 >> 172.21.15.5:0/3957748597 conn(0x5588062c4260 msgr2=0x55880634b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 -- 172.21.15.5:0/3957748597 shutdown_connections 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 -- 172.21.15.5:0/3957748597 wait complete. 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.837+0000 7f49c3154b80 1 -- start start 2023-09-23T04:46:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.838+0000 7f49c3154b80 1 Processor -- start 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.838+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.838+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806350e30 0x558806363110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806351cd0 0x55880636a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49c3154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588062a2e30 con 0x558806350e30 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49c3154b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880623f710 con 0x5588063504d0 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49c3154b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880623f3b0 con 0x558806351cd0 2023-09-23T04:46:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49c3154b80 10 client.0 _ll_register_callbacks cb 0x55880609c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59966/0 (socket says 172.21.15.5:59966) 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 -- 172.21.15.5:0/3775884296 learned_addr learned my addr 172.21.15.5:0/3775884296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49add9c700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806351cd0 0x55880636a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806350e30 0x558806363110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 -- 172.21.15.5:0/3775884296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806351cd0 msgr2=0x55880636a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806351cd0 0x55880636a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 -- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806350e30 msgr2=0x558806363110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.839+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806350e30 0x558806363110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f49ad59b700 1 -- 172.21.15.5:0/3775884296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a819f1c0 con 0x5588063504d0 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49a40046e0 tx=0x7f49a40052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b73109f90c6337a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f4990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a402a070 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558806343250 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f49a4004950 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a4004b10 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f49c3154b80 10 client.25559 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.840+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f49a4017070 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558806343250 con 0x5588063504d0 2023-09-23T04:46:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f49a4024020 con 0x5588063504d0 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 20 client.25559 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55880636e870 0x558806370c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 4 client.25559 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:06.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55880629be10 con 0x55880636e870 2023-09-23T04:46:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.841+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55880636e870 0x558806370c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.842+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55880636e870 0x558806370c90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6714d2c873e33970 server_cookie=9c5449e3d22f73ce in_seq=0 out_seq=0 2023-09-23T04:46:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.842+0000 7f4990ff9700 10 client.25559 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.842+0000 7f4990ff9700 1 -- 172.21.15.5:0/3775884296 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55880629be10 con 0x55880636e870 2023-09-23T04:46:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.842+0000 7f4990ff9700 10 client.25559 handle_command_reply: tid=0 2023-09-23T04:46:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 client.25559 shutdown 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 msgr2=0x558806362bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49a40046e0 tx=0x7f49a40052e0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 shutdown_connections 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55880636e870 0x558806370c90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806351cd0 0x55880636a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806362bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3775884296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806350e30 0x558806363110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 >> 172.21.15.5:0/3775884296 conn(0x5588062c4260 msgr2=0x55880634b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f4990ff9700 0 client.25559 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 shutdown_connections 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.843+0000 7f49c3154b80 1 -- 172.21.15.5:0/3775884296 wait complete. 2023-09-23T04:46:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.844+0000 7f49c3154b80 20 client.25559 trim_cache size 0 max 16384 2023-09-23T04:46:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.851+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0a1802d0b0 con 0x7f0a1810d160 2023-09-23T04:46:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 Processor -- start 2023-09-23T04:46:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 -- start start 2023-09-23T04:46:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x55880635dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x55880636a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558806123920 con 0x558806351cd0 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558806347dc0 con 0x5588063504d0 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.871+0000 7f49c3154b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880635e630 con 0x558806350e30 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59982/0 (socket says 172.21.15.5:59982) 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 -- 172.21.15.5:0/1586104944 learned_addr learned my addr 172.21.15.5:0/1586104944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49add9c700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x55880636a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49acd9a700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x55880635dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 -- 172.21.15.5:0/1586104944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 msgr2=0x55880635dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x55880635dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 -- 172.21.15.5:0/1586104944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 msgr2=0x55880636a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x55880636a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 -- 172.21.15.5:0/1586104944 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588060cdde0 con 0x5588063504d0 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49add9c700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x55880636a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f49ad59b700 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009880 tx=0x7f49a4004f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e2ba922f06611e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.872+0000 7f4993fff700 1 -- 172.21.15.5:0/1586104944 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a402a030 con 0x5588063504d0 2023-09-23T04:46:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f4993fff700 1 -- 172.21.15.5:0/1586104944 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49a402b040 con 0x5588063504d0 2023-09-23T04:46:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f4993fff700 1 -- 172.21.15.5:0/1586104944 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a401e420 con 0x5588063504d0 2023-09-23T04:46:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 -- 172.21.15.5:0/1586104944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 msgr2=0x55880635daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009880 tx=0x7f49a4004f80 comp rx=0 tx=0).stop 2023-09-23T04:46:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 -- 172.21.15.5:0/1586104944 shutdown_connections 2023-09-23T04:46:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x55880636a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x55880635daa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 --2- 172.21.15.5:0/1586104944 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x55880635dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 -- 172.21.15.5:0/1586104944 >> 172.21.15.5:0/1586104944 conn(0x5588062c4260 msgr2=0x558806349f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 -- 172.21.15.5:0/1586104944 shutdown_connections 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.873+0000 7f49c3154b80 1 -- 172.21.15.5:0/1586104944 wait complete. 2023-09-23T04:46:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 -- start start 2023-09-23T04:46:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 Processor -- start 2023-09-23T04:46:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806369960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806366400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558806366940 con 0x558806351cd0 2023-09-23T04:46:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558806366ab0 con 0x5588063504d0 2023-09-23T04:46:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558806366c70 con 0x558806350e30 2023-09-23T04:46:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.874+0000 7f49c3154b80 10 client.0 _ll_register_callbacks cb 0x55880609c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806369960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49ad59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806369960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59992/0 (socket says 172.21.15.5:59992) 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44800/0 (socket says 172.21.15.5:44800) 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49ad59b700 1 -- 172.21.15.5:0/3677830320 learned_addr learned my addr 172.21.15.5:0/3677830320 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49add9c700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806366400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 -- 172.21.15.5:0/3677830320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 msgr2=0x558806369960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806369960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 -- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 msgr2=0x558806366400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806366400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 -- 172.21.15.5:0/3677830320 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a4017070 con 0x558806350e30 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f49acd9a700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49980caf50 tx=0x7f4998085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17fdbabbc6146fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.875+0000 7f4990ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800cc90 con 0x558806350e30 2023-09-23T04:46:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f499809a700 con 0x558806350e30 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49a4003e70 con 0x558806350e30 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f499809e850 con 0x558806350e30 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 10 client.25612 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588060b6c60 con 0x558806350e30 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49a4003e70 con 0x558806350e30 2023-09-23T04:46:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4998096e10 con 0x558806350e30 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 20 client.25612 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5588063c3b70 0x5588063c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 4 client.25612 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.876+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55880629bb30 con 0x5588063c3b70 2023-09-23T04:46:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5588063c3b70 0x5588063c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f49ad59b700 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5588063c3b70 0x5588063c5f90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb6bdaaa5e154239 server_cookie=7fb63a422a7ea9f8 in_seq=0 out_seq=0 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f4990ff9700 10 client.25612 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f4990ff9700 1 -- 172.21.15.5:0/3677830320 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55880629bb30 con 0x5588063c3b70 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f4990ff9700 10 client.25612 handle_command_reply: tid=0 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f49c3154b80 1 client.25612 shutdown 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 msgr2=0x558806369ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.877+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49980caf50 tx=0x7f4998085040 comp rx=0 tx=0).stop 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 shutdown_connections 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5588063c3b70 0x5588063c5f90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558806351cd0 0x558806366400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5588063504d0 0x558806369960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 --2- 172.21.15.5:0/3677830320 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558806350e30 0x558806369ea0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f4990ff9700 0 client.25612 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 >> 172.21.15.5:0/3677830320 conn(0x5588062c4260 msgr2=0x55880635daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 shutdown_connections 2023-09-23T04:46:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 1 -- 172.21.15.5:0/3677830320 wait complete. 2023-09-23T04:46:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49c3154b80 20 client.25612 trim_cache size 0 max 16384 2023-09-23T04:46:06.889 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "fa1d376c-5ef9-479f-acf8-b1edfd47d953": { 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fa1d376c-5ef9-479f-acf8-b1edfd47d953", 2023-09-23T04:46:06.890 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:46:06.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:46:06.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:46:06.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:46:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 msgr2=0x7f49940a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 0x7f49940a3a60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49a81981c0 tx=0x7f49a4009260 comp rx=0 tx=0).stop 2023-09-23T04:46:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 msgr2=0x7f49a8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.878+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 0x7f49a8197230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f499800eb90 tx=0x7f499800eea0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 shutdown_connections 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f49940a15a0 0x7f49940a3a60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a8106bd0 0x7f49a819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49a8101970 0x7f49a8196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 --2- 172.21.15.5:0/4062853445 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f49a81042a0 0x7f49a8197230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 >> 172.21.15.5:0/4062853445 conn(0x7f49a80fb3a0 msgr2=0x7f49a81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 shutdown_connections 2023-09-23T04:46:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:06.879+0000 7f49af7ff700 1 -- 172.21.15.5:0/4062853445 wait complete. 2023-09-23T04:46:06.896 INFO:tasks.cephfs.filesystem:scrub status for tag:fa1d376c-5ef9-479f-acf8-b1edfd47d953 - {'path': '/', 'tag': 'fa1d376c-5ef9-479f-acf8-b1edfd47d953', 'options': 'recursive,force'} 2023-09-23T04:46:06.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:06 smithi118 ceph-mon[134297]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-23T04:46:06.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:06 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3379509832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:06.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:06 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/956014395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:06 smithi069 ceph-mon[133931]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2023-09-23T04:46:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:06 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3379509832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:06 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/956014395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T04:46:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.542+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 19) v1 ==== 136208+0+0 (secure 0 0 0) 0x7f0a10095d20 con 0x7f0a1810d160 2023-09-23T04:46:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.543+0000 7f0a15ffb700 1 -- 172.21.15.5:0/3499791883 <== mon.2 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0a10061f70 con 0x7f0a1810d160 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 msgr2=0x7f0a080a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 0x7f0a080a3ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b2e0 tx=0x7f0a0c005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 msgr2=0x7f0a181acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 0x7f0a181acef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000ab40 tx=0x7f0a1000ae50 comp rx=0 tx=0).stop 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 shutdown_connections 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f0a080a1620 0x7f0a080a3ae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a1810d160 0x7f0a181acef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a18107e20 0x7f0a181a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 --2- 172.21.15.5:0/3499791883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a1810a790 0x7f0a181a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 >> 172.21.15.5:0/3499791883 conn(0x7f0a1806dc30 msgr2=0x7f0a1810b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 shutdown_connections 2023-09-23T04:46:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.548+0000 7f0a1edff700 1 -- 172.21.15.5:0/3499791883 wait complete. 2023-09-23T04:46:07.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-23T04:46:07.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:46:07.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.797 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3499791883' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: ignoring --setuser ceph since I am not root 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: ignoring --setgroup ceph since I am not root 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-mgr[136459]: -- 172.21.15.118:0/3805206142 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5567a3f6b1e0 con 0x5567a3f40800 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:07.714+0000 7f54ed5c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:46:07.798 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:07.797+0000 7f54ed5c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 -- 172.21.15.5:0/531750523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20073820 msgr2=0x7efe20073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 --2- 172.21.15.5:0/531750523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20073820 0x7efe20073ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe14009990 tx=0x7efe14009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 -- 172.21.15.5:0/531750523 shutdown_connections 2023-09-23T04:46:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 --2- 172.21.15.5:0/531750523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20108d30 0x7efe2010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 --2- 172.21.15.5:0/531750523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 0x7efe20073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 --2- 172.21.15.5:0/531750523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20073820 0x7efe20073ca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.798+0000 7efe249ff700 1 -- 172.21.15.5:0/531750523 >> 172.21.15.5:0/531750523 conn(0x7efe200fc3d0 msgr2=0x7efe200fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.799+0000 7efe249ff700 1 -- 172.21.15.5:0/531750523 shutdown_connections 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.799+0000 7efe249ff700 1 -- 172.21.15.5:0/531750523 wait complete. 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 Processor -- start 2023-09-23T04:46:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 -- start start 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: ignoring --setuser ceph since I am not root 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: ignoring --setgroup ceph since I am not root 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mgr[135409]: -- 172.21.15.69:0/3460782749 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5578d7d171e0 con 0x5578d7ced000 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:07.711+0000 7fb76ce49200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:07.799+0000 7fb76ce49200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:07.801 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:46:07.802 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.802 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:46:07.802 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.802 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3499791883' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.802 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 0x7efe20199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20108d30 0x7efe201a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201a08c0 con 0x7efe20074e00 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201a0a00 con 0x7efe20073820 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe249ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe201a0b90 con 0x7efe20108d30 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:07.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe1e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60014/0 (socket says 172.21.15.5:60014) 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe1e59c700 1 -- 172.21.15.5:0/3704753963 learned_addr learned my addr 172.21.15.5:0/3704753963 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe1dd9b700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 0x7efe20199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.800+0000 7efe1ed9d700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20108d30 0x7efe201a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20108d30 msgr2=0x7efe201a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20108d30 0x7efe201a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 msgr2=0x7efe20199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 0x7efe20199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 -- 172.21.15.5:0/3704753963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140095f0 con 0x7efe20073820 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe1e59c700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e9a0 tx=0x7efe1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.801+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10009790 con 0x7efe20073820 2023-09-23T04:46:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.802+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe10010430 con 0x7efe20073820 2023-09-23T04:46:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.802+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10018680 con 0x7efe20073820 2023-09-23T04:46:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.802+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe201008d0 con 0x7efe20073820 2023-09-23T04:46:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.802+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe20100de0 con 0x7efe20073820 2023-09-23T04:46:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.803+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe20107430 con 0x7efe20073820 2023-09-23T04:46:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.806+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 136208+0+0 (secure 0 0 0) 0x7efe100105a0 con 0x7efe20073820 2023-09-23T04:46:07.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.808+0000 7efe0f7fe700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 0x7efe080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.808+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7efe10014070 con 0x7efe20073820 2023-09-23T04:46:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.808+0000 7efe1dd9b700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 msgr2=0x7efe080a3b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.808+0000 7efe1dd9b700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 0x7efe080a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:46:07.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:07.812+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7efe10051ec0 con 0x7efe20073820 2023-09-23T04:46:07.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:46:07.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3499791883' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setuser ceph since I am not root 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: ignoring --setgroup ceph since I am not root 2023-09-23T04:46:07.855 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:07.853+0000 7f68b0a73200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T04:46:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.009+0000 7efe1dd9b700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 msgr2=0x7efe080a3b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:08.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.009+0000 7efe1dd9b700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 0x7efe080a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:46:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.037+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efe2010e940 con 0x7efe20073820 2023-09-23T04:46:08.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.043+0000 7efe0f7fe700 1 -- 172.21.15.5:0/3704753963 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7efe1009a140 con 0x7efe20073820 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 msgr2=0x7efe080a3b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 0x7efe080a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 msgr2=0x7efe20198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e9a0 tx=0x7efe1000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 shutdown_connections 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7efe080a1670 0x7efe080a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efe20108d30 0x7efe201a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe20073820 0x7efe20198d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 --2- 172.21.15.5:0/3704753963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe20074e00 0x7efe20199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 >> 172.21.15.5:0/3704753963 conn(0x7efe200fc3d0 msgr2=0x7efe200fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:08.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 shutdown_connections 2023-09-23T04:46:08.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.046+0000 7efe249ff700 1 -- 172.21.15.5:0/3704753963 wait complete. 2023-09-23T04:46:08.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-23T04:46:08.138 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:07 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:07.965+0000 7f68b0a73200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:08.138 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:08.097+0000 7f68b0a73200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:46:08.158 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:07.892+0000 7f54ed5c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:46:08.159 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:07.986+0000 7f54ed5c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:46:08.160 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:07.894+0000 7fb76ce49200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T04:46:08.160 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:07.988+0000 7fb76ce49200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.286+0000 7fb5e5fff700 1 -- 172.21.15.5:0/3549921309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0102960 msgr2=0x7fb5e0109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.286+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/3549921309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0102960 0x7fb5e0109280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800b0a0 tx=0x7fb5d800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 -- 172.21.15.5:0/3549921309 shutdown_connections 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/3549921309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0102960 0x7fb5e0109280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/3549921309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0100e50 0x7fb5e0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/3549921309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 -- 172.21.15.5:0/3549921309 >> 172.21.15.5:0/3549921309 conn(0x7fb5e00fc3e0 msgr2=0x7fb5e00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 -- 172.21.15.5:0/3549921309 shutdown_connections 2023-09-23T04:46:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.287+0000 7fb5e5fff700 1 -- 172.21.15.5:0/3549921309 wait complete. 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 Processor -- start 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 -- start start 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 0x7fb5e0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 0x7fb5e01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e01a0750 con 0x7fb5e0100e50 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e01a0890 con 0x7fb5e0102960 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.288+0000 7fb5e5fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e01a0a20 con 0x7fb5e0101fa0 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44854/0 (socket says 172.21.15.5:44854) 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 -- 172.21.15.5:0/926720411 learned_addr learned my addr 172.21.15.5:0/926720411 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5dffff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 0x7fb5e01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5df7fe700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 0x7fb5e0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 msgr2=0x7fb5e01a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 0x7fb5e01a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 msgr2=0x7fb5e0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 0x7fb5e0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 -- 172.21.15.5:0/926720411 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0009520 con 0x7fb5e0101fa0 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5dffff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 0x7fb5e01a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5df7fe700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 0x7fb5e0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.289+0000 7fb5deffd700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400e9a0 tx=0x7fb5d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.290+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4009790 con 0x7fb5e0101fa0 2023-09-23T04:46:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.290+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5d400cbd0 con 0x7fb5e0101fa0 2023-09-23T04:46:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.290+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8009d00 con 0x7fb5e0101fa0 2023-09-23T04:46:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.290+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4018680 con 0x7fb5e0101fa0 2023-09-23T04:46:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.290+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e01a10d0 con 0x7fb5e0101fa0 2023-09-23T04:46:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.291+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e010c5e0 con 0x7fb5e0101fa0 2023-09-23T04:46:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.293+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 19) v1 ==== 136208+0+0 (secure 0 0 0) 0x7fb5d400cd40 con 0x7fb5e0101fa0 2023-09-23T04:46:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.294+0000 7fb5dcff9700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 0x7fb5c80a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.294+0000 7fb5df7fe700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 msgr2=0x7fb5c80a3b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.295+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7fb5d4014070 con 0x7fb5e0101fa0 2023-09-23T04:46:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.295+0000 7fb5df7fe700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 0x7fb5c80a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:46:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.298+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb5d4051ec0 con 0x7fb5e0101fa0 2023-09-23T04:46:08.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:08.233+0000 7f68b0a73200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T04:46:08.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:08.159+0000 7f54ed5c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:46:08.490 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:08.160+0000 7fb76ce49200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:46:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.496+0000 7fb5df7fe700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 msgr2=0x7fb5c80a3b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.496+0000 7fb5df7fe700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 0x7fb5c80a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:46:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.514+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb5e010c7f0 con 0x7fb5e0101fa0 2023-09-23T04:46:08.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.519+0000 7fb5dcff9700 1 -- 172.21.15.5:0/926720411 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb5d4096730 con 0x7fb5e0101fa0 2023-09-23T04:46:08.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 msgr2=0x7fb5c80a3b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:46:08.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 0x7fb5c80a3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 msgr2=0x7fb5e0199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400e9a0 tx=0x7fb5d400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 shutdown_connections 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7fb5c80a1670 0x7fb5c80a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5e0102960 0x7fb5e01a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5e0100e50 0x7fb5e0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 --2- 172.21.15.5:0/926720411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb5e0101fa0 0x7fb5e0199110 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 >> 172.21.15.5:0/926720411 conn(0x7fb5e00fc3e0 msgr2=0x7fb5e00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 shutdown_connections 2023-09-23T04:46:08.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.522+0000 7fb5e5fff700 1 -- 172.21.15.5:0/926720411 wait complete. 2023-09-23T04:46:08.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-09-23T04:46:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 -- 172.21.15.5:0/3383899219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4073880 msgr2=0x7f73d4073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3383899219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4073880 0x7f73d4073d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009990 tx=0x7f73c8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 -- 172.21.15.5:0/3383899219 shutdown_connections 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3383899219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4108d10 0x7f73d410b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3383899219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4074e60 0x7f73d40732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3383899219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4073880 0x7f73d4073d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 -- 172.21.15.5:0/3383899219 >> 172.21.15.5:0/3383899219 conn(0x7f73d40fc3a0 msgr2=0x7f73d40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 -- 172.21.15.5:0/3383899219 shutdown_connections 2023-09-23T04:46:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.771+0000 7f73d91ff700 1 -- 172.21.15.5:0/3383899219 wait complete. 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.772+0000 7f73d91ff700 1 Processor -- start 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.772+0000 7f73d91ff700 1 -- start start 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 0x7f73d4198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 0x7f73d4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4108d10 0x7f73d41a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41a08d0 con 0x7f73d4073880 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41a0a10 con 0x7f73d4074e60 2023-09-23T04:46:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d91ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41a0ba0 con 0x7f73d4108d10 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 0x7f73d4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 0x7f73d4198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d359e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4108d10 0x7f73d41a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 0x7f73d4199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60060/0 (socket says 172.21.15.5:60060) 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d259c700 1 -- 172.21.15.5:0/3991004617 learned_addr learned my addr 172.21.15.5:0/3991004617 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.773+0000 7f73d2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 0x7f73d4198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41314/0 (socket says 172.21.15.5:41314) 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4108d10 msgr2=0x7f73d41a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4108d10 0x7f73d41a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 msgr2=0x7f73d4198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 0x7f73d4198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80095f0 con 0x7f73d4074e60 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d359e700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f73d4108d10 0x7f73d41a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73d4073880 0x7f73d4198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d259c700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 0x7f73d4199290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005720 tx=0x7f73c8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c801d070 con 0x7f73d4074e60 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d41a0e20 con 0x7f73d4074e60 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c8021410 con 0x7f73d4074e60 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8017650 con 0x7f73d4074e60 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.774+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d41a1190 con 0x7f73d4074e60 2023-09-23T04:46:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.775+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d41073c0 con 0x7f73d4074e60 2023-09-23T04:46:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.777+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 136208+0+0 (secure 0 0 0) 0x7f73c8021580 con 0x7f73d4074e60 2023-09-23T04:46:08.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.778+0000 7f73bbfff700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:08.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.779+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:08.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.779+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9247+0+0 (secure 0 0 0) 0x7f73c80ce550 con 0x7f73d4074e60 2023-09-23T04:46:08.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.779+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T04:46:08.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.781+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f73c8097780 con 0x7f73d4074e60 2023-09-23T04:46:08.837 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:08.523+0000 7fb76ce49200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:08.653+0000 7fb76ce49200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:08.741+0000 7fb76ce49200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-mon[133931]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.838 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.839 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:08.522+0000 7f54ed5c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:46:08.839 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:08.654+0000 7f54ed5c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:46:08.839 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:08.742+0000 7f54ed5c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:46:08.839 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T04:46:08.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-mon[134297]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-23T04:46:08.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-23T04:46:08.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-mon[97638]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-09-23T04:46:08.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' 2023-09-23T04:46:08.880 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:08 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:08.507+0000 7f68b0a73200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T04:46:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.980+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:08.980+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-23T04:46:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:09.025+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d402d290 con 0x7f73bc0a15f0 2023-09-23T04:46:09.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:09.009+0000 7f68b0a73200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T04:46:09.160 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:08.839+0000 7f54ed5c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:46:09.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:08.837+0000 7fb76ce49200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:46:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:09.381+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:09.381+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-23T04:46:09.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:09.183+0000 7f68b0a73200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T04:46:09.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:09.304+0000 7f68b0a73200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T04:46:09.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:09.433+0000 7f68b0a73200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T04:46:09.661 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:09.260+0000 7f54ed5c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:46:09.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:09 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:09.255+0000 7fb76ce49200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:46:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:10.182+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:10.182+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-23T04:46:10.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:09 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:09.906+0000 7fb76ce49200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:46:10.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:09 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:09.987+0000 7fb76ce49200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:46:10.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:09.993+0000 7f68b0a73200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T04:46:10.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:09.914+0000 7f54ed5c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:46:10.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:09.995+0000 7f54ed5c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:46:11.021 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:10 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:10.758+0000 7fb76ce49200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:11.021 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:10 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:10.849+0000 7fb76ce49200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:46:11.029 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:10 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:10.767+0000 7f54ed5c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:11.030 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:10 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:10.858+0000 7f54ed5c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:46:11.154 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:10 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:10.881+0000 7f68b0a73200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T04:46:11.155 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:10 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:10.992+0000 7f68b0a73200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T04:46:11.297 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:11.020+0000 7fb76ce49200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:11.308 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:11.030+0000 7f54ed5c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:11.660 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:11.308+0000 7f54ed5c0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:46:11.661 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:11.533+0000 7f54ed5c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:46:11.673 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:11.297+0000 7fb76ce49200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:46:11.673 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:11.522+0000 7fb76ce49200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:46:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:11.784+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:11.784+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-23T04:46:11.937 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:11.672+0000 7fb76ce49200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:46:11.948 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:11.683+0000 7f54ed5c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:46:12.240 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:11.937+0000 7fb76ce49200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:46:12.356 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:11 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:11.995+0000 7f68b0a73200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:12.356 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:12 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:12.121+0000 7f68b0a73200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T04:46:12.410 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:11.947+0000 7f54ed5c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:46:12.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:12 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:12.356+0000 7f68b0a73200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T04:46:12.910 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:12.647+0000 7f54ed5c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:46:12.911 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:12.779+0000 7f54ed5c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:46:12.955 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:12.632+0000 7fb76ce49200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:46:12.955 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:12.762+0000 7fb76ce49200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:46:13.021 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:12 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:12.735+0000 7f68b0a73200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T04:46:13.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:12.955+0000 7fb76ce49200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:46:13.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:13 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:13.067+0000 7fb76ce49200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:13.301 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:12.971+0000 7f54ed5c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:46:13.301 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:13.084+0000 7f54ed5c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:13.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:13 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:13.021+0000 7f68b0a73200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T04:46:13.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:13 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:13.224+0000 7f68b0a73200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T04:46:13.661 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:13.300+0000 7f54ed5c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:46:13.741 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:13 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:13.284+0000 7fb76ce49200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:46:13.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:13 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:13.583+0000 7f68b0a73200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T04:46:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:14.218+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 20) v1 ==== 136208+0+0 (secure 0 0 0) 0x7f73c8097b50 con 0x7f73d4074e60 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y[135384]: 2023-09-23T04:46:14.141+0000 7fb76ce49200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: Standby manager daemon y restarted 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: Standby manager daemon y started 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: Standby manager daemon z restarted 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: Standby manager daemon z started 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:46:14.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:14 smithi069 ceph-mon[133931]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: Standby manager daemon y restarted 2023-09-23T04:46:14.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: Standby manager daemon y started 2023-09-23T04:46:14.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:46:14.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: Standby manager daemon z restarted 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: Standby manager daemon z started 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:46:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-mon[97638]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.660 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z[136416]: 2023-09-23T04:46:14.167+0000 7f54ed5c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: Standby manager daemon y restarted 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: Standby manager daemon y started 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.69:0/1655877347' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: Standby manager daemon z restarted 2023-09-23T04:46:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: Standby manager daemon z started 2023-09-23T04:46:14.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-23T04:46:14.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-23T04:46:14.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-23T04:46:14.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:14 smithi118 ceph-mon[134297]: from='mgr.? 172.21.15.118:0/1835549208' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-23T04:46:14.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:14.569+0000 7f68b0a73200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T04:46:14.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:14.755+0000 7f68b0a73200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T04:46:14.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:14.987+0000 7f73d2d9d700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4010853589 2023-09-23T04:46:14.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:14.987+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-23T04:46:15.404 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:15.016+0000 7f68b0a73200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T04:46:15.405 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:15.168+0000 7f68b0a73200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T04:46:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:15 smithi005 ceph-mon[97638]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-23T04:46:15.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:15 smithi069 ceph-mon[133931]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-23T04:46:15.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:15 smithi118 ceph-mon[134297]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-09-23T04:46:15.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:15.458+0000 7f68b0a73200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T04:46:16.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:16.603+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 21) v1 ==== 135877+0+0 (secure 0 0 0) 0x7f73c80cbc80 con 0x7f73d4074e60 2023-09-23T04:46:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:16.603+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 msgr2=0x7f73bc0a3ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:46:16.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:16.603+0000 7f73bbfff700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/4010853589,v1:172.21.15.5:6801/4010853589] conn(0x7f73bc0a15f0 0x7f73bc0a3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:16.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: Active manager daemon x restarted 2023-09-23T04:46:16.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: Activating manager daemon x 2023-09-23T04:46:16.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T04:46:16.898 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: mgrmap e21: x(active, starting, since 0.0133675s), standbys: y, z 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:46:16.899 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:46:16.900 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:46:16.901 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:16 smithi069 ceph-mon[133931]: Manager daemon x is now available 2023-09-23T04:46:16.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.584+0000 7f68b0a73200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.805+0000 7f6810b81700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.805+0000 7f6810b81700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.805+0000 7f6810b81700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.805+0000 7f6810b81700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.805+0000 7f6810b81700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.809+0000 7f6813b87700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.809+0000 7f6813b87700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.809+0000 7f6813b87700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.809+0000 7f6813b87700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:46:16.809+0000 7f6813b87700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: Active manager daemon x restarted 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: Activating manager daemon x 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: mgrmap e21: x(active, starting, since 0.0133675s), standbys: y, z 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:46:16.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:46:16.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:46:16.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:46:16.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:46:16.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:46:16.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:46:16.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:16 smithi005 ceph-mon[97638]: Manager daemon x is now available 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: Active manager daemon x restarted 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: Activating manager daemon x 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: osdmap e92: 12 total, 12 up, 12 in 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: mgrmap e21: x(active, starting, since 0.0133675s), standbys: y, z 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-23T04:46:16.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-23T04:46:16.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-23T04:46:16.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-23T04:46:16.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-23T04:46:16.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-23T04:46:16.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:16 smithi118 ceph-mon[134297]: Manager daemon x is now available 2023-09-23T04:46:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.608+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mon.2 v2:172.21.15.69:3300/0 9 ==== mgrmap(e 22) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f73c80aec60 con 0x7f73d4074e60 2023-09-23T04:46:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.609+0000 7f73bbfff700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f73bc01ba60 0x7f73bc01be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.609+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d402d290 con 0x7f73bc01ba60 2023-09-23T04:46:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.613+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f73bc01ba60 0x7f73bc01be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.618+0000 7f73d2d9d700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f73bc01ba60 0x7f73bc01be40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009c30 tx=0x7f73c4009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.683+0000 7f73bbfff700 1 -- 172.21.15.5:0/3991004617 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f73d402d290 con 0x7f73bc01ba60 2023-09-23T04:46:17.683 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2023-09-23T04:46:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.685+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f73bc01ba60 msgr2=0x7f73bc01be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.685+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f73bc01ba60 0x7f73bc01be40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009c30 tx=0x7f73c4009260 comp rx=0 tx=0).stop 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.685+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 msgr2=0x7f73d4199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.685+0000 7f73d91ff700 1 --2- 172.21.15.5:0/3991004617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73d4074e60 0x7f73d4199290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005720 tx=0x7f73c8009580 comp rx=0 tx=0).stop 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.685+0000 7f73d359e700 1 -- 172.21.15.5:0/3991004617 reap_dead start 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.686+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 shutdown_connections 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.686+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 >> 172.21.15.5:0/3991004617 conn(0x7f73d40fc3a0 msgr2=0x7f73d40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.686+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 shutdown_connections 2023-09-23T04:46:17.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.686+0000 7f73d91ff700 1 -- 172.21.15.5:0/3991004617 wait complete. 2023-09-23T04:46:17.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-09-23T04:46:17.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:17 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:17 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:17 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:46:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:17 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:46:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:17 smithi005 ceph-mon[97638]: mgrmap e22: x(active, since 1.02088s), standbys: y, z 2023-09-23T04:46:17.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:17 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:17 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:17 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:46:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:17 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:46:17.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:17 smithi118 ceph-mon[134297]: mgrmap e22: x(active, since 1.02088s), standbys: y, z 2023-09-23T04:46:17.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.920+0000 7f75923ff700 1 -- 172.21.15.5:0/1208800871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 msgr2=0x7f758c101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.920+0000 7f75923ff700 1 --2- 172.21.15.5:0/1208800871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c101250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f757c009990 tx=0x7f757c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 -- 172.21.15.5:0/1208800871 shutdown_connections 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 --2- 172.21.15.5:0/1208800871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 0x7f758c109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 --2- 172.21.15.5:0/1208800871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c101250 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 --2- 172.21.15.5:0/1208800871 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 -- 172.21.15.5:0/1208800871 >> 172.21.15.5:0/1208800871 conn(0x7f758c0fc3a0 msgr2=0x7f758c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 -- 172.21.15.5:0/1208800871 shutdown_connections 2023-09-23T04:46:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.921+0000 7f75923ff700 1 -- 172.21.15.5:0/1208800871 wait complete. 2023-09-23T04:46:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 Processor -- start 2023-09-23T04:46:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 -- start start 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 0x7f758c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a0870 con 0x7f758c100e30 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a09e0 con 0x7f758c102940 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.922+0000 7f75923ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a0ba0 con 0x7f758c101f80 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:60648/0 (socket says 172.21.15.5:60648) 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 -- 172.21.15.5:0/1499200983 learned_addr learned my addr 172.21.15.5:0/1499200983 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f759099c700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 0x7f758c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 -- 172.21.15.5:0/1499200983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 msgr2=0x7f758c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 0x7f758c1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 -- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 msgr2=0x7f758c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 -- 172.21.15.5:0/1499200983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0095f0 con 0x7f758c101f80 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758bfff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f758b7fe700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f758000e9a0 tx=0x7f758000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580009790 con 0x7f758c101f80 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.923+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7580010430 con 0x7f758c101f80 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.924+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c1a0e80 con 0x7f758c101f80 2023-09-23T04:46:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.924+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580018680 con 0x7f758c101f80 2023-09-23T04:46:17.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.924+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c1a1300 con 0x7f758c101f80 2023-09-23T04:46:17.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.925+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c193420 con 0x7f758c101f80 2023-09-23T04:46:17.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.927+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 22) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f75800105a0 con 0x7f758c101f80 2023-09-23T04:46:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.928+0000 7f75897fa700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 0x7f75740a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.928+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7580014070 con 0x7f758c101f80 2023-09-23T04:46:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.928+0000 7f758bfff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 0x7f75740a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.929+0000 7f758bfff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 0x7f75740a38e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f758c19a200 tx=0x7f757c005960 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:17.930+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f75800965c0 con 0x7f758c101f80 2023-09-23T04:46:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:17 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:17 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:17 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-23T04:46:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:17 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-23T04:46:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:17 smithi069 ceph-mon[133931]: mgrmap e22: x(active, since 1.02088s), standbys: y, z 2023-09-23T04:46:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.182+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f758c02d2d0 con 0x7f75740a1420 2023-09-23T04:46:18.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.240+0000 7f75897fa700 1 -- 172.21.15.5:0/1499200983 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f758c02d2d0 con 0x7f75740a1420 2023-09-23T04:46:18.240 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.242+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 msgr2=0x7f75740a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.242+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 0x7f75740a38e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f758c19a200 tx=0x7f757c005960 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.242+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 msgr2=0x7f758c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.242+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f758000e9a0 tx=0x7f758000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 shutdown_connections 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75740a1420 0x7f75740a38e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f758c102940 0x7f758c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f758c100e30 0x7f758c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 --2- 172.21.15.5:0/1499200983 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f758c101f80 0x7f758c199270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 >> 172.21.15.5:0/1499200983 conn(0x7f758c0fc3a0 msgr2=0x7f758c107950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:18.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 shutdown_connections 2023-09-23T04:46:18.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.243+0000 7f75923ff700 1 -- 172.21.15.5:0/1499200983 wait complete. 2023-09-23T04:46:18.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-23T04:46:18.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 -- 172.21.15.5:0/186899869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 msgr2=0x7f8ec80732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 --2- 172.21.15.5:0/186899869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec80732b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009990 tx=0x7f8ebc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 -- 172.21.15.5:0/186899869 shutdown_connections 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 --2- 172.21.15.5:0/186899869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 --2- 172.21.15.5:0/186899869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec80732b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 --2- 172.21.15.5:0/186899869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 -- 172.21.15.5:0/186899869 >> 172.21.15.5:0/186899869 conn(0x7f8ec80fc3a0 msgr2=0x7f8ec80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:18.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 -- 172.21.15.5:0/186899869 shutdown_connections 2023-09-23T04:46:18.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.468+0000 7f8ecebff700 1 -- 172.21.15.5:0/186899869 wait complete. 2023-09-23T04:46:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.469+0000 7f8ecebff700 1 Processor -- start 2023-09-23T04:46:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.469+0000 7f8ecebff700 1 -- start start 2023-09-23T04:46:18.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec81a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec81a0870 con 0x7f8ec8108d10 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec81a09e0 con 0x7f8ec8074e60 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecebff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec81a0ba0 con 0x7f8ec8073880 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecd19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec81a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecc99b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ec7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecc99b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:60680/0 (socket says 172.21.15.5:60680) 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.470+0000 7f8ecc99b700 1 -- 172.21.15.5:0/2577673092 learned_addr learned my addr 172.21.15.5:0/2577673092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 -- 172.21.15.5:0/2577673092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 msgr2=0x7f8ec8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 -- 172.21.15.5:0/2577673092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 msgr2=0x7f8ec8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 -- 172.21.15.5:0/2577673092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0095f0 con 0x7f8ec8108d10 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecd19c700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec81a02a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000efc0 tx=0x7f8ec000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:18.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0018070 con 0x7f8ec8108d10 2023-09-23T04:46:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec81a0e20 con 0x7f8ec8108d10 2023-09-23T04:46:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ec000f040 con 0x7f8ec8108d10 2023-09-23T04:46:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000cd40 con 0x7f8ec8108d10 2023-09-23T04:46:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.471+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec81a11f0 con 0x7f8ec8108d10 2023-09-23T04:46:18.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.472+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec81073c0 con 0x7f8ec8108d10 2023-09-23T04:46:18.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.474+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8ec0003a00 con 0x7f8ec8108d10 2023-09-23T04:46:18.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.476+0000 7f8ec5ffb700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 0x7f8eb80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.476+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8ec00ccc60 con 0x7f8ec8108d10 2023-09-23T04:46:18.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.476+0000 7f8ecc99b700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 0x7f8eb80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.477+0000 7f8ecc99b700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 0x7f8eb80a3860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00b2e0 tx=0x7f8ebc005fb0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:18.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.480+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8ec0096090 con 0x7f8ec8108d10 2023-09-23T04:46:18.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.662+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 23) v1 ==== 136311+0+0 (secure 0 0 0) 0x7f8ec0003cb0 con 0x7f8ec8108d10 2023-09-23T04:46:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.754+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ec802d220 con 0x7f8eb80a13a0 2023-09-23T04:46:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.763+0000 7f8ec5ffb700 1 -- 172.21.15.5:0/2577673092 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f8ec802d220 con 0x7f8eb80a13a0 2023-09-23T04:46:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-09-23T00:00:00", "created": "2023-09-23T04:46:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 msgr2=0x7f8eb80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 0x7f8eb80a3860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00b2e0 tx=0x7f8ebc005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 msgr2=0x7f8ec81a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec81a02a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000efc0 tx=0x7f8ec000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 shutdown_connections 2023-09-23T04:46:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8eb80a13a0 0x7f8eb80a3860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ec8108d10 0x7f8ec81a02a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8ec8073880 0x7f8ec8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 --2- 172.21.15.5:0/2577673092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ec8074e60 0x7f8ec8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 >> 172.21.15.5:0/2577673092 conn(0x7f8ec80fc3a0 msgr2=0x7f8ec80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 shutdown_connections 2023-09-23T04:46:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.765+0000 7f8ecebff700 1 -- 172.21.15.5:0/2577673092 wait complete. 2023-09-23T04:46:18.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-23T04:46:18.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: [23/Sep/2023:04:46:17] ENGINE Bus STARTING 2023-09-23T04:46:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: [23/Sep/2023:04:46:17] ENGINE Serving on http://172.21.15.5:8765 2023-09-23T04:46:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: [23/Sep/2023:04:46:17] ENGINE Serving on https://172.21.15.5:7150 2023-09-23T04:46:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: [23/Sep/2023:04:46:17] ENGINE Bus STARTED 2023-09-23T04:46:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: from='client.25598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:18.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:18 smithi005 ceph-mon[97638]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: [23/Sep/2023:04:46:17] ENGINE Bus STARTING 2023-09-23T04:46:18.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: [23/Sep/2023:04:46:17] ENGINE Serving on http://172.21.15.5:8765 2023-09-23T04:46:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: [23/Sep/2023:04:46:17] ENGINE Serving on https://172.21.15.5:7150 2023-09-23T04:46:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: [23/Sep/2023:04:46:17] ENGINE Bus STARTED 2023-09-23T04:46:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: from='client.25598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:18.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:18 smithi118 ceph-mon[134297]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: [23/Sep/2023:04:46:17] ENGINE Bus STARTING 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: [23/Sep/2023:04:46:17] ENGINE Serving on http://172.21.15.5:8765 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: [23/Sep/2023:04:46:17] ENGINE Serving on https://172.21.15.5:7150 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: [23/Sep/2023:04:46:17] ENGINE Bus STARTED 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: from='client.25598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:18 smithi069 ceph-mon[133931]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 -- 172.21.15.5:0/4041753614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078101fa0 msgr2=0x7f5078102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 --2- 172.21.15.5:0/4041753614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078101fa0 0x7f5078102420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5064009990 tx=0x7f5064009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 -- 172.21.15.5:0/4041753614 shutdown_connections 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 --2- 172.21.15.5:0/4041753614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078102960 0x7f5078109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 --2- 172.21.15.5:0/4041753614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 0x7f5078101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 --2- 172.21.15.5:0/4041753614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078101fa0 0x7f5078102420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 -- 172.21.15.5:0/4041753614 >> 172.21.15.5:0/4041753614 conn(0x7f50780fc3a0 msgr2=0x7f50780fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 -- 172.21.15.5:0/4041753614 shutdown_connections 2023-09-23T04:46:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.995+0000 7f507ebff700 1 -- 172.21.15.5:0/4041753614 wait complete. 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 Processor -- start 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 -- start start 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 0x7f5078198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078101fa0 0x7f5078199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781a0870 con 0x7f5078101fa0 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781a09e0 con 0x7f5078100e50 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.996+0000 7f507ebff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781a0ba0 con 0x7f5078102960 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:60710/0 (socket says 172.21.15.5:60710) 2023-09-23T04:46:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 -- 172.21.15.5:0/3066072823 learned_addr learned my addr 172.21.15.5:0/3066072823 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f5077fff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078101fa0 0x7f5078199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507c99b700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 0x7f5078198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 -- 172.21.15.5:0/3066072823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 msgr2=0x7f5078198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 0x7f5078198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 -- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078101fa0 msgr2=0x7f5078199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078101fa0 0x7f5078199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 -- 172.21.15.5:0/3066072823 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c009520 con 0x7f5078102960 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507d19c700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f507000ef30 tx=0x7f507000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070018070 con 0x7f5078102960 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.998+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f507000f040 con 0x7f5078102960 2023-09-23T04:46:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.997+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640095f0 con 0x7f5078102960 2023-09-23T04:46:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.998+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000cd00 con 0x7f5078102960 2023-09-23T04:46:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.998+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50781a1150 con 0x7f5078102960 2023-09-23T04:46:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:18.999+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50781933e0 con 0x7f5078102960 2023-09-23T04:46:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.001+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 23) v1 ==== 136311+0+0 (secure 0 0 0) 0x7f5070003a00 con 0x7f5078102960 2023-09-23T04:46:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.002+0000 7f5075ffb700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 0x7f50680a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.002+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f50700cdbe0 con 0x7f5078102960 2023-09-23T04:46:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.002+0000 7f507c99b700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 0x7f50680a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.003+0000 7f507c99b700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 0x7f50680a3bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f507819a200 tx=0x7f506c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:19.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.005+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5070096ae0 con 0x7f5078102960 2023-09-23T04:46:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.268+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f507802d1b0 con 0x7f50680a1710 2023-09-23T04:46:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.276+0000 7f5075ffb700 1 -- 172.21.15.5:0/3066072823 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f507802d1b0 con 0x7f50680a1710 2023-09-23T04:46:19.276 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2023-09-23T04:46:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 msgr2=0x7f50680a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 0x7f50680a3bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f507819a200 tx=0x7f506c009260 comp rx=0 tx=0).stop 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 msgr2=0x7f50781a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f507000ef30 tx=0x7f507000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 shutdown_connections 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50680a1710 0x7f50680a3bd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5078102960 0x7f50781a02a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5078100e50 0x7f5078198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 --2- 172.21.15.5:0/3066072823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5078101fa0 0x7f5078199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 >> 172.21.15.5:0/3066072823 conn(0x7f50780fc3a0 msgr2=0x7f5078107960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 shutdown_connections 2023-09-23T04:46:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:19.278+0000 7f507ebff700 1 -- 172.21.15.5:0/3066072823 wait complete. 2023-09-23T04:46:19.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-23T04:46:19.401 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-23T04:46:19.414 INFO:teuthology.run_tasks:Running task workunit... 2023-09-23T04:46:19.454 INFO:tasks.workunit:Pulling workunits from ref 862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:46:19.454 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-09-23T04:46:19.454 INFO:tasks.workunit:timeout=3h 2023-09-23T04:46:19.454 INFO:tasks.workunit:cleanup=True 2023-09-23T04:46:19.454 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:cephfs_t:s0 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-09-23 04:44:20.824081067 +0000 2023-09-23T04:46:19.503 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-09-23 04:44:20.824081067 +0000 2023-09-23T04:46:19.504 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-09-23 04:45:55.866641433 +0000 2023-09-23T04:46:19.504 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-09-23 04:44:20.824081067 +0000 2023-09-23T04:46:19.504 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-09-23T04:46:19.504 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-09-23T04:46:19.590 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T04:46:19.639 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-09-23T04:46:19.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:19 smithi005 ceph-mon[97638]: from='client.25687 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:19.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:19 smithi005 ceph-mon[97638]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:19.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:19 smithi005 ceph-mon[97638]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-23T04:46:19.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:19 smithi118 ceph-mon[134297]: from='client.25687 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:19.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:19 smithi118 ceph-mon[134297]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:19.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:19 smithi118 ceph-mon[134297]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-23T04:46:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:19 smithi069 ceph-mon[133931]: from='client.25687 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:19 smithi069 ceph-mon[133931]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:19 smithi069 ceph-mon[133931]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='client.25705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:20.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='client.25705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='client.25705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:46:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:46:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:21.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:21.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:21 smithi069 ceph-mon[133931]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-23T04:46:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:21 smithi005 ceph-mon[97638]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:21 smithi118 ceph-mon[134297]: mgrmap e24: x(active, since 4s), standbys: y, z 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:22.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:22 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:46:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:22 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:22 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:23.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:23 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:23 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi005:/etc/ceph/ceph.conf 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi069:/etc/ceph/ceph.conf 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi118:/etc/ceph/ceph.conf 2023-09-23T04:46:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:23 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:25.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.conf 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: Updating smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: Updating smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:46:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:46:25.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:46:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:25 smithi069 ceph-mon[133931]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:25 smithi069 ceph-mon[133931]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-23T04:46:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:25 smithi005 ceph-mon[97638]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:25 smithi005 ceph-mon[97638]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-23T04:46:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:25 smithi118 ceph-mon[134297]: Updating smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/config/ceph.client.admin.keyring 2023-09-23T04:46:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:25 smithi118 ceph-mon[134297]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-09-23T04:46:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:27 smithi069 ceph-mon[133931]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-23T04:46:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:27 smithi005 ceph-mon[97638]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-23T04:46:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:27 smithi118 ceph-mon[134297]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-09-23T04:46:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:29 smithi069 ceph-mon[133931]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-23T04:46:30.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:29 smithi005 ceph-mon[97638]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-23T04:46:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:29 smithi118 ceph-mon[134297]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-09-23T04:46:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:31 smithi005 ceph-mon[97638]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:31 smithi118 ceph-mon[134297]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:31 smithi069 ceph-mon[133931]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:33 smithi069 ceph-mon[133931]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:34.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:33 smithi005 ceph-mon[97638]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:34.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:33 smithi118 ceph-mon[134297]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-09-23T04:46:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:35 smithi069 ceph-mon[133931]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-23T04:46:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:35 smithi005 ceph-mon[97638]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-23T04:46:36.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:35 smithi118 ceph-mon[134297]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-09-23T04:46:36.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 -- 172.21.15.5:0/3340424562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073250 msgr2=0x7f8574073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 --2- 172.21.15.5:0/3340424562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073250 0x7f8574073630 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8564009990 tx=0x7f8564009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 -- 172.21.15.5:0/3340424562 shutdown_connections 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 --2- 172.21.15.5:0/3340424562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f857410d6a0 0x7f857410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 --2- 172.21.15.5:0/3340424562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073250 0x7f8574073630 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 --2- 172.21.15.5:0/3340424562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8574073b70 0x7f857410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 -- 172.21.15.5:0/3340424562 >> 172.21.15.5:0/3340424562 conn(0x7f85740fc990 msgr2=0x7f85740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 -- 172.21.15.5:0/3340424562 shutdown_connections 2023-09-23T04:46:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.117+0000 7f85797ff700 1 -- 172.21.15.5:0/3340424562 wait complete. 2023-09-23T04:46:37.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 Processor -- start 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 -- start start 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8574073250 0x7f8574199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f857410d6a0 0x7f857419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857419e1f0 con 0x7f8574073b70 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857419e360 con 0x7f8574073250 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.118+0000 7f85797ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857419e520 con 0x7f857410d6a0 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58850/0 (socket says 172.21.15.5:58850) 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 -- 172.21.15.5:0/3993757453 learned_addr learned my addr 172.21.15.5:0/3993757453 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85737fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f857410d6a0 0x7f857419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f8572ffd700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8574073250 0x7f8574199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 -- 172.21.15.5:0/3993757453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f857410d6a0 msgr2=0x7f857419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f857410d6a0 0x7f857419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 -- 172.21.15.5:0/3993757453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8574073250 msgr2=0x7f8574199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8574073250 0x7f8574199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 -- 172.21.15.5:0/3993757453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640095f0 con 0x7f8574073b70 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f85727fc700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f856800d950 tx=0x7f856800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.119+0000 7f855bfff700 1 -- 172.21.15.5:0/3993757453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800f890 con 0x7f8574073b70 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.120+0000 7f855bfff700 1 -- 172.21.15.5:0/3993757453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8568010460 con 0x7f8574073b70 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.120+0000 7f855bfff700 1 -- 172.21.15.5:0/3993757453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800e610 con 0x7f8574073b70 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.120+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857419e800 con 0x7f8574073b70 2023-09-23T04:46:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.120+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85741a6040 con 0x7f8574073b70 2023-09-23T04:46:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.123+0000 7f855bfff700 1 -- 172.21.15.5:0/3993757453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f85680108d0 con 0x7f8574073b70 2023-09-23T04:46:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.124+0000 7f855bfff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 0x7f855c0a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.124+0000 7f858d046b80 1 Processor -- start 2023-09-23T04:46:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.124+0000 7f855bfff700 1 -- 172.21.15.5:0/3993757453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f85680cd680 con 0x7f8574073b70 2023-09-23T04:46:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f8572ffd700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 0x7f855c0a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 -- start start 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360010b460 con 0x5636001a2400 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636000fcc20 con 0x5636001adbd0 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f858d046b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636000fa540 con 0x5636001a2d20 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f8572ffd700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 0x7f855c0a3c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f856400b5c0 tx=0x7f8564005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85737fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58852/0 (socket says 172.21.15.5:58852) 2023-09-23T04:46:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f8572ffd700 1 -- 172.21.15.5:0/3345922280 learned_addr learned my addr 172.21.15.5:0/3345922280 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.125+0000 7f85727fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 -- 172.21.15.5:0/3345922280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 msgr2=0x5636001affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 -- 172.21.15.5:0/3345922280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 msgr2=0x5636001a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 -- 172.21.15.5:0/3345922280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636001588d0 con 0x5636001a2d20 2023-09-23T04:46:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85727fc700 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001ad690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85680ca8d0 tx=0x7f8568099aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48cd8885b847185d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85597fa700 1 -- 172.21.15.5:0/3345922280 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856809e190 con 0x5636001a2d20 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85597fa700 1 -- 172.21.15.5:0/3345922280 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85680a23d0 con 0x5636001a2d20 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.126+0000 7f85597fa700 1 -- 172.21.15.5:0/3345922280 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856809a500 con 0x5636001a2d20 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 -- 172.21.15.5:0/3345922280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 msgr2=0x5636001ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001ad690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85680ca8d0 tx=0x7f8568099aa0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 -- 172.21.15.5:0/3345922280 shutdown_connections 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 --2- 172.21.15.5:0/3345922280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001ad690 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 -- 172.21.15.5:0/3345922280 >> 172.21.15.5:0/3345922280 conn(0x563600117260 msgr2=0x56360019e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 -- 172.21.15.5:0/3345922280 shutdown_connections 2023-09-23T04:46:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.127+0000 7f858d046b80 1 -- 172.21.15.5:0/3345922280 wait complete. 2023-09-23T04:46:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.128+0000 7f858d046b80 1 -- start start 2023-09-23T04:46:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.128+0000 7f858d046b80 1 Processor -- start 2023-09-23T04:46:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2400 0x5636001aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636000f5e30 con 0x5636001a2d20 2023-09-23T04:46:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563600092710 con 0x5636001adbd0 2023-09-23T04:46:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636000923b0 con 0x5636001a2400 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f858d046b80 10 client.0 _ll_register_callbacks cb 0x5635ffeef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f85727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2400 0x5636001aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f85737fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f85727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58854/0 (socket says 172.21.15.5:58854) 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.129+0000 7f85727fc700 1 -- 172.21.15.5:0/182236219 learned_addr learned my addr 172.21.15.5:0/182236219 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 -- 172.21.15.5:0/182236219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2400 msgr2=0x5636001aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2400 0x5636001aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 -- 172.21.15.5:0/182236219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 msgr2=0x5636001ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 -- 172.21.15.5:0/182236219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85741a6040 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f85727fc700 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f856800b2f0 tx=0x7f85680cae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c1710fc6cad45 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f8556ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856809e190 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563600196250 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f85680983e0 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.130+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85680cd220 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 10 client.15681 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5635fff20de0 con 0x5636001a2d20 2023-09-23T04:46:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563600196250 con 0x5636001a2d20 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f856809a2a0 con 0x5636001a2d20 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 20 client.15681 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5636001bd1e0 0x5636001bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 4 client.15681 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636000eee10 con 0x5636001bd1e0 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.131+0000 7f8572ffd700 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5636001bd1e0 0x5636001bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.132+0000 7f8572ffd700 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5636001bd1e0 0x5636001bf5d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72429a631c430710 server_cookie=399d8258c13bf720 in_seq=0 out_seq=0 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.132+0000 7f8556ff5700 10 client.15681 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f8556ff5700 1 -- 172.21.15.5:0/182236219 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5636000eee10 con 0x5636001bd1e0 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f8556ff5700 10 client.15681 handle_command_reply: tid=0 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 client.15681 shutdown 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 msgr2=0x5636001aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f856800b2f0 tx=0x7f85680cae30 comp rx=0 tx=0).stop 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 shutdown_connections 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5636001bd1e0 0x5636001bf5d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2400 0x5636001aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f8556ff5700 0 client.15681 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 --2- 172.21.15.5:0/182236219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2d20 0x5636001aaed0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 >> 172.21.15.5:0/182236219 conn(0x563600117260 msgr2=0x56360019e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 shutdown_connections 2023-09-23T04:46:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 1 -- 172.21.15.5:0/182236219 wait complete. 2023-09-23T04:46:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.133+0000 7f858d046b80 20 client.15681 trim_cache size 0 max 16384 2023-09-23T04:46:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 Processor -- start 2023-09-23T04:46:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 -- start start 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 0x5636001b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 0x5636001a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635fff76920 con 0x5636001a2400 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636001a68d0 con 0x5636001a2d20 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.151+0000 7f858d046b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636001b4cd0 con 0x5636001adbd0 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58858/0 (socket says 172.21.15.5:58858) 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 -- 172.21.15.5:0/1420433934 learned_addr learned my addr 172.21.15.5:0/1420433934 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f85737fe700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 0x5636001a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f85727fc700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 0x5636001b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 -- 172.21.15.5:0/1420433934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 msgr2=0x5636001a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 0x5636001a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 -- 172.21.15.5:0/1420433934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 msgr2=0x5636001b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 0x5636001b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 -- 172.21.15.5:0/1420433934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635fff09c60 con 0x5636001a2400 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f85727fc700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 0x5636001b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f85737fe700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 0x5636001a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.152+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8564000c00 tx=0x7f8564022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce8be78d4c1560f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f8559ffb700 1 -- 172.21.15.5:0/1420433934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564034a90 con 0x5636001a2400 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f8559ffb700 1 -- 172.21.15.5:0/1420433934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856401e040 con 0x5636001a2400 2023-09-23T04:46:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f8559ffb700 1 -- 172.21.15.5:0/1420433934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85640055d0 con 0x5636001a2400 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 -- 172.21.15.5:0/1420433934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 msgr2=0x5636001a9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8564000c00 tx=0x7f8564022000 comp rx=0 tx=0).stop 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 -- 172.21.15.5:0/1420433934 shutdown_connections 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001adbd0 0x5636001a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001a9c20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 --2- 172.21.15.5:0/1420433934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001a2d20 0x5636001b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 -- 172.21.15.5:0/1420433934 >> 172.21.15.5:0/1420433934 conn(0x563600117260 msgr2=0x5636001a5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.153+0000 7f858d046b80 1 -- 172.21.15.5:0/1420433934 shutdown_connections 2023-09-23T04:46:37.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.154+0000 7f858d046b80 1 -- 172.21.15.5:0/1420433934 wait complete. 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.154+0000 7f858d046b80 1 -- start start 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.154+0000 7f858d046b80 1 Processor -- start 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636001b3230 con 0x5636001a2400 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636001b3370 con 0x5636001adbd0 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636001b3500 con 0x5636001a2d20 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f858d046b80 10 client.0 _ll_register_callbacks cb 0x7f8564005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58864/0 (socket says 172.21.15.5:58864) 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 -- 172.21.15.5:0/1767801306 learned_addr learned my addr 172.21.15.5:0/1767801306 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f85727fc700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f85737fe700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 -- 172.21.15.5:0/1767801306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 msgr2=0x5636001b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 -- 172.21.15.5:0/1767801306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 msgr2=0x5636001b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f8572ffd700 1 -- 172.21.15.5:0/1767801306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c009bb0 con 0x5636001a2400 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.155+0000 7f85737fe700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8572ffd700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8564005f50 tx=0x7f856400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a6636166f70a53 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8556ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:37.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564034c50 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85680a23d0 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8564034380 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8564034540 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f858d046b80 10 client.15687 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f856402b040 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.156+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85680a23d0 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f856400fbe0 con 0x5636001a2400 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 20 client.15687 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563600212220 0x563600214640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 4 client.15687 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636000eeb30 con 0x563600212220 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f85727fc700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563600212220 0x563600214640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f85727fc700 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563600212220 0x563600214640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921647ab211f99d2 server_cookie=3f6c50b6c29dbe5 in_seq=0 out_seq=0 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.157+0000 7f8556ff5700 10 client.15687 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f8556ff5700 1 -- 172.21.15.5:0/1767801306 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5636000eeb30 con 0x563600212220 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f8556ff5700 10 client.15687 handle_command_reply: tid=0 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 client.15687 shutdown 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 msgr2=0x5636001b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8564005f50 tx=0x7f856400b670 comp rx=0 tx=0).stop 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 shutdown_connections 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563600212220 0x563600214640 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636001adbd0 0x5636001b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636001a2400 0x5636001b4470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 --2- 172.21.15.5:0/1767801306 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5636001a2d20 0x5636001b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f8556ff5700 0 client.15687 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 >> 172.21.15.5:0/1767801306 conn(0x563600117260 msgr2=0x5636001aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 shutdown_connections 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 1 -- 172.21.15.5:0/1767801306 wait complete. 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.158+0000 7f858d046b80 20 client.15687 trim_cache size 0 max 16384 2023-09-23T04:46:37.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:46:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:46:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:46:37.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:46:37.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 msgr2=0x7f855c0a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 0x7f855c0a3c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f856400b5c0 tx=0x7f8564005fd0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 msgr2=0x7f8574199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f856800d950 tx=0x7f856800dc60 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 shutdown_connections 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f855c0a17a0 0x7f855c0a3c60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f857410d6a0 0x7f857419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8574073250 0x7f8574199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 --2- 172.21.15.5:0/3993757453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8574073b70 0x7f8574199750 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 >> 172.21.15.5:0/3993757453 conn(0x7f85740fc990 msgr2=0x7f857410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 shutdown_connections 2023-09-23T04:46:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.159+0000 7f85797ff700 1 -- 172.21.15.5:0/3993757453 wait complete. 2023-09-23T04:46:37.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:46:37.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 -- 172.21.15.5:0/345407902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1624101a60 msgr2=0x7f1624105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 --2- 172.21.15.5:0/345407902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1624101a60 0x7f1624105aa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1618009990 tx=0x7f1618009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 -- 172.21.15.5:0/345407902 shutdown_connections 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 --2- 172.21.15.5:0/345407902 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 0x7f162410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 --2- 172.21.15.5:0/345407902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16241010b0 0x7f1624101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 --2- 172.21.15.5:0/345407902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1624101a60 0x7f1624105aa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 -- 172.21.15.5:0/345407902 >> 172.21.15.5:0/345407902 conn(0x7f16240fc930 msgr2=0x7f16240fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 -- 172.21.15.5:0/345407902 shutdown_connections 2023-09-23T04:46:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.382+0000 7f1629fff700 1 -- 172.21.15.5:0/345407902 wait complete. 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 Processor -- start 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 -- start start 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 0x7f1624197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162419b990 0x7f162419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419c410 con 0x7f16241010b0 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419c580 con 0x7f162419b990 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.383+0000 7f1629fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419c740 con 0x7f16241061d0 2023-09-23T04:46:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58882/0 (socket says 172.21.15.5:58882) 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 -- 172.21.15.5:0/3844389177 learned_addr learned my addr 172.21.15.5:0/3844389177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f1622ffd700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 0x7f1624197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f1623fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162419b990 0x7f162419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 -- 172.21.15.5:0/3844389177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 msgr2=0x7f1624197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 0x7f1624197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 -- 172.21.15.5:0/3844389177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162419b990 msgr2=0x7f162419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162419b990 0x7f162419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 -- 172.21.15.5:0/3844389177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16180095f0 con 0x7f16241010b0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.384+0000 7f16237fe700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9a0 tx=0x7f161400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.385+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614009790 con 0x7f16241010b0 2023-09-23T04:46:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.385+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16241a3ca0 con 0x7f16241010b0 2023-09-23T04:46:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.385+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1614010430 con 0x7f16241010b0 2023-09-23T04:46:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.385+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614018680 con 0x7f16241010b0 2023-09-23T04:46:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.385+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16241a4180 con 0x7f16241010b0 2023-09-23T04:46:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.386+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624067ca0 con 0x7f16241010b0 2023-09-23T04:46:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.388+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f16140105a0 con 0x7f16241010b0 2023-09-23T04:46:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.390+0000 7f1620ff9700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 0x7f160c0a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.390+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1614014070 con 0x7f16241010b0 2023-09-23T04:46:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.390+0000 7f1622ffd700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 0x7f160c0a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.391+0000 7f1622ffd700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 0x7f160c0a3c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1624198600 tx=0x7f1618005fb0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.394+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1614051ec0 con 0x7f16241010b0 2023-09-23T04:46:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.653+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1624198340 con 0x7f16241010b0 2023-09-23T04:46:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.654+0000 7f1620ff9700 1 -- 172.21.15.5:0/3844389177 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f1614096770 con 0x7f16241010b0 2023-09-23T04:46:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:46:37.655 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 msgr2=0x7f160c0a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 0x7f160c0a3c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1624198600 tx=0x7f1618005fb0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 msgr2=0x7f1624196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9a0 tx=0x7f161400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 shutdown_connections 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f160c0a17a0 0x7f160c0a3c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f162419b990 0x7f162419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16241010b0 0x7f1624196ff0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 --2- 172.21.15.5:0/3844389177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16241061d0 0x7f1624197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 >> 172.21.15.5:0/3844389177 conn(0x7f16240fc930 msgr2=0x7f16240fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 shutdown_connections 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.656+0000 7f1629fff700 1 -- 172.21.15.5:0/3844389177 wait complete. 2023-09-23T04:46:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:46:37.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:46:37.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 -- 172.21.15.5:0/377036235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe518101a80 msgr2=0x7fe518105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 --2- 172.21.15.5:0/377036235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe518101a80 0x7fe518105ac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009990 tx=0x7fe50c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 -- 172.21.15.5:0/377036235 shutdown_connections 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 --2- 172.21.15.5:0/377036235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5181061f0 0x7fe51810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 --2- 172.21.15.5:0/377036235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe5181014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 --2- 172.21.15.5:0/377036235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe518101a80 0x7fe518105ac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.882+0000 7fe51cdff700 1 -- 172.21.15.5:0/377036235 >> 172.21.15.5:0/377036235 conn(0x7fe5180fc990 msgr2=0x7fe5180fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.883+0000 7fe51cdff700 1 -- 172.21.15.5:0/377036235 shutdown_connections 2023-09-23T04:46:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.883+0000 7fe51cdff700 1 -- 172.21.15.5:0/377036235 wait complete. 2023-09-23T04:46:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.884+0000 7fe51cdff700 1 Processor -- start 2023-09-23T04:46:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.884+0000 7fe51cdff700 1 -- start start 2023-09-23T04:46:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51819b910 0x7fe51819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819c390 con 0x7fe5181061f0 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819c500 con 0x7fe51819b910 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51cdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51819c6c0 con 0x7fe5181010d0 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe515d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:50902/0 (socket says 172.21.15.5:50902) 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe515d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58898/0 (socket says 172.21.15.5:58898) 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe51659c700 1 -- 172.21.15.5:0/2251315889 learned_addr learned my addr 172.21.15.5:0/2251315889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.885+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51819b910 0x7fe51819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 -- 172.21.15.5:0/2251315889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 msgr2=0x7fe518196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 -- 172.21.15.5:0/2251315889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51819b910 msgr2=0x7fe51819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51819b910 0x7fe51819bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 -- 172.21.15.5:0/2251315889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0095f0 con 0x7fe5181061f0 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe51659c700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0059d0 tx=0x7fe50c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe5077fe700 1 -- 172.21.15.5:0/2251315889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c01d070 con 0x7fe5181061f0 2023-09-23T04:46:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5181a3bc0 con 0x7fe5181061f0 2023-09-23T04:46:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe5077fe700 1 -- 172.21.15.5:0/2251315889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c022470 con 0x7fe5181061f0 2023-09-23T04:46:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe5077fe700 1 -- 172.21.15.5:0/2251315889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c017650 con 0x7fe5181061f0 2023-09-23T04:46:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.886+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5181a3ff0 con 0x7fe5181061f0 2023-09-23T04:46:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.890+0000 7fe5077fe700 1 -- 172.21.15.5:0/2251315889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7fe50c0225e0 con 0x7fe5181061f0 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.891+0000 7fe5306e8b80 1 Processor -- start 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.891+0000 7fe5077fe700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 0x7fe5000a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.891+0000 7fe5077fe700 1 -- 172.21.15.5:0/2251315889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe50c0cea40 con 0x7fe5181061f0 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.891+0000 7fe5306e8b80 1 -- start start 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.891+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 0x7fe5000a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92aca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92a21460 con 0x55ec92ac7fe0 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92a12c20 con 0x55ec92ab8d20 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92a10540 con 0x55ec92ab8400 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe51659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe51659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:50904/0 (socket says 172.21.15.5:50904) 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe51659c700 1 -- 172.21.15.5:0/2960054572 learned_addr learned my addr 172.21.15.5:0/2960054572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 0x7fe5000a3c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe51000f540 tx=0x7fe51000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92aca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe515d9b700 1 -- 172.21.15.5:0/2960054572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 msgr2=0x55ec92ab87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.892+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe515d9b700 1 -- 172.21.15.5:0/2960054572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 msgr2=0x55ec92aca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92aca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe515d9b700 1 -- 172.21.15.5:0/2960054572 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec92a6e8d0 con 0x55ec92ab8d20 2023-09-23T04:46:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe51659c700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe515d9b700 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac7aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0cbc90 tx=0x7fe50c09b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c2f9e21a3eb094 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe504ff9700 1 -- 172.21.15.5:0/2960054572 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c09bb90 con 0x55ec92ab8d20 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe504ff9700 1 -- 172.21.15.5:0/2960054572 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c09b490 con 0x55ec92ab8d20 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.893+0000 7fe504ff9700 1 -- 172.21.15.5:0/2960054572 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c09b930 con 0x55ec92ab8d20 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2960054572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 msgr2=0x55ec92ac7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac7aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0cbc90 tx=0x7fe50c09b260 comp rx=0 tx=0).stop 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2960054572 shutdown_connections 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92aca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8400 0x55ec92ab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2960054572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac7aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2960054572 >> 172.21.15.5:0/2960054572 conn(0x55ec92a2d260 msgr2=0x55ec92ab4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2960054572 shutdown_connections 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2960054572 wait complete. 2023-09-23T04:46:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.894+0000 7fe5306e8b80 1 -- start start 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 Processor -- start 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92ac83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8d20 0x55ec92ac8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92a0be30 con 0x55ec92ac7fe0 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec929a8710 con 0x55ec92ab8400 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec929a83b0 con 0x55ec92ab8d20 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe5306e8b80 10 client.0 _ll_register_callbacks cb 0x55ec92805800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe516d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe51659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92ac83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe516d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58910/0 (socket says 172.21.15.5:58910) 2023-09-23T04:46:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe515d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8d20 0x55ec92ac8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe516d9d700 1 -- 172.21.15.5:0/2843537244 learned_addr learned my addr 172.21.15.5:0/2843537244 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.895+0000 7fe51659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92ac83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48460/0 (socket says 172.21.15.5:48460) 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 -- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8d20 msgr2=0x55ec92ac8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8d20 0x55ec92ac8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 -- 172.21.15.5:0/2843537244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 msgr2=0x55ec92ac83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92ac83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 -- 172.21.15.5:0/2843537244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe508011440 con 0x55ec92ac7fe0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe516d9d700 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe510008ac0 tx=0x7fe510004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f482a5d608a43ecd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe4fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000e040 con 0x55ec92ac7fe0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec92aac250 con 0x55ec92ac7fe0 2023-09-23T04:46:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.896+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe51001c040 con 0x55ec92ac7fe0 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5100093e0 con 0x55ec92ac7fe0 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 10 client.15705 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe5181a3ff0 con 0x55ec92ac7fe0 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec92aac250 con 0x55ec92ac7fe0 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe510005400 con 0x55ec92ac7fe0 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 10 client.15705 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 20 client.15705 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92ad7710 0x55ec92ad9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 4 client.15705 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.897+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec92a04e10 con 0x55ec92ad7710 2023-09-23T04:46:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.898+0000 7fe51659c700 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92ad7710 0x55ec92ad9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.898+0000 7fe4fdffb700 10 client.15705 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.898+0000 7fe51659c700 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92ad7710 0x55ec92ad9b30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1eca6b777b819847 server_cookie=9cdadc2fd5f2a250 in_seq=0 out_seq=0 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe4fdffb700 1 -- 172.21.15.5:0/2843537244 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec92a04e10 con 0x55ec92ad7710 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe4fdffb700 10 client.15705 handle_command_reply: tid=0 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 client.15705 shutdown 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 msgr2=0x55ec92ac8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe510008ac0 tx=0x7fe510004ad0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 shutdown_connections 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92ad7710 0x55ec92ad9b30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac8e80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92ac83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/2843537244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ab8d20 0x55ec92ac8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe4fdffb700 0 client.15705 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:46:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 >> 172.21.15.5:0/2843537244 conn(0x55ec92a2d260 msgr2=0x55ec92ab4380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 shutdown_connections 2023-09-23T04:46:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.899+0000 7fe5306e8b80 1 -- 172.21.15.5:0/2843537244 wait complete. 2023-09-23T04:46:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.900+0000 7fe5306e8b80 20 client.15705 trim_cache size 0 max 16384 2023-09-23T04:46:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.913+0000 7fe5306e8b80 1 Processor -- start 2023-09-23T04:46:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.913+0000 7fe5306e8b80 1 -- start start 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9288c920 con 0x55ec92ab8400 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92ac1840 con 0x55ec92ab8d20 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92ac20c0 con 0x55ec92ac7fe0 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58920/0 (socket says 172.21.15.5:58920) 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 -- 172.21.15.5:0/1964917841 learned_addr learned my addr 172.21.15.5:0/1964917841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe515d9b700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe516d9d700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 -- 172.21.15.5:0/1964917841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 msgr2=0x55ec92ac12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 -- 172.21.15.5:0/1964917841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 msgr2=0x55ec92ac6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe51659c700 1 -- 172.21.15.5:0/1964917841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec92ab0480 con 0x55ec92ab8400 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.914+0000 7fe515d9b700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe51659c700 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55ec92ada400 tx=0x7fe508009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dcf1c8f99b8954 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe5057fa700 1 -- 172.21.15.5:0/1964917841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508017030 con 0x55ec92ab8400 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe5057fa700 1 -- 172.21.15.5:0/1964917841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe508018040 con 0x55ec92ab8400 2023-09-23T04:46:37.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe5057fa700 1 -- 172.21.15.5:0/1964917841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508004d20 con 0x55ec92ab8400 2023-09-23T04:46:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe5306e8b80 1 -- 172.21.15.5:0/1964917841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 msgr2=0x55ec92acc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.915+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55ec92ada400 tx=0x7fe508009260 comp rx=0 tx=0).stop 2023-09-23T04:46:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 -- 172.21.15.5:0/1964917841 shutdown_connections 2023-09-23T04:46:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92ac12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8400 0x55ec92acc3f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/1964917841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8d20 0x55ec92ac6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 -- 172.21.15.5:0/1964917841 >> 172.21.15.5:0/1964917841 conn(0x55ec92a2d260 msgr2=0x55ec92ac2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 -- 172.21.15.5:0/1964917841 shutdown_connections 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 -- 172.21.15.5:0/1964917841 wait complete. 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 -- start start 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.916+0000 7fe5306e8b80 1 Processor -- start 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92abadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92abf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92abf930 con 0x55ec92ab8d20 2023-09-23T04:46:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92abfaa0 con 0x55ec92ab8400 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec92abfc60 con 0x55ec92ac7fe0 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe5306e8b80 10 client.0 _ll_register_callbacks cb 0x7fe51000ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.917+0000 7fe515d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe51659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92abadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 -- 172.21.15.5:0/3869641278 learned_addr learned my addr 172.21.15.5:0/3869641278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe516d9d700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92abf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 -- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 msgr2=0x55ec92abf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92abf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 -- 172.21.15.5:0/3869641278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 msgr2=0x55ec92abadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92abadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 -- 172.21.15.5:0/3869641278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe508004520 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe515d9b700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe50c097990 tx=0x7fe50c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121228b50414b782 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe4fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0b7070 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe50c09bb90 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.918+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe50c09b490 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe50c009290 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 10 client.15711 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec92836de0 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c080020 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe50c0b3d20 con 0x55ec92ab8d20 2023-09-23T04:46:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 10 client.15711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 20 client.15711 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92b30760 0x55ec92b32b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 4 client.15711 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec92a04b30 con 0x55ec92b30760 2023-09-23T04:46:37.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.919+0000 7fe51659c700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92b30760 0x55ec92b32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe51659c700 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92b30760 0x55ec92b32b80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0530af4b69fae5a server_cookie=b0ccfb807ff63730 in_seq=0 out_seq=0 2023-09-23T04:46:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe4fdffb700 10 client.15711 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:46:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe4fdffb700 1 -- 172.21.15.5:0/3869641278 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec92a04b30 con 0x55ec92b30760 2023-09-23T04:46:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe4fdffb700 10 client.15711 handle_command_reply: tid=0 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 client.15711 shutdown 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 msgr2=0x55ec92abb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe50c097990 tx=0x7fe50c005ef0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 shutdown_connections 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ec92ac7fe0 0x55ec92abf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ec92ab8400 0x55ec92abadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55ec92b30760 0x55ec92b32b80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 --2- 172.21.15.5:0/3869641278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec92ab8d20 0x55ec92abb2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe4fdffb700 0 client.15711 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.920+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 >> 172.21.15.5:0/3869641278 conn(0x55ec92a2d260 msgr2=0x55ec92ab39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 shutdown_connections 2023-09-23T04:46:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe5306e8b80 1 -- 172.21.15.5:0/3869641278 wait complete. 2023-09-23T04:46:37.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe5306e8b80 20 client.15711 trim_cache size 0 max 16384 2023-09-23T04:46:37.927 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:46:37.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 msgr2=0x7fe5000a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 0x7fe5000a3c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe51000f540 tx=0x7fe51000b040 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 msgr2=0x7fe5181974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0059d0 tx=0x7fe50c004730 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 shutdown_connections 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe5000a17a0 0x7fe5000a3c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe51819b910 0x7fe51819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe5181010d0 0x7fe518196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 --2- 172.21.15.5:0/2251315889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5181061f0 0x7fe5181974b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.921+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 >> 172.21.15.5:0/2251315889 conn(0x7fe5180fc990 msgr2=0x7fe5180fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.922+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 shutdown_connections 2023-09-23T04:46:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:37.922+0000 7fe51cdff700 1 -- 172.21.15.5:0/2251315889 wait complete. 2023-09-23T04:46:37.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:37 smithi005 ceph-mon[97638]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:37.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:37 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3844389177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:37.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:46:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:37 smithi069 ceph-mon[133931]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:37 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3844389177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 -- 172.21.15.5:0/3773578672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 msgr2=0x7f839010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 --2- 172.21.15.5:0/3773578672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f839010dd70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f838800b0a0 tx=0x7f838800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 -- 172.21.15.5:0/3773578672 shutdown_connections 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 --2- 172.21.15.5:0/3773578672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f839010dd70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 --2- 172.21.15.5:0/3773578672 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f83901010d0 0x7f83901014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 --2- 172.21.15.5:0/3773578672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8390101a80 0x7f8390105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 -- 172.21.15.5:0/3773578672 >> 172.21.15.5:0/3773578672 conn(0x7f83900fc990 msgr2=0x7f83900fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 -- 172.21.15.5:0/3773578672 shutdown_connections 2023-09-23T04:46:38.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.154+0000 7f83981ff700 1 -- 172.21.15.5:0/3773578672 wait complete. 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 Processor -- start 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 -- start start 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83901010d0 0x7f8390199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f839019dc30 0x7f839019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839019e6b0 con 0x7f83901061f0 2023-09-23T04:46:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839019e820 con 0x7f83901010d0 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.155+0000 7f83981ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839019e9e0 con 0x7f839019dc30 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 -- 172.21.15.5:0/812785275 learned_addr learned my addr 172.21.15.5:0/812785275 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839679c700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f839019dc30 0x7f839019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f8395f9b700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83901010d0 0x7f8390199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 -- 172.21.15.5:0/812785275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f839019dc30 msgr2=0x7f839019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f839019dc30 0x7f839019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 -- 172.21.15.5:0/812785275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83901010d0 msgr2=0x7f8390199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83901010d0 0x7f8390199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 -- 172.21.15.5:0/812785275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388009d00 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.156+0000 7f839579a700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f838000e970 tx=0x7f838000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.157+0000 7f8386ffd700 1 -- 172.21.15.5:0/812785275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380009790 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.157+0000 7f8386ffd700 1 -- 172.21.15.5:0/812785275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f838000c7e0 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.157+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901a5f40 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.157+0000 7f8386ffd700 1 -- 172.21.15.5:0/812785275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800054e0 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.157+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83901a6410 con 0x7f83901061f0 2023-09-23T04:46:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:37 smithi118 ceph-mon[134297]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:37 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3844389177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:46:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.160+0000 7f8386ffd700 1 -- 172.21.15.5:0/812785275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f838000c950 con 0x7f83901061f0 2023-09-23T04:46:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.161+0000 7f83abbccb80 1 Processor -- start 2023-09-23T04:46:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.161+0000 7f83abbccb80 1 -- start start 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f8386ffd700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 0x7f837c0a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f77467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7751bd0 0x55b3f7753fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f76af460 con 0x55b3f7746d20 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f76a0c20 con 0x55b3f7751bd0 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f83abbccb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f769e540 con 0x55b3f7746400 2023-09-23T04:46:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f8386ffd700 1 -- 172.21.15.5:0/812785275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8380014070 con 0x7f83901061f0 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f839679c700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 0x7f837c0a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58964/0 (socket says 172.21.15.5:58964) 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f839579a700 1 -- 172.21.15.5:0/3122223196 learned_addr learned my addr 172.21.15.5:0/3122223196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f8395f9b700 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f77467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.162+0000 7f839679c700 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7751bd0 0x55b3f7753fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 -- 172.21.15.5:0/3122223196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 msgr2=0x55b3f77467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f77467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 -- 172.21.15.5:0/3122223196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7751bd0 msgr2=0x55b3f7753fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7751bd0 0x55b3f7753fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 -- 172.21.15.5:0/3122223196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f76fc8d0 con 0x55b3f7746d20 2023-09-23T04:46:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839579a700 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f838009a350 tx=0x7f8380099f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb3aa615a1fe953 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f839679c700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 0x7f837c0a3ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8388006130 tx=0x7f8388009480 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f837bfff700 1 -- 172.21.15.5:0/3122223196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380077870 con 0x55b3f7746d20 2023-09-23T04:46:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f837bfff700 1 -- 172.21.15.5:0/3122223196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8380077690 con 0x55b3f7746d20 2023-09-23T04:46:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.163+0000 7f837bfff700 1 -- 172.21.15.5:0/3122223196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800774f0 con 0x55b3f7746d20 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 -- 172.21.15.5:0/3122223196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 msgr2=0x55b3f7751690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f838009a350 tx=0x7f8380099f80 comp rx=0 tx=0).stop 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 -- 172.21.15.5:0/3122223196 shutdown_connections 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7751bd0 0x55b3f7753fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f77467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 --2- 172.21.15.5:0/3122223196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7751690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 -- 172.21.15.5:0/3122223196 >> 172.21.15.5:0/3122223196 conn(0x55b3f76bb260 msgr2=0x55b3f7742790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 -- 172.21.15.5:0/3122223196 shutdown_connections 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.164+0000 7f83abbccb80 1 -- 172.21.15.5:0/3122223196 wait complete. 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 -- start start 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 Processor -- start 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746d20 0x55b3f774eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7751bd0 0x55b3f774f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f7699e30 con 0x55b3f7751bd0 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f7636710 con 0x55b3f7746d20 2023-09-23T04:46:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f76363b0 con 0x55b3f7746400 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.165+0000 7f83abbccb80 10 client.0 _ll_register_callbacks cb 0x55b3f7493800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:50972/0 (socket says 172.21.15.5:50972) 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 -- 172.21.15.5:0/2052919407 learned_addr learned my addr 172.21.15.5:0/2052919407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f839679c700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7751bd0 0x55b3f774f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f839579a700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746d20 0x55b3f774eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 -- 172.21.15.5:0/2052919407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746d20 msgr2=0x55b3f774eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746d20 0x55b3f774eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 -- 172.21.15.5:0/2052919407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7751bd0 msgr2=0x55b3f774f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7751bd0 0x55b3f774f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 -- 172.21.15.5:0/2052919407 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83901a6410 con 0x55b3f7746400 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f8395f9b700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838c011730 tx=0x7f838c011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa94f13806801e5a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.166+0000 7f83797fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c016070 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3f773a250 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f838c00f6f0 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838c00f8f0 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83abbccb80 10 client.25759 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b3f74c4de0 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.167+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3f773a250 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f838c01abb0 con 0x55b3f7746400 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 10 client.25759 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 10 client.25759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 20 client.25759 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77616c0 0x55b3f7763ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 4 client.25759 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3f7692e10 con 0x55b3f77616c0 2023-09-23T04:46:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.168+0000 7f839579a700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77616c0 0x55b3f7763ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.169+0000 7f839579a700 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77616c0 0x55b3f7763ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2251442a81601a49 server_cookie=848d7d98382d6972 in_seq=0 out_seq=0 2023-09-23T04:46:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.169+0000 7f83797fa700 10 client.25759 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:46:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.169+0000 7f83797fa700 1 -- 172.21.15.5:0/2052919407 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3f7692e10 con 0x55b3f77616c0 2023-09-23T04:46:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.169+0000 7f83797fa700 10 client.25759 handle_command_reply: tid=0 2023-09-23T04:46:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.169+0000 7f83abbccb80 1 client.25759 shutdown 2023-09-23T04:46:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 msgr2=0x55b3f774e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838c011730 tx=0x7f838c011a40 comp rx=0 tx=0).stop 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 shutdown_connections 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77616c0 0x55b3f7763ae0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7751bd0 0x55b3f774f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7746400 0x55b3f774e9b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 --2- 172.21.15.5:0/2052919407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746d20 0x55b3f774eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83797fa700 0 client.25759 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 >> 172.21.15.5:0/2052919407 conn(0x55b3f76bb260 msgr2=0x55b3f7742320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 shutdown_connections 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 1 -- 172.21.15.5:0/2052919407 wait complete. 2023-09-23T04:46:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.170+0000 7f83abbccb80 20 client.25759 trim_cache size 0 max 16384 2023-09-23T04:46:38.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 Processor -- start 2023-09-23T04:46:38.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 -- start start 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7749470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f774f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f751a920 con 0x55b3f7746d20 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f77435b0 con 0x55b3f7746400 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f83abbccb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f7758c50 con 0x55b3f7751bd0 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58986/0 (socket says 172.21.15.5:58986) 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f839579a700 1 -- 172.21.15.5:0/1406629726 learned_addr learned my addr 172.21.15.5:0/1406629726 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.184+0000 7f8395f9b700 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7749470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839679c700 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f774f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 -- 172.21.15.5:0/1406629726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 msgr2=0x55b3f774f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f774f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 -- 172.21.15.5:0/1406629726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 msgr2=0x55b3f7749470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7749470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 -- 172.21.15.5:0/1406629726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3f773e480 con 0x55b3f7746d20 2023-09-23T04:46:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f839579a700 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f838009a070 tx=0x7f8380005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e612ea55af653b59 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f8384ff9700 1 -- 172.21.15.5:0/1406629726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800b2070 con 0x55b3f7746d20 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f8384ff9700 1 -- 172.21.15.5:0/1406629726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f838009aac0 con 0x55b3f7746d20 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.185+0000 7f8384ff9700 1 -- 172.21.15.5:0/1406629726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800a2750 con 0x55b3f7746d20 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- 172.21.15.5:0/1406629726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 msgr2=0x55b3f7758710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f838009a070 tx=0x7f8380005910 comp rx=0 tx=0).stop 2023-09-23T04:46:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- 172.21.15.5:0/1406629726 shutdown_connections 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f774f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7749470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1406629726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7758710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- 172.21.15.5:0/1406629726 >> 172.21.15.5:0/1406629726 conn(0x55b3f76bb260 msgr2=0x55b3f7749cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- 172.21.15.5:0/1406629726 shutdown_connections 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- 172.21.15.5:0/1406629726 wait complete. 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.186+0000 7f83abbccb80 1 -- start start 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 Processor -- start 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7756210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f7756750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f77b45c0 con 0x55b3f7746d20 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f77b4730 con 0x55b3f7746400 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3f77b48f0 con 0x55b3f7751bd0 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f83abbccb80 10 client.0 _ll_register_callbacks cb 0x7f838c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7756210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f8395f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f839579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7756210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59000/0 (socket says 172.21.15.5:59000) 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f8395f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48526/0 (socket says 172.21.15.5:48526) 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f839579a700 1 -- 172.21.15.5:0/1171054406 learned_addr learned my addr 172.21.15.5:0/1171054406 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.187+0000 7f839679c700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f7756750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 -- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 msgr2=0x55b3f7756750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f7756750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 -- 172.21.15.5:0/1171054406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 msgr2=0x55b3f7756210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7756210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 -- 172.21.15.5:0/1171054406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3f76fc8d0 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f8395f9b700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f838c01af10 tx=0x7f838c0112f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95194e828c7fddb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f83797fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c016070 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8380077690 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.188+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f838c000e40 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838c005d60 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 10 client.25718 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b3f74adc60 con 0x55b3f7746400 2023-09-23T04:46:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8380077690 con 0x55b3f7746400 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f838c003680 con 0x55b3f7746400 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 10 client.25718 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 10 client.25718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 20 client.25718 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77b6010 0x55b3f77b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 4 client.25718 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.189+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b3f7692b30 con 0x55b3f77b6010 2023-09-23T04:46:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f839579a700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77b6010 0x55b3f77b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f839579a700 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77b6010 0x55b3f77b8430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bedb0a7941b00495 server_cookie=aaa4c648251ae50a in_seq=0 out_seq=0 2023-09-23T04:46:38.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f83797fa700 10 client.25718 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f83797fa700 1 -- 172.21.15.5:0/1171054406 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b3f7692b30 con 0x55b3f77b6010 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f83797fa700 10 client.25718 handle_command_reply: tid=0 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.190+0000 7f83abbccb80 1 client.25718 shutdown 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 msgr2=0x55b3f7755cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f838c01af10 tx=0x7f838c0112f0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 shutdown_connections 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b3f7751bd0 0x55b3f7756750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b3f7746400 0x55b3f7755cd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b3f77b6010 0x55b3f77b8430 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 --2- 172.21.15.5:0/1171054406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3f7746d20 0x55b3f7756210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 >> 172.21.15.5:0/1171054406 conn(0x55b3f76bb260 msgr2=0x55b3f774ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83797fa700 0 client.25718 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:46:38.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 shutdown_connections 2023-09-23T04:46:38.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 1 -- 172.21.15.5:0/1171054406 wait complete. 2023-09-23T04:46:38.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83abbccb80 20 client.25718 trim_cache size 0 max 16384 2023-09-23T04:46:38.197 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 msgr2=0x7f837c0a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.191+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 0x7f837c0a3ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8388006130 tx=0x7f8388009480 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 msgr2=0x7f83901997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f838000e970 tx=0x7f838000ec80 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 shutdown_connections 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f837c0a1820 0x7f837c0a3ce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f839019dc30 0x7f839019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83901010d0 0x7f8390199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 --2- 172.21.15.5:0/812785275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83901061f0 0x7f83901997d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 >> 172.21.15.5:0/812785275 conn(0x7f83900fc990 msgr2=0x7f839010c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 shutdown_connections 2023-09-23T04:46:38.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.192+0000 7f83981ff700 1 -- 172.21.15.5:0/812785275 wait complete. 2023-09-23T04:46:38.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:46:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 -- 172.21.15.5:0/4172909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 msgr2=0x7f862c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 --2- 172.21.15.5:0/4172909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c1014b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8620009990 tx=0x7f8620009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 -- 172.21.15.5:0/4172909981 shutdown_connections 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 --2- 172.21.15.5:0/4172909981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c1061f0 0x7f862c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 --2- 172.21.15.5:0/4172909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c1014b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 --2- 172.21.15.5:0/4172909981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c101a80 0x7f862c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.416+0000 7f86331ff700 1 -- 172.21.15.5:0/4172909981 >> 172.21.15.5:0/4172909981 conn(0x7f862c0fc990 msgr2=0x7f862c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.417+0000 7f86331ff700 1 -- 172.21.15.5:0/4172909981 shutdown_connections 2023-09-23T04:46:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.417+0000 7f86331ff700 1 -- 172.21.15.5:0/4172909981 wait complete. 2023-09-23T04:46:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 Processor -- start 2023-09-23T04:46:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 -- start start 2023-09-23T04:46:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c1061f0 0x7f862c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 0x7f862c1a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1a57f0 con 0x7f862c1010d0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1a5960 con 0x7f862c073ba0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f86331ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1a5b20 con 0x7f862c1061f0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59016/0 (socket says 172.21.15.5:59016) 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f8630f9b700 1 -- 172.21.15.5:0/752253116 learned_addr learned my addr 172.21.15.5:0/752253116 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f862bfff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c1061f0 0x7f862c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.418+0000 7f863179c700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 0x7f862c1a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 -- 172.21.15.5:0/752253116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c1061f0 msgr2=0x7f862c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c1061f0 0x7f862c073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 -- 172.21.15.5:0/752253116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 msgr2=0x7f862c1a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 0x7f862c1a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 -- 172.21.15.5:0/752253116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009520 con 0x7f862c1010d0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f863179c700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 0x7f862c1a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8630f9b700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8620005910 tx=0x7f8620005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8629ffb700 1 -- 172.21.15.5:0/752253116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862001d070 con 0x7f862c1010d0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8629ffb700 1 -- 172.21.15.5:0/752253116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8620022470 con 0x7f862c1010d0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f8629ffb700 1 -- 172.21.15.5:0/752253116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620017650 con 0x7f862c1010d0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.419+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200095f0 con 0x7f862c1010d0 2023-09-23T04:46:38.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.420+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c1a5fe0 con 0x7f862c1010d0 2023-09-23T04:46:38.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.422+0000 7f8629ffb700 1 -- 172.21.15.5:0/752253116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f86200225e0 con 0x7f862c1010d0 2023-09-23T04:46:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.424+0000 7f8629ffb700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 0x7f861c0a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.424+0000 7f8646a87b80 1 Processor -- start 2023-09-23T04:46:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.424+0000 7f8629ffb700 1 -- 172.21.15.5:0/752253116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f86200cecb0 con 0x7f862c1010d0 2023-09-23T04:46:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.424+0000 7f8646a87b80 1 -- start start 2023-09-23T04:46:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.424+0000 7f862bfff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 0x7f861c0a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadebc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 0x560dadec9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dade25460 con 0x560dadebcd20 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dade16c20 con 0x560dadec7bd0 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8646a87b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dade14540 con 0x560dadebc400 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f862bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f863179c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 0x560dadec9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f862bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f862bfff700 1 -- 172.21.15.5:0/983215874 learned_addr learned my addr 172.21.15.5:0/983215874 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f8630f9b700 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadebc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f862bfff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 0x7f861c0a3c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f862c074910 tx=0x7f8618009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f863179c700 1 -- 172.21.15.5:0/983215874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 msgr2=0x560dadebc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f863179c700 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadebc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f863179c700 1 -- 172.21.15.5:0/983215874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 msgr2=0x560dadec7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.425+0000 7f863179c700 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f863179c700 1 -- 172.21.15.5:0/983215874 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dade728d0 con 0x560dadec7bd0 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f862bfff700 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f863179c700 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 0x560dadec9fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86240093a0 tx=0x7f8624009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5178afb84ed2613 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f8616ffd700 1 -- 172.21.15.5:0/983215874 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400e070 con 0x560dadec7bd0 2023-09-23T04:46:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f8616ffd700 1 -- 172.21.15.5:0/983215874 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862400f040 con 0x560dadec7bd0 2023-09-23T04:46:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.426+0000 7f8616ffd700 1 -- 172.21.15.5:0/983215874 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624015db0 con 0x560dadec7bd0 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- 172.21.15.5:0/983215874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 msgr2=0x560dadec9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 0x560dadec9fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86240093a0 tx=0x7f8624009920 comp rx=0 tx=0).stop 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- 172.21.15.5:0/983215874 shutdown_connections 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadec7bd0 0x560dadec9fc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadebc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 --2- 172.21.15.5:0/983215874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebcd20 0x560dadec7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- 172.21.15.5:0/983215874 >> 172.21.15.5:0/983215874 conn(0x560dade31260 msgr2=0x560dadeb8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- 172.21.15.5:0/983215874 shutdown_connections 2023-09-23T04:46:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- 172.21.15.5:0/983215874 wait complete. 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.427+0000 7f8646a87b80 1 -- start start 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 Processor -- start 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebcd20 0x560dadec4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadec50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dade0fe30 con 0x560dadec7bd0 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560daddac710 con 0x560dadebc400 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560daddac3b0 con 0x560dadebcd20 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8646a87b80 10 client.0 _ll_register_callbacks cb 0x560dadc09800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48564/0 (socket says 172.21.15.5:48564) 2023-09-23T04:46:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f8630f9b700 1 -- 172.21.15.5:0/1381044780 learned_addr learned my addr 172.21.15.5:0/1381044780 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.428+0000 7f863179c700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadec50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f862bfff700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebcd20 0x560dadec4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 -- 172.21.15.5:0/1381044780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebcd20 msgr2=0x560dadec4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebcd20 0x560dadec4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 -- 172.21.15.5:0/1381044780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 msgr2=0x560dadec50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadec50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 -- 172.21.15.5:0/1381044780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8618017070 con 0x560dadebc400 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8620006010 tx=0x7f862009b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc1b6b3da7d9e44 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f860bfff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200b9de0 con 0x560dadebc400 2023-09-23T04:46:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dadeb0250 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f862009be00 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.429+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86200bb0c0 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 10 client.25730 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8624009050 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dadeb0250 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f86200b0020 con 0x560dadebc400 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 10 client.25730 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 10 client.25730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 20 client.25730 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560daded7390 0x560daded97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 4 client.25730 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:38.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dade08e10 con 0x560daded7390 2023-09-23T04:46:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.430+0000 7f862bfff700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560daded7390 0x560daded97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.431+0000 7f862bfff700 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560daded7390 0x560daded97b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33e5b1418a1a79f7 server_cookie=f1b49bef4a046bcc in_seq=0 out_seq=0 2023-09-23T04:46:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.431+0000 7f860bfff700 10 client.25730 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f860bfff700 1 -- 172.21.15.5:0/1381044780 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560dade08e10 con 0x560daded7390 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f860bfff700 10 client.25730 handle_command_reply: tid=0 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 client.25730 shutdown 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 msgr2=0x560dadec7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8620006010 tx=0x7f862009b5e0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 shutdown_connections 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560daded7390 0x560daded97b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadec50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebc400 0x560dadec7680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1381044780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebcd20 0x560dadec4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f860bfff700 0 client.25730 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:46:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 >> 172.21.15.5:0/1381044780 conn(0x560dade31260 msgr2=0x560dadeb8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 shutdown_connections 2023-09-23T04:46:38.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 1 -- 172.21.15.5:0/1381044780 wait complete. 2023-09-23T04:46:38.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.432+0000 7f8646a87b80 20 client.25730 trim_cache size 0 max 16384 2023-09-23T04:46:38.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.445+0000 7f8646a87b80 1 Processor -- start 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 -- start start 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadec7bd0 0x560dadec4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadc90920 con 0x560dadebc400 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadeafd90 con 0x560dadebcd20 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8646a87b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadecec60 con 0x560dadec7bd0 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59042/0 (socket says 172.21.15.5:59042) 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 -- 172.21.15.5:0/552167277 learned_addr learned my addr 172.21.15.5:0/552167277 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f863179c700 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadec7bd0 0x560dadec4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f862bfff700 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 -- 172.21.15.5:0/552167277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadec7bd0 msgr2=0x560dadec4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadec7bd0 0x560dadec4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 -- 172.21.15.5:0/552167277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 msgr2=0x560dadece720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.446+0000 7f8630f9b700 1 -- 172.21.15.5:0/552167277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dadc3ade0 con 0x560dadebc400 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f8630f9b700 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f862009b400 tx=0x7f86200b3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932fb2a4f4367537 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f86177fe700 1 -- 172.21.15.5:0/552167277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200cbe40 con 0x560dadebc400 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f86177fe700 1 -- 172.21.15.5:0/552167277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862009be00 con 0x560dadebc400 2023-09-23T04:46:38.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f86177fe700 1 -- 172.21.15.5:0/552167277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a3770 con 0x560dadebc400 2023-09-23T04:46:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f8646a87b80 1 -- 172.21.15.5:0/552167277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 msgr2=0x560dadebf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.447+0000 7f8646a87b80 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f862009b400 tx=0x7f86200b3f90 comp rx=0 tx=0).stop 2023-09-23T04:46:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 -- 172.21.15.5:0/552167277 shutdown_connections 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadec7bd0 0x560dadec4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadebc400 0x560dadebf2a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 --2- 172.21.15.5:0/552167277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 -- 172.21.15.5:0/552167277 >> 172.21.15.5:0/552167277 conn(0x560dade31260 msgr2=0x560dadeb79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 -- 172.21.15.5:0/552167277 shutdown_connections 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 -- 172.21.15.5:0/552167277 wait complete. 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 -- start start 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.448+0000 7f8646a87b80 1 Processor -- start 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadecded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadecccd0 con 0x560dadec7bd0 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadecce40 con 0x560dadebcd20 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dadecd000 con 0x560dadebc400 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8646a87b80 10 client.0 _ll_register_callbacks cb 0x7f8620097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f862bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f862bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48578/0 (socket says 172.21.15.5:48578) 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59052/0 (socket says 172.21.15.5:59052) 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f862bfff700 1 -- 172.21.15.5:0/1571804525 learned_addr learned my addr 172.21.15.5:0/1571804525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadecded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 -- 172.21.15.5:0/1571804525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 msgr2=0x560dadecded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadecded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 -- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 msgr2=0x560dadece410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f863179c700 1 -- 172.21.15.5:0/1571804525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8624009050 con 0x560dadec7bd0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadecded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.449+0000 7f862bfff700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f863179c700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8624009370 tx=0x7f8624015670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cecc040a2fbd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f860bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400e040 con 0x560dadec7bd0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f862400f040 con 0x560dadec7bd0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8624015890 con 0x560dadec7bd0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86240095e0 con 0x560dadec7bd0 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f8646a87b80 10 client.15747 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.450+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560dadc23c60 con 0x560dadec7bd0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862400f040 con 0x560dadec7bd0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8624028c90 con 0x560dadec7bd0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 10 client.15747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 20 client.15747 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560dadf2c2e0 0x560dadf2e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 4 client.15747 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560dade08b30 con 0x560dadf2c2e0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.451+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560dadf2c2e0 0x560dadf2e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8630f9b700 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560dadf2c2e0 0x560dadf2e700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=652221c3b3267cfe server_cookie=da61a40f1732971 in_seq=0 out_seq=0 2023-09-23T04:46:38.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f860bfff700 10 client.15747 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:46:38.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f860bfff700 1 -- 172.21.15.5:0/1571804525 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560dade08b30 con 0x560dadf2c2e0 2023-09-23T04:46:38.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f860bfff700 10 client.15747 handle_command_reply: tid=0 2023-09-23T04:46:38.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 client.15747 shutdown 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 msgr2=0x560dadecc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8624009370 tx=0x7f8624015670 comp rx=0 tx=0).stop 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 shutdown_connections 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x560dadf2c2e0 0x560dadf2e700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dadec7bd0 0x560dadecc700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560dadebc400 0x560dadecded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f860bfff700 0 client.15747 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 --2- 172.21.15.5:0/1571804525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dadebcd20 0x560dadece410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.452+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 >> 172.21.15.5:0/1571804525 conn(0x560dade31260 msgr2=0x560dadeb8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 shutdown_connections 2023-09-23T04:46:38.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f8646a87b80 1 -- 172.21.15.5:0/1571804525 wait complete. 2023-09-23T04:46:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f8646a87b80 20 client.15747 trim_cache size 0 max 16384 2023-09-23T04:46:38.459 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:46:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 msgr2=0x7f861c0a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 0x7f861c0a3c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f862c074910 tx=0x7f8618009260 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 msgr2=0x7f862c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8620005910 tx=0x7f8620005740 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 shutdown_connections 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f861c0a17a0 0x7f861c0a3c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073ba0 0x7f862c1a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c1010d0 0x7f862c073120 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 --2- 172.21.15.5:0/752253116 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c1061f0 0x7f862c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.453+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 >> 172.21.15.5:0/752253116 conn(0x7f862c0fc990 msgr2=0x7f862c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.454+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 shutdown_connections 2023-09-23T04:46:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.454+0000 7f86331ff700 1 -- 172.21.15.5:0/752253116 wait complete. 2023-09-23T04:46:38.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.678+0000 7fc064fff700 1 -- 172.21.15.5:0/1248180781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc060101a80 msgr2=0x7fc060105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.678+0000 7fc064fff700 1 --2- 172.21.15.5:0/1248180781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc060101a80 0x7fc060105ac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc048009990 tx=0x7fc048009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 -- 172.21.15.5:0/1248180781 shutdown_connections 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 --2- 172.21.15.5:0/1248180781 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc0601061f0 0x7fc06010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 --2- 172.21.15.5:0/1248180781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601010d0 0x7fc0601014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 --2- 172.21.15.5:0/1248180781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc060101a80 0x7fc060105ac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 -- 172.21.15.5:0/1248180781 >> 172.21.15.5:0/1248180781 conn(0x7fc0600fc990 msgr2=0x7fc0600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 -- 172.21.15.5:0/1248180781 shutdown_connections 2023-09-23T04:46:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.679+0000 7fc064fff700 1 -- 172.21.15.5:0/1248180781 wait complete. 2023-09-23T04:46:38.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 Processor -- start 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 -- start start 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 0x7fc06019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06019e550 con 0x7fc0601061f0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06019e6c0 con 0x7fc0601010d0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.680+0000 7fc064fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06019e880 con 0x7fc06019dad0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc057fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48598/0 (socket says 172.21.15.5:48598) 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 0x7fc06019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc057fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59062/0 (socket says 172.21.15.5:59062) 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05e59c700 1 -- 172.21.15.5:0/1302832583 learned_addr learned my addr 172.21.15.5:0/1302832583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 -- 172.21.15.5:0/1302832583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 msgr2=0x7fc060199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 -- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 msgr2=0x7fc060199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 -- 172.21.15.5:0/1302832583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050009520 con 0x7fc06019dad0 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05e59c700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc057fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 0x7fc06019df50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc05800efb0 tx=0x7fc05800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.681+0000 7fc0577fe700 1 -- 172.21.15.5:0/1302832583 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058003a00 con 0x7fc06019dad0 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.682+0000 7fc0577fe700 1 -- 172.21.15.5:0/1302832583 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc05800f040 con 0x7fc06019dad0 2023-09-23T04:46:38.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.682+0000 7fc0577fe700 1 -- 172.21.15.5:0/1302832583 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058004540 con 0x7fc06019dad0 2023-09-23T04:46:38.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.682+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc058009c40 con 0x7fc06019dad0 2023-09-23T04:46:38.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.682+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0480095f0 con 0x7fc06019dad0 2023-09-23T04:46:38.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.685+0000 7fc0577fe700 1 -- 172.21.15.5:0/1302832583 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7fc058003bf0 con 0x7fc06019dad0 2023-09-23T04:46:38.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.687+0000 7fc078a22b80 1 Processor -- start 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.687+0000 7fc078a22b80 1 -- start start 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.687+0000 7fc0577fe700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 0x7fc04c0a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc0577fe700 1 -- 172.21.15.5:0/1302832583 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc0580cd430 con 0x7fc06019dad0 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc057fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 0x7fc04c0a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8d20 0x5625f66c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6621460 con 0x5625f66b8400 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6612c20 con 0x5625f66b8d20 2023-09-23T04:46:38.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc078a22b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6610540 con 0x5625f66c3bd0 2023-09-23T04:46:38.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc05e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc05e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59070/0 (socket says 172.21.15.5:59070) 2023-09-23T04:46:38.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc05e59c700 1 -- 172.21.15.5:0/2773593584 learned_addr learned my addr 172.21.15.5:0/2773593584 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.688+0000 7fc057fff700 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8d20 0x5625f66c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc057fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 0x7fc04c0a3ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc06019a740 tx=0x7fc04800bc40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 -- 172.21.15.5:0/2773593584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 msgr2=0x5625f66c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 -- 172.21.15.5:0/2773593584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8d20 msgr2=0x5625f66c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8d20 0x5625f66c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 -- 172.21.15.5:0/2773593584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f666e8d0 con 0x5625f66b8400 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc05e59c700 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc050011840 tx=0x7fc050011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0e04513c450fab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc054ff9700 1 -- 172.21.15.5:0/2773593584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050011030 con 0x5625f66b8400 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc054ff9700 1 -- 172.21.15.5:0/2773593584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc050011190 con 0x5625f66b8400 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.689+0000 7fc054ff9700 1 -- 172.21.15.5:0/2773593584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500107f0 con 0x5625f66b8400 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 -- 172.21.15.5:0/2773593584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 msgr2=0x5625f66b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc050011840 tx=0x7fc050011b50 comp rx=0 tx=0).stop 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 -- 172.21.15.5:0/2773593584 shutdown_connections 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8400 0x5625f66b87e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 --2- 172.21.15.5:0/2773593584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8d20 0x5625f66c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 -- 172.21.15.5:0/2773593584 >> 172.21.15.5:0/2773593584 conn(0x5625f662d260 msgr2=0x5625f66b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 -- 172.21.15.5:0/2773593584 shutdown_connections 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.690+0000 7fc078a22b80 1 -- 172.21.15.5:0/2773593584 wait complete. 2023-09-23T04:46:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.691+0000 7fc078a22b80 1 -- start start 2023-09-23T04:46:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.691+0000 7fc078a22b80 1 Processor -- start 2023-09-23T04:46:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.691+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.691+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc078a22b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f660be30 con 0x5625f66b8d20 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc078a22b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f65a8710 con 0x5625f66b8400 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc078a22b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f65a83b0 con 0x5625f66c3bd0 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc078a22b80 10 client.0 _ll_register_callbacks cb 0x5625f6405800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59074/0 (socket says 172.21.15.5:59074) 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 -- 172.21.15.5:0/3135946227 learned_addr learned my addr 172.21.15.5:0/3135946227 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 -- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 msgr2=0x5625f66c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 -- 172.21.15.5:0/3135946227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 msgr2=0x5625f66c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc057fff700 1 -- 172.21.15.5:0/3135946227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0500114a0 con 0x5625f66b8d20 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.692+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc057fff700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc048009990 tx=0x7fc048009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b81514104662 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc045ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048004e50 con 0x5625f66b8d20 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625f66ac250 con 0x5625f66b8d20 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc04802c040 con 0x5625f66b8d20 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc04800f460 con 0x5625f66b8d20 2023-09-23T04:46:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc078a22b80 10 client.15759 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625f6436de0 con 0x5625f66b8d20 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.693+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625f66ac250 con 0x5625f66b8d20 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc048030610 con 0x5625f66b8d20 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 10 client.15759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 20 client.15759 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f66d36f0 0x5625f66d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 4 client.15759 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625f6604e10 con 0x5625f66d36f0 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.694+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f66d36f0 0x5625f66d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.695+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f66d36f0 0x5625f66d5b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84b1ae29477da681 server_cookie=1c95027083563c2 in_seq=0 out_seq=0 2023-09-23T04:46:38.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.695+0000 7fc045ffb700 10 client.15759 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:46:38.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc045ffb700 1 -- 172.21.15.5:0/3135946227 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5625f6604e10 con 0x5625f66d36f0 2023-09-23T04:46:38.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc045ffb700 10 client.15759 handle_command_reply: tid=0 2023-09-23T04:46:38.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 client.15759 shutdown 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 msgr2=0x5625f66c0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc048009990 tx=0x7fc048009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 shutdown_connections 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f66d36f0 0x5625f66d5b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3135946227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66b8d20 0x5625f66c0e80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc045ffb700 0 client.15759 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 >> 172.21.15.5:0/3135946227 conn(0x5625f662d260 msgr2=0x5625f66b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 shutdown_connections 2023-09-23T04:46:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.696+0000 7fc078a22b80 1 -- 172.21.15.5:0/3135946227 wait complete. 2023-09-23T04:46:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.697+0000 7fc078a22b80 20 client.15759 trim_cache size 0 max 16384 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 Processor -- start 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 -- start start 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66bbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f648c920 con 0x5625f66da470 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f66b3580 con 0x5625f66b8400 2023-09-23T04:46:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.711+0000 7fc078a22b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f66b9100 con 0x5625f66c3bd0 2023-09-23T04:46:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc057fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66bbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59082/0 (socket says 172.21.15.5:59082) 2023-09-23T04:46:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc057fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66bbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:51058/0 (socket says 172.21.15.5:51058) 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 -- 172.21.15.5:0/4138147875 learned_addr learned my addr 172.21.15.5:0/4138147875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05e59c700 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 -- 172.21.15.5:0/4138147875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 msgr2=0x5625f66bbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66bbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 -- 172.21.15.5:0/4138147875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 msgr2=0x5625f66ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 -- 172.21.15.5:0/4138147875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f641fc60 con 0x5625f66da470 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c6f0 tx=0x7fc0580a0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82070df8b2328440 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.712+0000 7fc0557fa700 1 -- 172.21.15.5:0/4138147875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05809a580 con 0x5625f66da470 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc0557fa700 1 -- 172.21.15.5:0/4138147875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0580cae90 con 0x5625f66da470 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc0557fa700 1 -- 172.21.15.5:0/4138147875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058004900 con 0x5625f66da470 2023-09-23T04:46:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 -- 172.21.15.5:0/4138147875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 msgr2=0x5625f66bc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c6f0 tx=0x7fc0580a0040 comp rx=0 tx=0).stop 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 -- 172.21.15.5:0/4138147875 shutdown_connections 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f66bc210 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f66ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 --2- 172.21.15.5:0/4138147875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f66bbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 -- 172.21.15.5:0/4138147875 >> 172.21.15.5:0/4138147875 conn(0x5625f662d260 msgr2=0x5625f66c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 -- 172.21.15.5:0/4138147875 shutdown_connections 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.713+0000 7fc078a22b80 1 -- 172.21.15.5:0/4138147875 wait complete. 2023-09-23T04:46:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.714+0000 7fc078a22b80 1 -- start start 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.714+0000 7fc078a22b80 1 Processor -- start 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f671b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f671bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6720ae0 con 0x5625f66da470 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6720c50 con 0x5625f66b8400 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f6720e10 con 0x5625f66c3bd0 2023-09-23T04:46:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc078a22b80 10 client.0 _ll_register_callbacks cb 0x5625f6405800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc05ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc05ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59098/0 (socket says 172.21.15.5:59098) 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc05ed9d700 1 -- 172.21.15.5:0/3917039953 learned_addr learned my addr 172.21.15.5:0/3917039953 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f671b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.715+0000 7fc057fff700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f671bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 -- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 msgr2=0x5625f671bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f671bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 -- 172.21.15.5:0/3917039953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 msgr2=0x5625f671b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f671b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 -- 172.21.15.5:0/3917039953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0480095f0 con 0x5625f66da470 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc05ed9d700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc0580968a0 tx=0x7fc0580ca5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452f24311f26d6ba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc045ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058099e80 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc050011190 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc05809dfd0 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0580a2180 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc078a22b80 10 client.15765 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.716+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625f645d250 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc050011190 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc058099370 con 0x5625f66da470 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 20 client.15765 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f6728390 0x5625f672a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 4 client.15765 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625f6604b30 con 0x5625f6728390 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.717+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f6728390 0x5625f672a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc05e59c700 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f6728390 0x5625f672a780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3adbbc79f4f7a8f0 server_cookie=231587d228ba5ffd in_seq=0 out_seq=0 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc045ffb700 10 client.15765 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc045ffb700 1 -- 172.21.15.5:0/3917039953 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625f6604b30 con 0x5625f6728390 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc045ffb700 10 client.15765 handle_command_reply: tid=0 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 client.15765 shutdown 2023-09-23T04:46:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 msgr2=0x5625f6720570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc0580968a0 tx=0x7fc0580ca5f0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 shutdown_connections 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f66da470 0x5625f6720570 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x5625f6728390 0x5625f672a780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5625f66b8400 0x5625f671b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 --2- 172.21.15.5:0/3917039953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5625f66c3bd0 0x5625f671bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 >> 172.21.15.5:0/3917039953 conn(0x5625f662d260 msgr2=0x5625f66bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc045ffb700 0 client.15765 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.718+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 shutdown_connections 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc078a22b80 1 -- 172.21.15.5:0/3917039953 wait complete. 2023-09-23T04:46:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc078a22b80 20 client.15765 trim_cache size 0 max 16384 2023-09-23T04:46:38.725 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:46:38.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 msgr2=0x7fc04c0a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 0x7fc04c0a3ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc06019a740 tx=0x7fc04800bc40 comp rx=0 tx=0).stop 2023-09-23T04:46:38.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 msgr2=0x7fc06019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 0x7fc06019df50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc05800efb0 tx=0x7fc05800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 shutdown_connections 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc04c0a1820 0x7fc04c0a3ce0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc06019dad0 0x7fc06019df50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0601010d0 0x7fc060199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 --2- 172.21.15.5:0/1302832583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0601061f0 0x7fc060199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 >> 172.21.15.5:0/1302832583 conn(0x7fc0600fc990 msgr2=0x7fc060102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.719+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 shutdown_connections 2023-09-23T04:46:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.720+0000 7fc064fff700 1 -- 172.21.15.5:0/1302832583 wait complete. 2023-09-23T04:46:38.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 -- 172.21.15.5:0/4082635817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75901010d0 msgr2=0x7f75901014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 --2- 172.21.15.5:0/4082635817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75901010d0 0x7f75901014b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f758c009990 tx=0x7f758c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 -- 172.21.15.5:0/4082635817 shutdown_connections 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 --2- 172.21.15.5:0/4082635817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901061f0 0x7f759010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 --2- 172.21.15.5:0/4082635817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75901010d0 0x7f75901014b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 --2- 172.21.15.5:0/4082635817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7590101a80 0x7f7590105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 -- 172.21.15.5:0/4082635817 >> 172.21.15.5:0/4082635817 conn(0x7f75900fc990 msgr2=0x7f75900fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 -- 172.21.15.5:0/4082635817 shutdown_connections 2023-09-23T04:46:38.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.945+0000 7f75975ff700 1 -- 172.21.15.5:0/4082635817 wait complete. 2023-09-23T04:46:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 Processor -- start 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 -- start start 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 0x7f7590199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 0x7f7590199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e660 con 0x7f759019dbe0 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e7d0 con 0x7f75901061f0 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.946+0000 7f75975ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e990 con 0x7f75901010d0 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59120/0 (socket says 172.21.15.5:59120) 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 -- 172.21.15.5:0/1235206580 learned_addr learned my addr 172.21.15.5:0/1235206580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f759539b700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 0x7f7590199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 0x7f7590199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 -- 172.21.15.5:0/1235206580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 msgr2=0x7f7590199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 0x7f7590199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 -- 172.21.15.5:0/1235206580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 msgr2=0x7f7590199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 0x7f7590199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7595b9c700 1 -- 172.21.15.5:0/1235206580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0095f0 con 0x7f759019dbe0 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f759539b700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 0x7f7590199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.947+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 0x7f7590199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7588007ee0 tx=0x7f758800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f75867fc700 1 -- 172.21.15.5:0/1235206580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588017070 con 0x7f759019dbe0 2023-09-23T04:46:38.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f75867fc700 1 -- 172.21.15.5:0/1235206580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758800f040 con 0x7f759019dbe0 2023-09-23T04:46:38.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f75867fc700 1 -- 172.21.15.5:0/1235206580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588005580 con 0x7f759019dbe0 2023-09-23T04:46:38.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75901a5ef0 con 0x7f759019dbe0 2023-09-23T04:46:38.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.948+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75901a6380 con 0x7f759019dbe0 2023-09-23T04:46:38.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.951+0000 7f75867fc700 1 -- 172.21.15.5:0/1235206580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f758800db20 con 0x7f759019dbe0 2023-09-23T04:46:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.952+0000 7f75aaf29b80 1 Processor -- start 2023-09-23T04:46:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 -- start start 2023-09-23T04:46:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75867fc700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 0x7f757c0a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75867fc700 1 -- 172.21.15.5:0/1235206580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f75880ccf40 con 0x7f759019dbe0 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1d20 0x5608e77bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77befc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e771a460 con 0x5608e77bcbd0 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e770bc20 con 0x5608e77b1d20 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7709540 con 0x5608e77b1400 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 0x7f757c0a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f7594b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1d20 0x5608e77bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f759539b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f759539b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:51114/0 (socket says 172.21.15.5:51114) 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.953+0000 7f759539b700 1 -- 172.21.15.5:0/948152579 learned_addr learned my addr 172.21.15.5:0/948152579 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 -- 172.21.15.5:0/948152579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1d20 msgr2=0x5608e77bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1d20 0x5608e77bc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 -- 172.21.15.5:0/948152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 msgr2=0x5608e77befc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77befc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 -- 172.21.15.5:0/948152579 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e77678d0 con 0x5608e77b1400 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 0x7f757c0a3ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75800097c0 tx=0x7f75800139f0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f759539b700 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b660 tx=0x7f758c005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcd66dde5062400 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f757b7fe700 1 -- 172.21.15.5:0/948152579 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c01d070 con 0x5608e77b1400 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f757b7fe700 1 -- 172.21.15.5:0/948152579 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758c01e040 con 0x5608e77b1400 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.954+0000 7f757b7fe700 1 -- 172.21.15.5:0/948152579 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c004d20 con 0x5608e77b1400 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 -- 172.21.15.5:0/948152579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 msgr2=0x5608e77b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b660 tx=0x7f758c005ee0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 -- 172.21.15.5:0/948152579 shutdown_connections 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77befc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1400 0x5608e77b17e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/948152579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1d20 0x5608e77bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 -- 172.21.15.5:0/948152579 >> 172.21.15.5:0/948152579 conn(0x5608e7726260 msgr2=0x5608e77ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 -- 172.21.15.5:0/948152579 shutdown_connections 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.955+0000 7f75aaf29b80 1 -- 172.21.15.5:0/948152579 wait complete. 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.956+0000 7f75aaf29b80 1 -- start start 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.956+0000 7f75aaf29b80 1 Processor -- start 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1d20 0x5608e77b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7704e30 con 0x5608e77bcbd0 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e76a1710 con 0x5608e77b1400 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e76a13b0 con 0x5608e77b1d20 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f75aaf29b80 10 client.0 _ll_register_callbacks cb 0x5608e74fe800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59138/0 (socket says 172.21.15.5:59138) 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 -- 172.21.15.5:0/1312685605 learned_addr learned my addr 172.21.15.5:0/1312685605 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f759539b700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1d20 0x5608e77b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 -- 172.21.15.5:0/1312685605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1d20 msgr2=0x5608e77b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1d20 0x5608e77b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 -- 172.21.15.5:0/1312685605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 msgr2=0x5608e77bc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.957+0000 7f7595b9c700 1 -- 172.21.15.5:0/1312685605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759010e5e0 con 0x5608e77bcbd0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f75880ca220 tx=0x7f758809a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c0f32e043888fe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758809da50 con 0x5608e77bcbd0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608e77a5250 con 0x5608e77bcbd0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f75880a1c90 con 0x5608e77bcbd0 2023-09-23T04:46:38.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7588022250 con 0x5608e77bcbd0 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f75aaf29b80 10 client.15777 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608e752fde0 con 0x5608e77bcbd0 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608e77a5250 con 0x5608e77bcbd0 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.958+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7588098df0 con 0x5608e77bcbd0 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 20 client.15777 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e77cc630 0x5608e77cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 4 client.15777 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608e76fde10 con 0x5608e77cc630 2023-09-23T04:46:38.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f759539b700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e77cc630 0x5608e77cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f7578ff9700 10 client.15777 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.959+0000 7f759539b700 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e77cc630 0x5608e77cea50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7fd1aec5409f7a server_cookie=56c804ad70378244 in_seq=0 out_seq=0 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f7578ff9700 1 -- 172.21.15.5:0/1312685605 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608e76fde10 con 0x5608e77cc630 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f7578ff9700 10 client.15777 handle_command_reply: tid=0 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 client.15777 shutdown 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 msgr2=0x5608e77b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f75880ca220 tx=0x7f758809a000 comp rx=0 tx=0).stop 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 shutdown_connections 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e77cc630 0x5608e77cea50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77bcbd0 0x5608e77b9d90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f7578ff9700 0 client.15777 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1312685605 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77b1d20 0x5608e77b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.960+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 >> 172.21.15.5:0/1312685605 conn(0x5608e7726260 msgr2=0x5608e77ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.961+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 shutdown_connections 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.961+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1312685605 wait complete. 2023-09-23T04:46:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.961+0000 7f75aaf29b80 20 client.15777 trim_cache size 0 max 16384 2023-09-23T04:46:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.974+0000 7f75aaf29b80 1 Processor -- start 2023-09-23T04:46:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.974+0000 7f75aaf29b80 1 -- start start 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e77be500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7585920 con 0x5608e77b1d20 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e77b7d90 con 0x5608e77b1400 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e77b56a0 con 0x5608e77bcbd0 2023-09-23T04:46:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7594b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7594b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77bdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59148/0 (socket says 172.21.15.5:59148) 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7594b9a700 1 -- 172.21.15.5:0/1993579099 learned_addr learned my addr 172.21.15.5:0/1993579099 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e77be500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f759539b700 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 -- 172.21.15.5:0/1993579099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 msgr2=0x5608e77bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 -- 172.21.15.5:0/1993579099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 msgr2=0x5608e77bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.975+0000 7f7595b9c700 1 -- 172.21.15.5:0/1993579099 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e7518c60 con 0x5608e77bcbd0 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e77be500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7588095e00 tx=0x7f7588097eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547d9361aeb91d98 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f757bfff700 1 -- 172.21.15.5:0/1993579099 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758809a070 con 0x5608e77bcbd0 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f757bfff700 1 -- 172.21.15.5:0/1993579099 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75880a1c90 con 0x5608e77bcbd0 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f757bfff700 1 -- 172.21.15.5:0/1993579099 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588098df0 con 0x5608e77bcbd0 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1993579099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 msgr2=0x5608e77be500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e77be500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7588095e00 tx=0x7f7588097eb0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1993579099 shutdown_connections 2023-09-23T04:46:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e77be500 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1993579099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.976+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1993579099 >> 172.21.15.5:0/1993579099 conn(0x5608e7726260 msgr2=0x5608e77c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.977+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1993579099 shutdown_connections 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.977+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1993579099 wait complete. 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.977+0000 7f75aaf29b80 1 -- start start 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.977+0000 7f75aaf29b80 1 Processor -- start 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e781f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e77c24e0 con 0x5608e77b1d20 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e781f9a0 con 0x5608e77b1400 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e781fae0 con 0x5608e77bcbd0 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f75aaf29b80 10 client.0 _ll_register_callbacks cb 0x7f7588099440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59154/0 (socket says 172.21.15.5:59154) 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 -- 172.21.15.5:0/1954142258 learned_addr learned my addr 172.21.15.5:0/1954142258 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f759539b700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 -- 172.21.15.5:0/1954142258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 msgr2=0x5608e781f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e781f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e781f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 -- 172.21.15.5:0/1954142258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 msgr2=0x5608e77c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7594b9a700 1 -- 172.21.15.5:0/1954142258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608e752fde0 con 0x5608e77b1d20 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f759539b700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.978+0000 7f7595b9c700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e781f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f7594b9a700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7580009300 tx=0x7f758001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6163014dd071afa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f7578ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580026040 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f758c01e040 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f758002c400 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7580021e70 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f75aaf29b80 10 client.15783 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.979+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608e7556250 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758c01e040 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7580029030 con 0x5608e77b1d20 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 20 client.15783 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:38.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e7821330 0x5608e7823750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:38.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 4 client.15783 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:46:38.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608e76fdb30 con 0x5608e7821330 2023-09-23T04:46:38.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.980+0000 7f759539b700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e7821330 0x5608e7823750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:38.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f759539b700 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e7821330 0x5608e7823750 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a54519482205826 server_cookie=ddb8ed6738f40a57 in_seq=0 out_seq=0 2023-09-23T04:46:38.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f7578ff9700 10 client.15783 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:38.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f7578ff9700 1 -- 172.21.15.5:0/1954142258 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608e76fdb30 con 0x5608e7821330 2023-09-23T04:46:38.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f7578ff9700 10 client.15783 handle_command_reply: tid=0 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 client.15783 shutdown 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 msgr2=0x5608e77c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7580009300 tx=0x7f758001d000 comp rx=0 tx=0).stop 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 shutdown_connections 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5608e7821330 0x5608e7823750 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5608e77bcbd0 0x5608e781f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5608e77b1400 0x5608e77c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f7578ff9700 0 client.15783 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 --2- 172.21.15.5:0/1954142258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e77b1d20 0x5608e77c1dd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 >> 172.21.15.5:0/1954142258 conn(0x5608e7726260 msgr2=0x5608e77ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 shutdown_connections 2023-09-23T04:46:38.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 1 -- 172.21.15.5:0/1954142258 wait complete. 2023-09-23T04:46:38.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.981+0000 7f75aaf29b80 20 client.15783 trim_cache size 0 max 16384 2023-09-23T04:46:38.989 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:46:38.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 msgr2=0x7f757c0a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 0x7f757c0a3ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f75800097c0 tx=0x7f75800139f0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 msgr2=0x7f759019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:38.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7588007ee0 tx=0x7f758800d3b0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 shutdown_connections 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f757c0a1820 0x7f757c0a3ce0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f759019dbe0 0x7f759019e060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75901010d0 0x7f7590199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 --2- 172.21.15.5:0/1235206580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75901061f0 0x7f7590199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 >> 172.21.15.5:0/1235206580 conn(0x7f75900fc990 msgr2=0x7f75900fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 shutdown_connections 2023-09-23T04:46:38.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:38.982+0000 7f75975ff700 1 -- 172.21.15.5:0/1235206580 wait complete. 2023-09-23T04:46:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:39 smithi069 ceph-mon[133931]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-23T04:46:40.001 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:46:40.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:46:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:39 smithi005 ceph-mon[97638]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-23T04:46:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:39 smithi118 ceph-mon[134297]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-23T04:46:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 -- 172.21.15.5:0/1552083060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c1010d0 msgr2=0x7f903c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 --2- 172.21.15.5:0/1552083060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c1010d0 0x7f903c1014b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f902c009990 tx=0x7f902c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 -- 172.21.15.5:0/1552083060 shutdown_connections 2023-09-23T04:46:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 --2- 172.21.15.5:0/1552083060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 0x7f903c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 --2- 172.21.15.5:0/1552083060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c1010d0 0x7f903c1014b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 --2- 172.21.15.5:0/1552083060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c101a80 0x7f903c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 -- 172.21.15.5:0/1552083060 >> 172.21.15.5:0/1552083060 conn(0x7f903c0fc990 msgr2=0x7f903c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 -- 172.21.15.5:0/1552083060 shutdown_connections 2023-09-23T04:46:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.220+0000 7f90429ff700 1 -- 172.21.15.5:0/1552083060 wait complete. 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 Processor -- start 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 -- start start 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c1010d0 0x7f903c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 0x7f903c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19e580 con 0x7f903c19db00 2023-09-23T04:46:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19e6f0 con 0x7f903c1010d0 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.221+0000 7f90429ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19e8b0 con 0x7f903c1061f0 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59178/0 (socket says 172.21.15.5:59178) 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 -- 172.21.15.5:0/720514769 learned_addr learned my addr 172.21.15.5:0/720514769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f903b7fe700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 0x7f903c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f903bfff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c1010d0 0x7f903c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 -- 172.21.15.5:0/720514769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 msgr2=0x7f903c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 0x7f903c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 -- 172.21.15.5:0/720514769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c1010d0 msgr2=0x7f903c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c1010d0 0x7f903c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 -- 172.21.15.5:0/720514769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0095f0 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f9040f9c700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f903400ef30 tx=0x7f903400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.222+0000 7f90397fa700 1 -- 172.21.15.5:0/720514769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034018070 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.223+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c1a5db0 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.223+0000 7f90397fa700 1 -- 172.21.15.5:0/720514769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903400f040 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.223+0000 7f90397fa700 1 -- 172.21.15.5:0/720514769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903400cd00 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.223+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c1a6290 con 0x7f903c19db00 2023-09-23T04:46:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.226+0000 7f90397fa700 1 -- 172.21.15.5:0/720514769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7f9034003a00 con 0x7f903c19db00 2023-09-23T04:46:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.227+0000 7f905628fb80 1 Processor -- start 2023-09-23T04:46:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.227+0000 7f905628fb80 1 -- start start 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213be7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213bf2c50 0x556213bf5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213b504e0 con 0x556213be7da0 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213b41ca0 con 0x556213bf2c50 2023-09-23T04:46:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f905628fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213b3f5c0 con 0x556213be7480 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f90397fa700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 0x7f90240a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f903bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f90397fa700 1 -- 172.21.15.5:0/720514769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f90340cd1c0 con 0x7f903c19db00 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f903bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59182/0 (socket says 172.21.15.5:59182) 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f903bfff700 1 -- 172.21.15.5:0/2500844714 learned_addr learned my addr 172.21.15.5:0/2500844714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f9040f9c700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 0x7f90240a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f903b7fe700 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213be7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.228+0000 7f903b7fe700 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213bf2c50 0x556213bf5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 -- 172.21.15.5:0/2500844714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 msgr2=0x556213be7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213be7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 -- 172.21.15.5:0/2500844714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213bf2c50 msgr2=0x556213bf5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213bf2c50 0x556213bf5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 -- 172.21.15.5:0/2500844714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556213b9d950 con 0x556213be7da0 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f9040f9c700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 0x7f90240a3c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90340ca470 tx=0x7f90340092e0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f903bfff700 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f902c015000 tx=0x7f902c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b548903388c6d087 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f90227fc700 1 -- 172.21.15.5:0/2500844714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c01d030 con 0x556213be7da0 2023-09-23T04:46:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f90227fc700 1 -- 172.21.15.5:0/2500844714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902c01e040 con 0x556213be7da0 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.229+0000 7f90227fc700 1 -- 172.21.15.5:0/2500844714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0048c0 con 0x556213be7da0 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 -- 172.21.15.5:0/2500844714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 msgr2=0x556213bf2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f902c015000 tx=0x7f902c00b5b0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 -- 172.21.15.5:0/2500844714 shutdown_connections 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213bf2c50 0x556213bf5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213be7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 --2- 172.21.15.5:0/2500844714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf2710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 -- 172.21.15.5:0/2500844714 >> 172.21.15.5:0/2500844714 conn(0x556213b5c2e0 msgr2=0x556213be3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 -- 172.21.15.5:0/2500844714 shutdown_connections 2023-09-23T04:46:40.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.230+0000 7f905628fb80 1 -- 172.21.15.5:0/2500844714 wait complete. 2023-09-23T04:46:40.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 -- start start 2023-09-23T04:46:40.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 Processor -- start 2023-09-23T04:46:40.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213beff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213b3aeb0 con 0x556213be7480 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213ad7790 con 0x556213be7da0 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213ad7430 con 0x556213bf2c50 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.231+0000 7f905628fb80 10 client.0 _ll_register_callbacks cb 0x556213934880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59186/0 (socket says 172.21.15.5:59186) 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903b7fe700 1 -- 172.21.15.5:0/4029333715 learned_addr learned my addr 172.21.15.5:0/4029333715 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903b7fe700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213beff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 -- 172.21.15.5:0/4029333715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 msgr2=0x556213beff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213beff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 -- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 msgr2=0x556213bf2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 -- 172.21.15.5:0/4029333715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c10e5e0 con 0x556213be7da0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903b7fe700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f903bfff700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef8c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f902c000c00 tx=0x7f902c00baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc7ad6b5f787f86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.232+0000 7f901b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c01d030 con 0x556213be7da0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556213bdb2d0 con 0x556213be7da0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f902c004910 con 0x556213be7da0 2023-09-23T04:46:40.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f902c00bd50 con 0x556213be7da0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f905628fb80 10 client.25784 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556213965e60 con 0x556213be7da0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.233+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556213bdb2d0 con 0x556213be7da0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f902c00fa30 con 0x556213be7da0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 10 client.25784 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 20 client.25784 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c020d0 0x556213c044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 4 client.25784 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556213b33e90 con 0x556213c020d0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.234+0000 7f903b7fe700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c020d0 0x556213c044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f901b7fe700 10 client.25784 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f903b7fe700 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c020d0 0x556213c044f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62eab416b23c3ae server_cookie=8385fd6195667523 in_seq=0 out_seq=0 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f901b7fe700 1 -- 172.21.15.5:0/4029333715 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556213b33e90 con 0x556213c020d0 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f901b7fe700 10 client.25784 handle_command_reply: tid=0 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f905628fb80 1 client.25784 shutdown 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 msgr2=0x556213bef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.235+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef8c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f902c000c00 tx=0x7f902c00baa0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 shutdown_connections 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c020d0 0x556213c044f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213beff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7480 0x556213bf2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f901b7fe700 0 client.25784 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 --2- 172.21.15.5:0/4029333715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef8c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 >> 172.21.15.5:0/4029333715 conn(0x556213b5c2e0 msgr2=0x556213be33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 shutdown_connections 2023-09-23T04:46:40.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 1 -- 172.21.15.5:0/4029333715 wait complete. 2023-09-23T04:46:40.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.236+0000 7f905628fb80 20 client.25784 trim_cache size 0 max 16384 2023-09-23T04:46:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.253+0000 7f905628fb80 1 Processor -- start 2023-09-23T04:46:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.253+0000 7f905628fb80 1 -- start start 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213bfeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562139bb9a0 con 0x556213bf2c50 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213be86a0 con 0x556213be7da0 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f905628fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213bf0ca0 con 0x556213be7480 2023-09-23T04:46:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59190/0 (socket says 172.21.15.5:59190) 2023-09-23T04:46:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 -- 172.21.15.5:0/1427320572 learned_addr learned my addr 172.21.15.5:0/1427320572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903bfff700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213bfeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 -- 172.21.15.5:0/1427320572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 msgr2=0x556213bfeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213bfeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 -- 172.21.15.5:0/1427320572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 msgr2=0x556213bef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.254+0000 7f903b7fe700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f903b7fe700 1 -- 172.21.15.5:0/1427320572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556213bdf500 con 0x556213bf2c50 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f903bfff700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f903b7fe700 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556213c04dc0 tx=0x7f9030009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6a8939a707c94c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f9022ffd700 1 -- 172.21.15.5:0/1427320572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030016040 con 0x556213bf2c50 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f9022ffd700 1 -- 172.21.15.5:0/1427320572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9030025410 con 0x556213bf2c50 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.255+0000 7f9022ffd700 1 -- 172.21.15.5:0/1427320572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300116e0 con 0x556213bf2c50 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- 172.21.15.5:0/1427320572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 msgr2=0x556213bef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x556213c04dc0 tx=0x7f9030009260 comp rx=0 tx=0).stop 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- 172.21.15.5:0/1427320572 shutdown_connections 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213bf2c50 0x556213bef5e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213be7480 0x556213bfeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 --2- 172.21.15.5:0/1427320572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7da0 0x556213bef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- 172.21.15.5:0/1427320572 >> 172.21.15.5:0/1427320572 conn(0x556213b5c2e0 msgr2=0x556213beac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- 172.21.15.5:0/1427320572 shutdown_connections 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- 172.21.15.5:0/1427320572 wait complete. 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 -- start start 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.256+0000 7f905628fb80 1 Processor -- start 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213bf80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213bf8780 con 0x556213be7da0 2023-09-23T04:46:40.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213bf88f0 con 0x556213be7480 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556213c55ed0 con 0x556213bf2c50 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f905628fb80 10 client.0 _ll_register_callbacks cb 0x7f9030009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59206/0 (socket says 172.21.15.5:59206) 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48752/0 (socket says 172.21.15.5:48752) 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903bfff700 1 -- 172.21.15.5:0/317885972 learned_addr learned my addr 172.21.15.5:0/317885972 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.257+0000 7f903b7fe700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213bf80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 -- 172.21.15.5:0/317885972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 msgr2=0x556213bf80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213bf80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 -- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 msgr2=0x556213bf7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 -- 172.21.15.5:0/317885972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556213965e60 con 0x556213be7480 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f903b7fe700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9030004580 tx=0x7f9030013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eec3c9f5351932a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f901b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030016070 con 0x556213be7480 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f902c01e040 con 0x556213be7480 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9030000e40 con 0x556213be7480 2023-09-23T04:46:40.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9030025580 con 0x556213be7480 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f905628fb80 10 client.25790 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.258+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f903002d040 con 0x556213be7480 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9030039030 con 0x556213be7480 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9030003680 con 0x556213be7480 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 20 client.25790 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c57430 0x556213c59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 4 client.25790 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:46:40.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556213b33bb0 con 0x556213c57430 2023-09-23T04:46:40.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.259+0000 7f903bfff700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c57430 0x556213c59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.260+0000 7f903bfff700 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c57430 0x556213c59850 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2102c6cdf6bb469 server_cookie=a967dfcf9f909b70 in_seq=0 out_seq=0 2023-09-23T04:46:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.260+0000 7f901b7fe700 10 client.25790 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f901b7fe700 1 -- 172.21.15.5:0/317885972 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556213b33bb0 con 0x556213c57430 2023-09-23T04:46:40.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f901b7fe700 10 client.25790 handle_command_reply: tid=0 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 client.25790 shutdown 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 msgr2=0x556213bf96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9030004580 tx=0x7f9030013040 comp rx=0 tx=0).stop 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 shutdown_connections 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x556213c57430 0x556213c59850 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556213bf2c50 0x556213bf80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556213be7480 0x556213bf96c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f901b7fe700 0 client.25790 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 --2- 172.21.15.5:0/317885972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556213be7da0 0x556213bf7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.261+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 >> 172.21.15.5:0/317885972 conn(0x556213b5c2e0 msgr2=0x556213bf11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.262+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 shutdown_connections 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.262+0000 7f905628fb80 1 -- 172.21.15.5:0/317885972 wait complete. 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.262+0000 7f905628fb80 20 client.25790 trim_cache size 0 max 16384 2023-09-23T04:46:40.269 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:46:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:46:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "f888b1b3-d5a1-43d1-b7f4-20d4ddc71dfd", 2023-09-23T04:46:40.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:46:40.270 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 msgr2=0x7f90240a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 0x7f90240a3c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90340ca470 tx=0x7f90340092e0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 msgr2=0x7f903c19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f903400ef30 tx=0x7f903400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 shutdown_connections 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f90240a17a0 0x7f90240a3c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f903c19db00 0x7f903c19df80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f903c1010d0 0x7f903c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 --2- 172.21.15.5:0/720514769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f903c1061f0 0x7f903c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 >> 172.21.15.5:0/720514769 conn(0x7f903c0fc990 msgr2=0x7f903c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 shutdown_connections 2023-09-23T04:46:40.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.263+0000 7f90429ff700 1 -- 172.21.15.5:0/720514769 wait complete. 2023-09-23T04:46:40.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 -- 172.21.15.5:0/1327049739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c1010e0 msgr2=0x7fa38c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 --2- 172.21.15.5:0/1327049739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c1010e0 0x7fa38c1014c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009990 tx=0x7fa37c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 -- 172.21.15.5:0/1327049739 shutdown_connections 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 --2- 172.21.15.5:0/1327049739 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 0x7fa38c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 --2- 172.21.15.5:0/1327049739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c1010e0 0x7fa38c1014c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 --2- 172.21.15.5:0/1327049739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c101a90 0x7fa38c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.488+0000 7fa3929ff700 1 -- 172.21.15.5:0/1327049739 >> 172.21.15.5:0/1327049739 conn(0x7fa38c0fc960 msgr2=0x7fa38c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.489+0000 7fa3929ff700 1 -- 172.21.15.5:0/1327049739 shutdown_connections 2023-09-23T04:46:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.489+0000 7fa3929ff700 1 -- 172.21.15.5:0/1327049739 wait complete. 2023-09-23T04:46:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.490+0000 7fa3929ff700 1 Processor -- start 2023-09-23T04:46:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.490+0000 7fa3929ff700 1 -- start start 2023-09-23T04:46:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 0x7fa38c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a58e0 con 0x7fa38c073ba0 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a5a50 con 0x7fa38c1010e0 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa3929ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1a5c10 con 0x7fa38c106200 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa390f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48776/0 (socket says 172.21.15.5:48776) 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa38bfff700 1 -- 172.21.15.5:0/2206443342 learned_addr learned my addr 172.21.15.5:0/2206443342 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa390f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59216/0 (socket says 172.21.15.5:59216) 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.491+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 0x7fa38c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 -- 172.21.15.5:0/2206443342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 msgr2=0x7fa38c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 0x7fa38c073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 -- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 msgr2=0x7fa38c1a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 -- 172.21.15.5:0/2206443342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0095f0 con 0x7fa38c1010e0 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa390f9c700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:40.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa38bfff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c010 tx=0x7fa37c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa3897fa700 1 -- 172.21.15.5:0/2206443342 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c01d070 con 0x7fa38c1010e0 2023-09-23T04:46:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa3897fa700 1 -- 172.21.15.5:0/2206443342 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c022470 con 0x7fa38c1010e0 2023-09-23T04:46:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c1a5e90 con 0x7fa38c1010e0 2023-09-23T04:46:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa3897fa700 1 -- 172.21.15.5:0/2206443342 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c017650 con 0x7fa38c1010e0 2023-09-23T04:46:40.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.492+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c1a6250 con 0x7fa38c1010e0 2023-09-23T04:46:40.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.496+0000 7fa3897fa700 1 -- 172.21.15.5:0/2206443342 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) v1 ==== 136367+0+0 (secure 0 0 0) 0x7fa37c0225e0 con 0x7fa38c1010e0 2023-09-23T04:46:40.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 Processor -- start 2023-09-23T04:46:40.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 -- start start 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3897fa700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 0x7fa3740a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698e9bd0 0x5589698ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558969847460 con 0x5589698e9bd0 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558969838c20 con 0x5589698de400 2023-09-23T04:46:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558969836540 con 0x5589698ded20 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa3897fa700 1 -- 172.21.15.5:0/2206443342 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa37c0cee00 con 0x7fa38c1010e0 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48786/0 (socket says 172.21.15.5:48786) 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa38bfff700 1 -- 172.21.15.5:0/589146232 learned_addr learned my addr 172.21.15.5:0/589146232 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.497+0000 7fa390f9c700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 0x7fa3740a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa390f9c700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698e9bd0 0x5589698ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 -- 172.21.15.5:0/589146232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 msgr2=0x5589698e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 -- 172.21.15.5:0/589146232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698e9bd0 msgr2=0x5589698ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698e9bd0 0x5589698ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 -- 172.21.15.5:0/589146232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589698948d0 con 0x5589698de400 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa38bfff700 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0982c0 tx=0x7fa37c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dccfd6e02f0ec04 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa390f9c700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 0x7fa3740a3c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa384009550 tx=0x7fa38400b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa3727fc700 1 -- 172.21.15.5:0/589146232 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c09bf50 con 0x5589698de400 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa3727fc700 1 -- 172.21.15.5:0/589146232 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c09b850 con 0x5589698de400 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.498+0000 7fa3727fc700 1 -- 172.21.15.5:0/589146232 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c099c80 con 0x5589698de400 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 -- 172.21.15.5:0/589146232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 msgr2=0x5589698de7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0982c0 tx=0x7fa37c009580 comp rx=0 tx=0).stop 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 -- 172.21.15.5:0/589146232 shutdown_connections 2023-09-23T04:46:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698e9bd0 0x5589698ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698de7e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/589146232 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 -- 172.21.15.5:0/589146232 >> 172.21.15.5:0/589146232 conn(0x558969853260 msgr2=0x5589698da790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 -- 172.21.15.5:0/589146232 shutdown_connections 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.499+0000 7fa3a6347b80 1 -- 172.21.15.5:0/589146232 wait complete. 2023-09-23T04:46:40.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.500+0000 7fa3a6347b80 1 -- start start 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.500+0000 7fa3a6347b80 1 Processor -- start 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 0x5589698eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558969831e30 con 0x5589698de400 2023-09-23T04:46:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589697ce710 con 0x5589698e9bd0 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589697ce3b0 con 0x5589698ded20 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa3a6347b80 10 client.0 _ll_register_callbacks cb 0x55896962b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59246/0 (socket says 172.21.15.5:59246) 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 -- 172.21.15.5:0/1416316569 learned_addr learned my addr 172.21.15.5:0/1416316569 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa390f9c700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 0x5589698eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 -- 172.21.15.5:0/1416316569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 msgr2=0x5589698ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 -- 172.21.15.5:0/1416316569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 msgr2=0x5589698eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 0x5589698eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa38bfff700 1 -- 172.21.15.5:0/1416316569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c1a6250 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.501+0000 7fa390f9c700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 0x5589698eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa38bfff700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009ce0 tx=0x7fa37c0cbfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6513874cb5e2f189 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa36b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0a3b50 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa37c0865b0 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589698d2250 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa37c0860c0 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa3a6347b80 10 client.15813 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.502+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55896965cde0 con 0x5589698de400 2023-09-23T04:46:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589698d2250 con 0x5589698de400 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa37c0b9020 con 0x5589698de400 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 20 client.15813 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5589698f8e30 0x5589698fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 4 client.15813 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55896982ae10 con 0x5589698f8e30 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.503+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5589698f8e30 0x5589698fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.504+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5589698f8e30 0x5589698fb250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9cc4a7d1f5a146 server_cookie=f3a0536477085496 in_seq=0 out_seq=0 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.504+0000 7fa36b7fe700 10 client.15813 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.504+0000 7fa36b7fe700 1 -- 172.21.15.5:0/1416316569 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55896982ae10 con 0x5589698f8e30 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.504+0000 7fa36b7fe700 10 client.15813 handle_command_reply: tid=0 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.504+0000 7fa3a6347b80 1 client.15813 shutdown 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 msgr2=0x5589698e9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009ce0 tx=0x7fa37c0cbfc0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 shutdown_connections 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5589698f8e30 0x5589698fb250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698e9bd0 0x5589698eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698e9fb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa36b7fe700 0 client.15813 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/1416316569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698ded20 0x5589698ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 >> 172.21.15.5:0/1416316569 conn(0x558969853260 msgr2=0x5589698da380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 shutdown_connections 2023-09-23T04:46:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 1 -- 172.21.15.5:0/1416316569 wait complete. 2023-09-23T04:46:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.505+0000 7fa3a6347b80 20 client.15813 trim_cache size 0 max 16384 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 Processor -- start 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 -- start start 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698ded20 0x5589698e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589696b2920 con 0x5589698ded20 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589698e92f0 con 0x5589698de400 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589698e90c0 con 0x5589698e9bd0 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa38b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698ded20 0x5589698e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48802/0 (socket says 172.21.15.5:48802) 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.523+0000 7fa38bfff700 1 -- 172.21.15.5:0/4017801242 learned_addr learned my addr 172.21.15.5:0/4017801242 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa390f9c700 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 -- 172.21.15.5:0/4017801242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 msgr2=0x5589698ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 -- 172.21.15.5:0/4017801242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698ded20 msgr2=0x5589698e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698ded20 0x5589698e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 -- 172.21.15.5:0/4017801242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589698d6480 con 0x5589698de400 2023-09-23T04:46:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa38bfff700 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa37c09b550 tx=0x7fa37c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be76aaf947cde92e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa372ffd700 1 -- 172.21.15.5:0/4017801242 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c066070 con 0x5589698de400 2023-09-23T04:46:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa372ffd700 1 -- 172.21.15.5:0/4017801242 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c0b6070 con 0x5589698de400 2023-09-23T04:46:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.524+0000 7fa372ffd700 1 -- 172.21.15.5:0/4017801242 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c09f9a0 con 0x5589698de400 2023-09-23T04:46:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 -- 172.21.15.5:0/4017801242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 msgr2=0x5589698eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa37c09b550 tx=0x7fa37c009310 comp rx=0 tx=0).stop 2023-09-23T04:46:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 -- 172.21.15.5:0/4017801242 shutdown_connections 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698de400 0x5589698eea80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/4017801242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698ded20 0x5589698e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 -- 172.21.15.5:0/4017801242 >> 172.21.15.5:0/4017801242 conn(0x558969853260 msgr2=0x5589698f55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 -- 172.21.15.5:0/4017801242 shutdown_connections 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.525+0000 7fa3a6347b80 1 -- 172.21.15.5:0/4017801242 wait complete. 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 -- start start 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 Processor -- start 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698ee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589698ece40 con 0x5589698de400 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589698ecfb0 con 0x5589698ded20 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589698ed170 con 0x5589698e9bd0 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.526+0000 7fa3a6347b80 10 client.0 _ll_register_callbacks cb 0x55896962b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa390f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa390f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:51268/0 (socket says 172.21.15.5:51268) 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48804/0 (socket says 172.21.15.5:48804) 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa390f9c700 1 -- 172.21.15.5:0/3147790178 learned_addr learned my addr 172.21.15.5:0/3147790178 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:46:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 -- 172.21.15.5:0/3147790178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 msgr2=0x5589698ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 -- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 msgr2=0x5589698ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 -- 172.21.15.5:0/3147790178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55896965cde0 con 0x5589698ded20 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38bfff700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa38b7fe700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5589698fbaf0 tx=0x7fa3800109f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5920b58a005b79e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.527+0000 7fa36b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380017050 con 0x5589698ded20 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa37c09b850 con 0x5589698ded20 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa38001eb80 con 0x5589698ded20 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa38001ed40 con 0x5589698ded20 2023-09-23T04:46:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 10 client.25808 fetch_fsmap learned FSMap version 27 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558969645c60 con 0x5589698ded20 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c09b850 con 0x5589698ded20 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa380005350 con 0x5589698ded20 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 20 client.25808 populate_metadata read hostname 'smithi005' 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55896994de00 0x5589699501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 4 client.25808 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.528+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55896982ab30 con 0x55896994de00 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa38bfff700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55896994de00 0x5589699501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:46:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa38bfff700 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55896994de00 0x5589699501f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff69d8dafe9f9f5 server_cookie=d3cdc3980494bbaf in_seq=0 out_seq=0 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa36b7fe700 10 client.25808 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa36b7fe700 1 -- 172.21.15.5:0/3147790178 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55896982ab30 con 0x55896994de00 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa36b7fe700 10 client.25808 handle_command_reply: tid=0 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.529+0000 7fa3a6347b80 1 client.25808 shutdown 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 msgr2=0x5589698ee960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5589698fbaf0 tx=0x7fa3800109f0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 shutdown_connections 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55896994de00 0x5589699501f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5589698e9bd0 0x5589698ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589698de400 0x5589698ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa36b7fe700 0 client.25808 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:46:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 --2- 172.21.15.5:0/3147790178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5589698ded20 0x5589698ee960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 >> 172.21.15.5:0/3147790178 conn(0x558969853260 msgr2=0x5589698e3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 shutdown_connections 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 1 -- 172.21.15.5:0/3147790178 wait complete. 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.530+0000 7fa3a6347b80 20 client.25808 trim_cache size 0 max 16384 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stdout: "f888b1b3-d5a1-43d1-b7f4-20d4ddc71dfd": { 2023-09-23T04:46:40.538 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "f888b1b3-d5a1-43d1-b7f4-20d4ddc71dfd", 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:46:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 msgr2=0x7fa3740a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 0x7fa3740a3c60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa384009550 tx=0x7fa38400b040 comp rx=0 tx=0).stop 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 msgr2=0x7fa38c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c010 tx=0x7fa37c005740 comp rx=0 tx=0).stop 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 shutdown_connections 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa3740a17a0 0x7fa3740a3c60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa38c073ba0 0x7fa38c1a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa38c1010e0 0x7fa38c073120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 --2- 172.21.15.5:0/2206443342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa38c106200 0x7fa38c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:46:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 >> 172.21.15.5:0/2206443342 conn(0x7fa38c0fc960 msgr2=0x7fa38c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:46:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 shutdown_connections 2023-09-23T04:46:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:46:40.531+0000 7fa3929ff700 1 -- 172.21.15.5:0/2206443342 wait complete. 2023-09-23T04:46:40.550 INFO:tasks.cephfs.filesystem:scrub status for tag:f888b1b3-d5a1-43d1-b7f4-20d4ddc71dfd - {'path': '/', 'tag': 'f888b1b3-d5a1-43d1-b7f4-20d4ddc71dfd', 'options': 'recursive,force'} 2023-09-23T04:46:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:41 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:46:41.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:41 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:41 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:46:41.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:41 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:41 smithi005 ceph-mon[97638]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:41.982 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:41 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:46:41.982 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:41 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.982 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:41 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:46:41.982 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:41 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.983 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:41 smithi118 ceph-mon[134297]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:41 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:46:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:41 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:41 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:46:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:41 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:46:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:41 smithi069 ceph-mon[133931]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:43 smithi069 ceph-mon[133931]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:43 smithi005 ceph-mon[97638]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:43 smithi118 ceph-mon[134297]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T04:46:45.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:45 smithi069 ceph-mon[133931]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:45.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:45 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:46:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:45 smithi005 ceph-mon[97638]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:46.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:45 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:45 smithi118 ceph-mon[134297]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:45 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:46:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:47 smithi069 ceph-mon[133931]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:47 smithi005 ceph-mon[97638]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:47 smithi118 ceph-mon[134297]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:49 smithi069 ceph-mon[133931]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:50.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:49 smithi005 ceph-mon[97638]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:49 smithi118 ceph-mon[134297]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2023-09-23T04:46:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:51 smithi005 ceph-mon[97638]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:51 smithi118 ceph-mon[134297]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:51 smithi069 ceph-mon[133931]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:46:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:53 smithi069 ceph-mon[133931]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-23T04:46:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:53 smithi005 ceph-mon[97638]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-23T04:46:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:53 smithi118 ceph-mon[134297]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-23T04:46:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:55 smithi069 ceph-mon[133931]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:46:56.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:55 smithi005 ceph-mon[97638]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:46:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:55 smithi118 ceph-mon[134297]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-23T04:46:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:57 smithi069 ceph-mon[133931]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-23T04:46:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:57 smithi005 ceph-mon[97638]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-23T04:46:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:57 smithi118 ceph-mon[134297]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-23T04:46:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:46:59 smithi069 ceph-mon[133931]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-09-23T04:47:00.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:46:59 smithi005 ceph-mon[97638]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-09-23T04:47:00.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:46:59 smithi118 ceph-mon[134297]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-09-23T04:47:00.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:47:00 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:47:00.253+0000 7f67f1783700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:47:00.654 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:47:00 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:47:00.253+0000 7f67f1783700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:47:00.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:47:00 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:47:00.253+0000 7f67f1783700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:47:00.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:47:00 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:47:00.253+0000 7f67f1783700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:47:00.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 04:47:00 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T04:47:00.253+0000 7f67f1783700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-23T04:47:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:01 smithi005 ceph-mon[97638]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-23T04:47:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:01 smithi118 ceph-mon[134297]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-23T04:47:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:01 smithi069 ceph-mon[133931]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-23T04:47:02.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:02 smithi005 ceph-mon[97638]: mgrmap e25: x(active, since 44s), standbys: y, z 2023-09-23T04:47:02.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:02 smithi118 ceph-mon[134297]: mgrmap e25: x(active, since 44s), standbys: y, z 2023-09-23T04:47:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:02 smithi069 ceph-mon[133931]: mgrmap e25: x(active, since 44s), standbys: y, z 2023-09-23T04:47:03.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:03 smithi005 ceph-mon[97638]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-23T04:47:03.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:03 smithi118 ceph-mon[134297]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-23T04:47:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:03 smithi069 ceph-mon[133931]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-23T04:47:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:05 smithi069 ceph-mon[133931]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:05 smithi005 ceph-mon[97638]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:05 smithi118 ceph-mon[134297]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:07.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:07 smithi069 ceph-mon[133931]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:07 smithi005 ceph-mon[97638]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:07 smithi118 ceph-mon[134297]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:09.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:09 smithi069 ceph-mon[133931]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:10.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:09 smithi005 ceph-mon[97638]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:10.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:09 smithi118 ceph-mon[134297]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2023-09-23T04:47:10.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:47:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 -- 172.21.15.5:0/2140783435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc01083f0 msgr2=0x7ffbc010ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/2140783435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc01083f0 0x7ffbc010ff70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b0a0 tx=0x7ffbb800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 -- 172.21.15.5:0/2140783435 shutdown_connections 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/2140783435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc01083f0 0x7ffbc010ff70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/2140783435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01032d0 0x7ffbc01036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/2140783435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc0103c80 0x7ffbc0107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 -- 172.21.15.5:0/2140783435 >> 172.21.15.5:0/2140783435 conn(0x7ffbc00feb50 msgr2=0x7ffbc0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 -- 172.21.15.5:0/2140783435 shutdown_connections 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.904+0000 7ffbc49ff700 1 -- 172.21.15.5:0/2140783435 wait complete. 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.906+0000 7ffbc49ff700 1 Processor -- start 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.906+0000 7ffbc49ff700 1 -- start start 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 0x7ffbc01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019e6b0 con 0x7ffbc019dc30 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019e820 con 0x7ffbc01083f0 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbc49ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc019e9e0 con 0x7ffbc01032d0 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52058/0 (socket says 172.21.15.5:52058) 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52318/0 (socket says 172.21.15.5:52318) 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbed9d700 1 -- 172.21.15.5:0/3906026094 learned_addr learned my addr 172.21.15.5:0/3906026094 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.907+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 0x7ffbc01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 -- 172.21.15.5:0/3906026094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 msgr2=0x7ffbc0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 -- 172.21.15.5:0/3906026094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 msgr2=0x7ffbc01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 0x7ffbc01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 -- 172.21.15.5:0/3906026094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8009d00 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 0x7ffbc01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.908+0000 7ffbbed9d700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b070 tx=0x7ffbb800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.909+0000 7ffbaf7fe700 1 -- 172.21.15.5:0/3906026094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800e040 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.909+0000 7ffbaf7fe700 1 -- 172.21.15.5:0/3906026094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb8003e10 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.909+0000 7ffbaf7fe700 1 -- 172.21.15.5:0/3906026094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80052a0 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.909+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc01a5ee0 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.909+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc01a62b0 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.916+0000 7ffbaf7fe700 1 -- 172.21.15.5:0/3906026094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7ffbb801d480 con 0x7ffbc019dc30 2023-09-23T04:47:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.916+0000 7ffbd8267b80 1 Processor -- start 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.917+0000 7ffbd8267b80 1 -- start start 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258abbb460 con 0x56258ac5dbd0 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258abacc20 con 0x56258ac52400 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.918+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258abaa540 con 0x56258ac52d20 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52630/0 (socket says 172.21.15.5:52630) 2023-09-23T04:47:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/333172669 learned_addr learned my addr 172.21.15.5:0/333172669 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbaf7fe700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 0x7ffba80a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 -- 172.21.15.5:0/333172669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 msgr2=0x56258ac527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 -- 172.21.15.5:0/333172669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 msgr2=0x56258ac5ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbbe59c700 1 -- 172.21.15.5:0/333172669 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258ac088d0 con 0x56258ac52d20 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.919+0000 7ffbaf7fe700 1 -- 172.21.15.5:0/3906026094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffbb80ce3a0 con 0x7ffbc019dc30 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.920+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac5d690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0010400 tx=0x7ffbb0010710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff66850fc9bd7ebd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.920+0000 7ffbacff9700 1 -- 172.21.15.5:0/333172669 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004a30 con 0x56258ac52d20 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.920+0000 7ffbacff9700 1 -- 172.21.15.5:0/333172669 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb0015070 con 0x56258ac52d20 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.920+0000 7ffbacff9700 1 -- 172.21.15.5:0/333172669 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb001a6e0 con 0x56258ac52d20 2023-09-23T04:47:10.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.920+0000 7ffbbed9d700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 0x7ffba80a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 -- 172.21.15.5:0/333172669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 msgr2=0x56258ac5d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac5d690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0010400 tx=0x7ffbb0010710 comp rx=0 tx=0).stop 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbbed9d700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 0x7ffba80a3d70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cb590 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 -- 172.21.15.5:0/333172669 shutdown_connections 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/333172669 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac5d690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.921+0000 7ffbd8267b80 1 -- 172.21.15.5:0/333172669 >> 172.21.15.5:0/333172669 conn(0x56258abc7260 msgr2=0x56258ac4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.925+0000 7ffbd8267b80 1 -- 172.21.15.5:0/333172669 shutdown_connections 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.925+0000 7ffbd8267b80 1 -- 172.21.15.5:0/333172669 wait complete. 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.925+0000 7ffbd8267b80 1 -- start start 2023-09-23T04:47:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.926+0000 7ffbd8267b80 1 Processor -- start 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52400 0x56258ac5d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258aba5e30 con 0x56258ac5dbd0 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ab42710 con 0x56258ac52d20 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ab423b0 con 0x56258ac52400 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.927+0000 7ffbd8267b80 10 client.0 _ll_register_callbacks cb 0x56258a99f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52644/0 (socket says 172.21.15.5:52644) 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 -- 172.21.15.5:0/2851535412 learned_addr learned my addr 172.21.15.5:0/2851535412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52400 0x56258ac5d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 -- 172.21.15.5:0/2851535412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52400 msgr2=0x56258ac5d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52400 0x56258ac5d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 -- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 msgr2=0x56258ac5b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.928+0000 7ffbbe59c700 1 -- 172.21.15.5:0/2851535412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb0007c50 con 0x56258ac52d20 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.929+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0004d00 tx=0x7ffbb0010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d9623a6e1dafd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.929+0000 7ffba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0020070 con 0x56258ac52d20 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffbb0004e00 con 0x56258ac52d20 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56258ac46250 con 0x56258ac52d20 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbb0004b00 con 0x56258ac52d20 2023-09-23T04:47:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffbd8267b80 10 client.25823 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.931+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56258a9d0de0 con 0x56258ac52d20 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56258ac46250 con 0x56258ac52d20 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffbb001aaa0 con 0x56258ac52d20 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 20 client.25823 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258ac6d5e0 0x56258ac6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 4 client.25823 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.932+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56258ab9ee10 con 0x56258ac6d5e0 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.933+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258ac6d5e0 0x56258ac6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.933+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258ac6d5e0 0x56258ac6fa00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a95538818f4212 server_cookie=4cbd5f751697cc22 in_seq=0 out_seq=0 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.933+0000 7ffba5ffb700 10 client.25823 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.934+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2851535412 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56258ab9ee10 con 0x56258ac6d5e0 2023-09-23T04:47:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.934+0000 7ffba5ffb700 10 client.25823 handle_command_reply: tid=0 2023-09-23T04:47:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.934+0000 7ffbd8267b80 1 client.25823 shutdown 2023-09-23T04:47:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 msgr2=0x56258ac5a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0004d00 tx=0x7ffbb0010f00 comp rx=0 tx=0).stop 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 shutdown_connections 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258ac6d5e0 0x56258ac6fa00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258ac5b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52400 0x56258ac5d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2851535412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52d20 0x56258ac5a8e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffba5ffb700 0 client.25823 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 >> 172.21.15.5:0/2851535412 conn(0x56258abc7260 msgr2=0x56258ac4e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 shutdown_connections 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.935+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2851535412 wait complete. 2023-09-23T04:47:10.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.936+0000 7ffbd8267b80 20 client.25823 trim_cache size 0 max 16384 2023-09-23T04:47:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.970+0000 7ffbd8267b80 1 Processor -- start 2023-09-23T04:47:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.970+0000 7ffbd8267b80 1 -- start start 2023-09-23T04:47:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 0x56258ac56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac5dbd0 0x56258ac5a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258aa26920 con 0x56258ac52d20 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ac53100 con 0x56258ac52400 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ac70060 con 0x56258ac5dbd0 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52650/0 (socket says 172.21.15.5:52650) 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2372628223 learned_addr learned my addr 172.21.15.5:0/2372628223 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.971+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac5dbd0 0x56258ac5a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 0x56258ac56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2372628223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac5dbd0 msgr2=0x56258ac5a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac5dbd0 0x56258ac5a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2372628223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 msgr2=0x56258ac56170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 0x56258ac56170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2372628223 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258ac4a480 con 0x56258ac52400 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 0x56258ac56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56258ac702d0 tx=0x7ffbb4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec07d3a23bbf344 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.972+0000 7ffbad7fa700 1 -- 172.21.15.5:0/2372628223 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4020a60 con 0x56258ac52400 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.973+0000 7ffbad7fa700 1 -- 172.21.15.5:0/2372628223 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb4027070 con 0x56258ac52400 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.973+0000 7ffbad7fa700 1 -- 172.21.15.5:0/2372628223 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4003970 con 0x56258ac52400 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.973+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2372628223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 msgr2=0x56258ac55c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.973+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56258ac702d0 tx=0x7ffbb4009260 comp rx=0 tx=0).stop 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2372628223 shutdown_connections 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac5dbd0 0x56258ac5a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac55c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2372628223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac52d20 0x56258ac56170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2372628223 >> 172.21.15.5:0/2372628223 conn(0x56258abc7260 msgr2=0x56258ac68e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2372628223 shutdown_connections 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2372628223 wait complete. 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.974+0000 7ffbd8267b80 1 -- start start 2023-09-23T04:47:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.975+0000 7ffbd8267b80 1 Processor -- start 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.975+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258acba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258acba950 con 0x56258ac5dbd0 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258acbaac0 con 0x56258ac52400 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258acbac80 con 0x56258ac52d20 2023-09-23T04:47:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbd8267b80 10 client.0 _ll_register_callbacks cb 0x7ffbb4009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbbdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52666/0 (socket says 172.21.15.5:52666) 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2910510823 learned_addr learned my addr 172.21.15.5:0/2910510823 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.976+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258acba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2910510823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 msgr2=0x56258ac62e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac62e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 msgr2=0x56258acba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258acba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 -- 172.21.15.5:0/2910510823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb401a040 con 0x56258ac52400 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:47:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffbbdd9b700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009c90 tx=0x7ffbb4009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66467cdba2461b1a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4004020 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.977+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffbb4004180 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.978+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbb0015070 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.978+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbb4004340 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.978+0000 7ffbd8267b80 10 client.25829 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.978+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56258a9b9c60 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb0015070 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffbb40049c0 con 0x56258ac52400 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 10 client.25829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 20 client.25829 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258accaac0 0x56258acccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 4 client.25829 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.979+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56258ab9eb30 con 0x56258accaac0 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.980+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258accaac0 0x56258acccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.980+0000 7ffbbe59c700 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258accaac0 0x56258acccee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32f1dc967df7dbc server_cookie=e0b705d09ce51868 in_seq=0 out_seq=0 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.980+0000 7ffba5ffb700 10 client.25829 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffba5ffb700 1 -- 172.21.15.5:0/2910510823 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56258ab9eb30 con 0x56258accaac0 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffba5ffb700 10 client.25829 handle_command_reply: tid=0 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 client.25829 shutdown 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 msgr2=0x56258ac628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009c90 tx=0x7ffbb4009f30 comp rx=0 tx=0).stop 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 shutdown_connections 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56258accaac0 0x56258acccee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56258ac5dbd0 0x56258acba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56258ac52400 0x56258ac628f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 --2- 172.21.15.5:0/2910510823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56258ac52d20 0x56258ac62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 >> 172.21.15.5:0/2910510823 conn(0x56258abc7260 msgr2=0x56258ac55c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffba5ffb700 0 client.25829 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.981+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 shutdown_connections 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.982+0000 7ffbd8267b80 1 -- 172.21.15.5:0/2910510823 wait complete. 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.982+0000 7ffbd8267b80 20 client.25829 trim_cache size 0 max 16384 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:47:10.986 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 msgr2=0x7ffba80a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 0x7ffba80a3d70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80cb590 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 msgr2=0x7ffbc019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b070 tx=0x7ffbb800bc80 comp rx=0 tx=0).stop 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 shutdown_connections 2023-09-23T04:47:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffba80a18b0 0x7ffba80a3d70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbc019dc30 0x7ffbc019e0b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffbc01032d0 0x7ffbc0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 --2- 172.21.15.5:0/3906026094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc01083f0 0x7ffbc01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 >> 172.21.15.5:0/3906026094 conn(0x7ffbc00feb50 msgr2=0x7ffbc00ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 shutdown_connections 2023-09-23T04:47:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:10.983+0000 7ffbc49ff700 1 -- 172.21.15.5:0/3906026094 wait complete. 2023-09-23T04:47:11.012 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:47:11.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 -- 172.21.15.5:0/677499107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb41017d0 msgr2=0x7f7fb4105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/677499107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb41017d0 0x7f7fb4105810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009990 tx=0x7f7fa8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 -- 172.21.15.5:0/677499107 shutdown_connections 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/677499107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4105f40 0x7f7fb410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/677499107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb4100e20 0x7f7fb4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/677499107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb41017d0 0x7f7fb4105810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 -- 172.21.15.5:0/677499107 >> 172.21.15.5:0/677499107 conn(0x7f7fb40fc680 msgr2=0x7f7fb40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 -- 172.21.15.5:0/677499107 shutdown_connections 2023-09-23T04:47:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.357+0000 7f7fb89fe700 1 -- 172.21.15.5:0/677499107 wait complete. 2023-09-23T04:47:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.359+0000 7f7fb89fe700 1 Processor -- start 2023-09-23T04:47:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.359+0000 7f7fb89fe700 1 -- start start 2023-09-23T04:47:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.359+0000 7f7fb89fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb89fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb4105f40 0x7f7fb4072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb89fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 0x7f7fb406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:11.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb89fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb406bd20 con 0x7f7fb4100e20 2023-09-23T04:47:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb89fe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb406be90 con 0x7f7fb406b260 2023-09-23T04:47:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb89fe700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb406c050 con 0x7f7fb4105f40 2023-09-23T04:47:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:11.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 0x7f7fb406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52088/0 (socket says 172.21.15.5:52088) 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb2d9d700 1 -- 172.21.15.5:0/973041559 learned_addr learned my addr 172.21.15.5:0/973041559 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.360+0000 7f7fb259c700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb4105f40 0x7f7fb4072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 -- 172.21.15.5:0/973041559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb4105f40 msgr2=0x7f7fb4072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb4105f40 0x7f7fb4072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 -- 172.21.15.5:0/973041559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 msgr2=0x7f7fb406b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 0x7f7fb406b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 -- 172.21.15.5:0/973041559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80095f0 con 0x7f7fb4100e20 2023-09-23T04:47:11.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb359e700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 0x7f7fb406b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7fb2d9d700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9a0 tx=0x7f7fa400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.361+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4009790 con 0x7f7fb4100e20 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.362+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb406fea0 con 0x7f7fb4100e20 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.362+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fa4010430 con 0x7f7fb4100e20 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.362+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4018680 con 0x7f7fb4100e20 2023-09-23T04:47:11.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.362+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40703d0 con 0x7f7fb4100e20 2023-09-23T04:47:11.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.363+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb404eb20 con 0x7f7fb4100e20 2023-09-23T04:47:11.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.367+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f7fa40105a0 con 0x7f7fb4100e20 2023-09-23T04:47:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.370+0000 7f7f9bfff700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 0x7f7f9c0a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.370+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7fa4014070 con 0x7f7fb4100e20 2023-09-23T04:47:11.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.370+0000 7f7fb259c700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 0x7f7f9c0a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.371+0000 7f7fb259c700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 0x7f7f9c0a3d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fb406cb90 tx=0x7f7fa8005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:11.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.374+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7fa40970b0 con 0x7f7fb4100e20 2023-09-23T04:47:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.883+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fb41017d0 con 0x7f7fb4100e20 2023-09-23T04:47:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.884+0000 7f7f9bfff700 1 -- 172.21.15.5:0/973041559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f7fa409a6d0 con 0x7f7fb4100e20 2023-09-23T04:47:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:47:11.884 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:44:20.444513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:47:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.887+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 msgr2=0x7f7f9c0a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.887+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 0x7f7f9c0a3d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7fb406cb90 tx=0x7f7fa8005fd0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.887+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 msgr2=0x7f7fb40722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.887+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9a0 tx=0x7f7fa400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 shutdown_connections 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7f9c0a18b0 0x7f7f9c0a3d70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fb406b260 0x7f7fb406b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb4100e20 0x7f7fb40722c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 --2- 172.21.15.5:0/973041559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7fb4105f40 0x7f7fb4072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 >> 172.21.15.5:0/973041559 conn(0x7f7fb40fc680 msgr2=0x7f7fb40fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 shutdown_connections 2023-09-23T04:47:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:11.888+0000 7f7fb89fe700 1 -- 172.21.15.5:0/973041559 wait complete. 2023-09-23T04:47:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2023-09-23T04:47:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:11 smithi005 ceph-mon[97638]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:11 smithi118 ceph-mon[134297]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:11.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:47:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:11 smithi069 ceph-mon[133931]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-09-23T04:47:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/1350892021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf18106210 msgr2=0x7fdf1810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/1350892021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf18106210 0x7fdf1810dd90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b0a0 tx=0x7fdf1000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/1350892021 shutdown_connections 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/1350892021 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf18106210 0x7fdf1810dd90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/1350892021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf181010f0 0x7fdf181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/1350892021 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf18101aa0 0x7fdf18105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.264+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/1350892021 >> 172.21.15.5:0/1350892021 conn(0x7fdf180fc9d0 msgr2=0x7fdf180fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.265+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/1350892021 shutdown_connections 2023-09-23T04:47:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.265+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/1350892021 wait complete. 2023-09-23T04:47:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.266+0000 7fdf1d1ff700 1 Processor -- start 2023-09-23T04:47:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.266+0000 7fdf1d1ff700 1 -- start start 2023-09-23T04:47:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf18106210 0x7fdf181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1819db00 0x7fdf1819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1819e580 con 0x7fdf18106210 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1819e6f0 con 0x7fdf1819db00 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf1d1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1819e8b0 con 0x7fdf181010f0 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.267+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52374/0 (socket says 172.21.15.5:52374) 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 -- 172.21.15.5:0/2738915575 learned_addr learned my addr 172.21.15.5:0/2738915575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf1659c700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf18106210 0x7fdf181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf1759e700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1819db00 0x7fdf1819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 -- 172.21.15.5:0/2738915575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1819db00 msgr2=0x7fdf1819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1819db00 0x7fdf1819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 -- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf18106210 msgr2=0x7fdf181996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf18106210 0x7fdf181996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.268+0000 7fdf16d9d700 1 -- 172.21.15.5:0/2738915575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009d00 con 0x7fdf181010f0 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9a0 tx=0x7fdf0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdefffff700 1 -- 172.21.15.5:0/2738915575 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800ca10 con 0x7fdf181010f0 2023-09-23T04:47:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdefffff700 1 -- 172.21.15.5:0/2738915575 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0800cb70 con 0x7fdf181010f0 2023-09-23T04:47:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdefffff700 1 -- 172.21.15.5:0/2738915575 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08018680 con 0x7fdf181010f0 2023-09-23T04:47:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf181a5e10 con 0x7fdf181010f0 2023-09-23T04:47:12.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.269+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf181a6240 con 0x7fdf181010f0 2023-09-23T04:47:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.275+0000 7fdefffff700 1 -- 172.21.15.5:0/2738915575 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7fdf0800cce0 con 0x7fdf181010f0 2023-09-23T04:47:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.276+0000 7fdf30b61b80 1 Processor -- start 2023-09-23T04:47:12.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.276+0000 7fdf30b61b80 1 -- start start 2023-09-23T04:47:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 0x55eb3ed507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed643e0 0x55eb3ed667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ecb9460 con 0x55eb3ed50d20 2023-09-23T04:47:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ecaac20 con 0x55eb3ed643e0 2023-09-23T04:47:12.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3eca8540 con 0x55eb3ed50400 2023-09-23T04:47:12.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.277+0000 7fdefffff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 0x7fdf000a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdefffff700 1 -- 172.21.15.5:0/2738915575 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fdf08014070 con 0x7fdf181010f0 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf1759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 0x55eb3ed507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf1759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 0x55eb3ed507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52388/0 (socket says 172.21.15.5:52388) 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf1759e700 1 -- 172.21.15.5:0/3703986391 learned_addr learned my addr 172.21.15.5:0/3703986391 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf1759e700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 0x7fdf000a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.278+0000 7fdf1659c700 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed643e0 0x55eb3ed667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 -- 172.21.15.5:0/3703986391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 msgr2=0x55eb3ed507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 0x55eb3ed507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 -- 172.21.15.5:0/3703986391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed643e0 msgr2=0x55eb3ed667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed643e0 0x55eb3ed667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 -- 172.21.15.5:0/3703986391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb3ed068d0 con 0x55eb3ed50d20 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cab70 tx=0x7fdf08010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9de901dfe47f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.279+0000 7fdf1759e700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 0x7fdf000a3d70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009850 tx=0x7fdf100061f0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.280+0000 7fdefd7fa700 1 -- 172.21.15.5:0/3703986391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0809a9c0 con 0x55eb3ed50d20 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.280+0000 7fdefd7fa700 1 -- 172.21.15.5:0/3703986391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0809a2c0 con 0x55eb3ed50d20 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.280+0000 7fdefd7fa700 1 -- 172.21.15.5:0/3703986391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080a21c0 con 0x55eb3ed50d20 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 -- 172.21.15.5:0/3703986391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 msgr2=0x55eb3ed63ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf080cab70 tx=0x7fdf08010be0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 -- 172.21.15.5:0/3703986391 shutdown_connections 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed643e0 0x55eb3ed667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50400 0x55eb3ed507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/3703986391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63ea0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 -- 172.21.15.5:0/3703986391 >> 172.21.15.5:0/3703986391 conn(0x55eb3ecc5260 msgr2=0x55eb3ed4c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 -- 172.21.15.5:0/3703986391 shutdown_connections 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.281+0000 7fdf30b61b80 1 -- 172.21.15.5:0/3703986391 wait complete. 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.282+0000 7fdf30b61b80 1 -- start start 2023-09-23T04:47:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.282+0000 7fdf30b61b80 1 Processor -- start 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed61070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3eca3e30 con 0x55eb3ed643e0 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ec40710 con 0x55eb3ed50400 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ec403b0 con 0x55eb3ed50d20 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.283+0000 7fdf30b61b80 10 client.0 _ll_register_callbacks cb 0x55eb3ea9d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52722/0 (socket says 172.21.15.5:52722) 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 -- 172.21.15.5:0/1467167176 learned_addr learned my addr 172.21.15.5:0/1467167176 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf1759e700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 -- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 msgr2=0x55eb3ed61070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed61070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 -- 172.21.15.5:0/1467167176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 msgr2=0x55eb3ed61760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed61760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.284+0000 7fdf16d9d700 1 -- 172.21.15.5:0/1467167176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf181a6240 con 0x55eb3ed50400 2023-09-23T04:47:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.285+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf080a2120 tx=0x7fdf08010ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c51faac2305ec8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.285+0000 7fdefaff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:12.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.285+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080a23d0 con 0x55eb3ed50400 2023-09-23T04:47:12.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.285+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fdf080b2be0 con 0x55eb3ed50400 2023-09-23T04:47:12.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.289+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb3ed44250 con 0x55eb3ed50400 2023-09-23T04:47:12.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.289+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf080b2da0 con 0x55eb3ed50400 2023-09-23T04:47:12.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.289+0000 7fdf30b61b80 10 client.25847 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:12.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.289+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55eb3eacede0 con 0x55eb3ed50400 2023-09-23T04:47:12.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.289+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb3ed44250 con 0x55eb3ed50400 2023-09-23T04:47:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.290+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf080992b0 con 0x55eb3ed50400 2023-09-23T04:47:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 10 client.25847 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:47:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 20 client.25847 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3ed739a0 0x55eb3ed75d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 4 client.25847 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb3ec9ce10 con 0x55eb3ed739a0 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.291+0000 7fdf1659c700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3ed739a0 0x55eb3ed75d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.292+0000 7fdf1659c700 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3ed739a0 0x55eb3ed75d90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf60f0648d0c4fc2 server_cookie=1f4945b389db6f6 in_seq=0 out_seq=0 2023-09-23T04:47:12.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.292+0000 7fdefaff5700 10 client.25847 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:12.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.292+0000 7fdefaff5700 1 -- 172.21.15.5:0/1467167176 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eb3ec9ce10 con 0x55eb3ed739a0 2023-09-23T04:47:12.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.292+0000 7fdefaff5700 10 client.25847 handle_command_reply: tid=0 2023-09-23T04:47:12.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.292+0000 7fdf30b61b80 1 client.25847 shutdown 2023-09-23T04:47:12.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 msgr2=0x55eb3ed63e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf080a2120 tx=0x7fdf08010ae0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 shutdown_connections 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3ed739a0 0x55eb3ed75d90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed61760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed63e60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdefaff5700 0 client.25847 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1467167176 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed61070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 >> 172.21.15.5:0/1467167176 conn(0x55eb3ecc5260 msgr2=0x55eb3ed4c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 shutdown_connections 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.293+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1467167176 wait complete. 2023-09-23T04:47:12.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.294+0000 7fdf30b61b80 20 client.25847 trim_cache size 0 max 16384 2023-09-23T04:47:12.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 Processor -- start 2023-09-23T04:47:12.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 -- start start 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3eb24920 con 0x55eb3ed643e0 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ed51bf0 con 0x55eb3ed50400 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.320+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ed532a0 con 0x55eb3ed50d20 2023-09-23T04:47:12.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52730/0 (socket says 172.21.15.5:52730) 2023-09-23T04:47:12.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf1759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed60910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf16d9d700 1 -- 172.21.15.5:0/385266735 learned_addr learned my addr 172.21.15.5:0/385266735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf1659c700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf16d9d700 1 -- 172.21.15.5:0/385266735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 msgr2=0x55eb3ed63990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.321+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf16d9d700 1 -- 172.21.15.5:0/385266735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 msgr2=0x55eb3ed60910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed60910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf16d9d700 1 -- 172.21.15.5:0/385266735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb3ed48480 con 0x55eb3ed50400 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf1659c700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf1759e700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf08098610 tx=0x7fdf08010b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e6ff9c1d931143 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdefdffb700 1 -- 172.21.15.5:0/385266735 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0809e410 con 0x55eb3ed50400 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdefdffb700 1 -- 172.21.15.5:0/385266735 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf080a23d0 con 0x55eb3ed50400 2023-09-23T04:47:12.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.322+0000 7fdefdffb700 1 -- 172.21.15.5:0/385266735 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08096310 con 0x55eb3ed50400 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 -- 172.21.15.5:0/385266735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 msgr2=0x55eb3ed60300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf08098610 tx=0x7fdf08010b50 comp rx=0 tx=0).stop 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 -- 172.21.15.5:0/385266735 shutdown_connections 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3ed60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed60300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/385266735 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed63990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 -- 172.21.15.5:0/385266735 >> 172.21.15.5:0/385266735 conn(0x55eb3ecc5260 msgr2=0x55eb3ed5c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 -- 172.21.15.5:0/385266735 shutdown_connections 2023-09-23T04:47:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.323+0000 7fdf30b61b80 1 -- 172.21.15.5:0/385266735 wait complete. 2023-09-23T04:47:12.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.324+0000 7fdf30b61b80 1 -- start start 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.324+0000 7fdf30b61b80 1 Processor -- start 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed69610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3ed69db0 con 0x55eb3ed643e0 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3edc6d20 con 0x55eb3ed50400 2023-09-23T04:47:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb3edc6eb0 con 0x55eb3ed50d20 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf30b61b80 10 client.0 _ll_register_callbacks cb 0x55eb3ea9d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.325+0000 7fdf16d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52134/0 (socket says 172.21.15.5:52134) 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 -- 172.21.15.5:0/1878297142 learned_addr learned my addr 172.21.15.5:0/1878297142 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1659c700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed69610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 -- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 msgr2=0x55eb3ed69610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed69610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 -- 172.21.15.5:0/1878297142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 msgr2=0x55eb3ed690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.326+0000 7fdf1759e700 1 -- 172.21.15.5:0/1878297142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c004260 con 0x55eb3ed643e0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.327+0000 7fdf1759e700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b0a0 tx=0x7fdf1000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0dcfb4994a12e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.327+0000 7fdefaff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.327+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000e040 con 0x55eb3ed643e0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.327+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fdf1000c640 con 0x55eb3ed643e0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.327+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf0809a2c0 con 0x55eb3ed643e0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.328+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf1000c800 con 0x55eb3ed643e0 2023-09-23T04:47:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.328+0000 7fdf30b61b80 10 client.15864 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:12.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.328+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fdf10028040 con 0x55eb3ed643e0 2023-09-23T04:47:12.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.328+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0809a2c0 con 0x55eb3ed643e0 2023-09-23T04:47:12.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdf1001fcd0 con 0x55eb3ed643e0 2023-09-23T04:47:12.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:12.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 10 client.15864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:47:12.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 20 client.15864 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3edc8630 0x55eb3edcaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 4 client.15864 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb3ec9cb30 con 0x55eb3edc8630 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.329+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3edc8630 0x55eb3edcaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf16d9d700 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3edc8630 0x55eb3edcaa50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4309c6dae1969605 server_cookie=c8cef02f70bd1ab3 in_seq=0 out_seq=0 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdefaff5700 10 client.15864 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdefaff5700 1 -- 172.21.15.5:0/1878297142 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb3ec9cb30 con 0x55eb3edc8630 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdefaff5700 10 client.15864 handle_command_reply: tid=0 2023-09-23T04:47:12.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 client.15864 shutdown 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 msgr2=0x55eb3edc67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b0a0 tx=0x7fdf1000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 shutdown_connections 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55eb3edc8630 0x55eb3edcaa50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3ed643e0 0x55eb3edc67e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb3ed50400 0x55eb3ed690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdefaff5700 0 client.15864 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 --2- 172.21.15.5:0/1878297142 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55eb3ed50d20 0x55eb3ed69610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.330+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 >> 172.21.15.5:0/1878297142 conn(0x55eb3ecc5260 msgr2=0x55eb3ed608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.331+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 shutdown_connections 2023-09-23T04:47:12.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.331+0000 7fdf30b61b80 1 -- 172.21.15.5:0/1878297142 wait complete. 2023-09-23T04:47:12.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.331+0000 7fdf30b61b80 20 client.15864 trim_cache size 0 max 16384 2023-09-23T04:47:12.335 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 msgr2=0x7fdf000a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 0x7fdf000a3d70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009850 tx=0x7fdf100061f0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 msgr2=0x7fdf18199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9a0 tx=0x7fdf0800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 shutdown_connections 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdf000a18b0 0x7fdf000a3d70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf1819db00 0x7fdf1819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdf181010f0 0x7fdf18199160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 --2- 172.21.15.5:0/2738915575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf18106210 0x7fdf181996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 >> 172.21.15.5:0/2738915575 conn(0x7fdf180fc9d0 msgr2=0x7fdf18102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 shutdown_connections 2023-09-23T04:47:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.332+0000 7fdf1d1ff700 1 -- 172.21.15.5:0/2738915575 wait complete. 2023-09-23T04:47:12.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:47:12.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:12 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/973041559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/1547436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 msgr2=0x7f1fd81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/1547436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd81014c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009990 tx=0x7f1fc8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/1547436652 shutdown_connections 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/1547436652 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/1547436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd81014c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/1547436652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd8101a90 0x7f1fd8105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/1547436652 >> 172.21.15.5:0/1547436652 conn(0x7f1fd80fc980 msgr2=0x7f1fd80feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.709+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/1547436652 shutdown_connections 2023-09-23T04:47:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.710+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/1547436652 wait complete. 2023-09-23T04:47:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.711+0000 7f1fdd1ff700 1 Processor -- start 2023-09-23T04:47:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.711+0000 7f1fdd1ff700 1 -- start start 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 0x7f1fd819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819c3f0 con 0x7f1fd81010e0 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819c560 con 0x7f1fd819b970 2023-09-23T04:47:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fdd1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd819c720 con 0x7f1fd8106200 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 0x7f1fd819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 0x7f1fd819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52764/0 (socket says 172.21.15.5:52764) 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fd759e700 1 -- 172.21.15.5:0/3076031962 learned_addr learned my addr 172.21.15.5:0/3076031962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.712+0000 7f1fd659c700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/3076031962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 msgr2=0x7f1fd8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/3076031962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 msgr2=0x7f1fd819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 0x7f1fd819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/3076031962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80095f0 con 0x7f1fd81010e0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd659c700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.713+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd8196fd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8000c00 tx=0x7f1fc8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.714+0000 7f1fbffff700 1 -- 172.21.15.5:0/3076031962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc801d070 con 0x7f1fd81010e0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.714+0000 7f1fbffff700 1 -- 172.21.15.5:0/3076031962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc8021410 con 0x7f1fd81010e0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.714+0000 7f1fbffff700 1 -- 172.21.15.5:0/3076031962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8021ea0 con 0x7f1fd81010e0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.714+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd8073050 con 0x7f1fd81010e0 2023-09-23T04:47:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.714+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80734b0 con 0x7f1fd81010e0 2023-09-23T04:47:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.720+0000 7f1fbffff700 1 -- 172.21.15.5:0/3076031962 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f1fc8021580 con 0x7f1fd81010e0 2023-09-23T04:47:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.721+0000 7f1ff0bddb80 1 Processor -- start 2023-09-23T04:47:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.721+0000 7f1ff0bddb80 1 -- start start 2023-09-23T04:47:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.721+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 0x56109add67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.721+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 0x56109ade1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.721+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109ade3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ad3f460 con 0x56109ade1bd0 2023-09-23T04:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ad30c20 con 0x56109add6d20 2023-09-23T04:47:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ad2e540 con 0x56109add6400 2023-09-23T04:47:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 0x56109add67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 0x56109add67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52446/0 (socket says 172.21.15.5:52446) 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1fd759e700 1 -- 172.21.15.5:0/1471956474 learned_addr learned my addr 172.21.15.5:0/1471956474 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.722+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 0x56109ade1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/1471956474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 msgr2=0x56109add67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 0x56109add67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/1471956474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 msgr2=0x56109ade3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109ade3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 -- 172.21.15.5:0/1471956474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56109ad8c8d0 con 0x56109add6d20 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 0x56109ade1690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80cbc00 tx=0x7f1fc80d2c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad83ab2808a7b0bc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.723+0000 7f1fbffff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 0x7f1fc00a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.724+0000 7f1fbffff700 1 -- 172.21.15.5:0/3076031962 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1fc80cea70 con 0x7f1fd81010e0 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.724+0000 7f1fbd7fa700 1 -- 172.21.15.5:0/1471956474 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80d25d0 con 0x56109add6d20 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.724+0000 7f1fbd7fa700 1 -- 172.21.15.5:0/1471956474 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc8097720 con 0x56109add6d20 2023-09-23T04:47:12.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.724+0000 7f1fbd7fa700 1 -- 172.21.15.5:0/1471956474 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc809bae0 con 0x56109add6d20 2023-09-23T04:47:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.724+0000 7f1fd759e700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 0x7f1fc00a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1471956474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 msgr2=0x56109ade1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 0x56109ade1690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80cbc00 tx=0x7f1fc80d2c20 comp rx=0 tx=0).stop 2023-09-23T04:47:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1471956474 shutdown_connections 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109ade3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6400 0x56109add67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1471956474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6d20 0x56109ade1690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1471956474 >> 172.21.15.5:0/1471956474 conn(0x56109ad4b260 msgr2=0x56109add2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1471956474 shutdown_connections 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1471956474 wait complete. 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.725+0000 7f1fd759e700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 0x7f1fc00a3d70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fa0 tx=0x7f1fd0009040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.726+0000 7f1ff0bddb80 1 -- start start 2023-09-23T04:47:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 Processor -- start 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109aded030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ad29e30 con 0x56109ade1bd0 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109acc6710 con 0x56109add6400 2023-09-23T04:47:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109acc63b0 con 0x56109add6d20 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.727+0000 7f1ff0bddb80 10 client.0 _ll_register_callbacks cb 0x56109ab23800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52448/0 (socket says 172.21.15.5:52448) 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 -- 172.21.15.5:0/2812492749 learned_addr learned my addr 172.21.15.5:0/2812492749 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd759e700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109aded030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 -- 172.21.15.5:0/2812492749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 msgr2=0x56109ade1fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade1fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 -- 172.21.15.5:0/2812492749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 msgr2=0x56109aded030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.728+0000 7f1fd659c700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109aded030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1fd659c700 1 -- 172.21.15.5:0/2812492749 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd80734b0 con 0x56109add6d20 2023-09-23T04:47:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1fd659c700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc011660 tx=0x7f1fcc011970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41777a8b103fd6ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1fbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc012860 con 0x56109add6d20 2023-09-23T04:47:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1fcc0129c0 con 0x56109add6d20 2023-09-23T04:47:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.729+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56109adca250 con 0x56109add6d20 2023-09-23T04:47:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.730+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fcc012b80 con 0x56109add6d20 2023-09-23T04:47:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.730+0000 7f1ff0bddb80 10 client.25906 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.730+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56109ab54de0 con 0x56109add6d20 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.730+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56109adca250 con 0x56109add6d20 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.730+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1fcc01a3f0 con 0x56109add6d20 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 10 client.25906 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 10 client.25906 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 20 client.25906 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adf9be0 0x56109adfc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 4 client.25906 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56109ad22e10 con 0x56109adf9be0 2023-09-23T04:47:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.731+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adf9be0 0x56109adfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.732+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adf9be0 0x56109adfc000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab2d6536b5ece207 server_cookie=46d193cdb7383124 in_seq=0 out_seq=0 2023-09-23T04:47:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.732+0000 7f1fbaff5700 10 client.25906 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.732+0000 7f1fbaff5700 1 -- 172.21.15.5:0/2812492749 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56109ad22e10 con 0x56109adf9be0 2023-09-23T04:47:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.732+0000 7f1fbaff5700 10 client.25906 handle_command_reply: tid=0 2023-09-23T04:47:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 client.25906 shutdown 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 msgr2=0x56109ade24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc011660 tx=0x7f1fcc011970 comp rx=0 tx=0).stop 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 shutdown_connections 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adf9be0 0x56109adfc000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109aded030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/2812492749 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109ade24f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 >> 172.21.15.5:0/2812492749 conn(0x56109ad4b260 msgr2=0x56109add2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1fbaff5700 0 client.25906 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 shutdown_connections 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.733+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/2812492749 wait complete. 2023-09-23T04:47:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.734+0000 7f1ff0bddb80 20 client.25906 trim_cache size 0 max 16384 2023-09-23T04:47:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.760+0000 7f1ff0bddb80 1 Processor -- start 2023-09-23T04:47:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.760+0000 7f1ff0bddb80 1 -- start start 2023-09-23T04:47:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.760+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109adec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109abaa920 con 0x56109ade1bd0 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109adced60 con 0x56109add6400 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109add7bf0 con 0x56109add6d20 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52184/0 (socket says 172.21.15.5:52184) 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.761+0000 7f1fd759e700 1 -- 172.21.15.5:0/1134270251 learned_addr learned my addr 172.21.15.5:0/1134270251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 -- 172.21.15.5:0/1134270251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 msgr2=0x56109adec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd659c700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109adec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109adec730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 -- 172.21.15.5:0/1134270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 msgr2=0x56109ade0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 -- 172.21.15.5:0/1134270251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56109ae00c50 con 0x56109ade1bd0 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0016590 tx=0x7f1fd000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb926354c1eb7c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.762+0000 7f1fbdffb700 1 -- 172.21.15.5:0/1134270251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0003a00 con 0x56109ade1bd0 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.763+0000 7f1fbdffb700 1 -- 172.21.15.5:0/1134270251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fd000f040 con 0x56109ade1bd0 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.763+0000 7f1fbdffb700 1 -- 172.21.15.5:0/1134270251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0024650 con 0x56109ade1bd0 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.763+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1134270251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 msgr2=0x56109adecc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.763+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0016590 tx=0x7f1fd000b050 comp rx=0 tx=0).stop 2023-09-23T04:47:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1134270251 shutdown_connections 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109ade1bd0 0x56109adecc70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1134270251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109add6d20 0x56109adec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1134270251 >> 172.21.15.5:0/1134270251 conn(0x56109ad4b260 msgr2=0x56109ade1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1134270251 shutdown_connections 2023-09-23T04:47:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.764+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1134270251 wait complete. 2023-09-23T04:47:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.765+0000 7f1ff0bddb80 1 -- start start 2023-09-23T04:47:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.765+0000 7f1ff0bddb80 1 Processor -- start 2023-09-23T04:47:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 0x56109ade5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ae46f40 con 0x56109add6d20 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ae470b0 con 0x56109add6400 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56109ae47270 con 0x56109ade1bd0 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1ff0bddb80 10 client.0 _ll_register_callbacks cb 0x7f1fcc009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 0x56109ade5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1fd759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52458/0 (socket says 172.21.15.5:52458) 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.766+0000 7f1fd759e700 1 -- 172.21.15.5:0/1226880037 learned_addr learned my addr 172.21.15.5:0/1226880037 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 -- 172.21.15.5:0/1226880037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 msgr2=0x56109ade4dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 -- 172.21.15.5:0/1226880037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 msgr2=0x56109ade5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 0x56109ade5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 -- 172.21.15.5:0/1226880037 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd0007d70 con 0x56109ade1bd0 2023-09-23T04:47:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd659c700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 0x56109ade5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fd759e700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009550 tx=0x7f1fd0003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a5d3ed91631b3e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fbaff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004380 con 0x56109ade1bd0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fc8097720 con 0x56109ade1bd0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.767+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1fd00044e0 con 0x56109ade1bd0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd002c400 con 0x56109ade1bd0 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 10 client.25909 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56109ab3dc60 con 0x56109ade1bd0 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fc8097720 con 0x56109ade1bd0 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1fd002a030 con 0x56109ade1bd0 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 10 client.25909 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 10 client.25909 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 20 client.25909 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adef750 0x56109adf1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 4 client.25909 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.768+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56109ad22b30 con 0x56109adef750 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.769+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adef750 0x56109adf1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.769+0000 7f1fd6d9d700 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adef750 0x56109adf1b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27a74a901183e999 server_cookie=5588bc24b1ceb008 in_seq=0 out_seq=0 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.769+0000 7f1fbaff5700 10 client.25909 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1fbaff5700 1 -- 172.21.15.5:0/1226880037 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56109ad22b30 con 0x56109adef750 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1fbaff5700 10 client.25909 handle_command_reply: tid=0 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 client.25909 shutdown 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 msgr2=0x56109ae46a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009550 tx=0x7f1fd0003c30 comp rx=0 tx=0).stop 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 shutdown_connections 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56109ade1bd0 0x56109ae46a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56109add6400 0x56109ade4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56109adef750 0x56109adf1b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 --2- 172.21.15.5:0/1226880037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56109add6d20 0x56109ade5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 >> 172.21.15.5:0/1226880037 conn(0x56109ad4b260 msgr2=0x56109adda300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1fbaff5700 0 client.25909 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 shutdown_connections 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.770+0000 7f1ff0bddb80 1 -- 172.21.15.5:0/1226880037 wait complete. 2023-09-23T04:47:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.771+0000 7f1ff0bddb80 20 client.25909 trim_cache size 0 max 16384 2023-09-23T04:47:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.771+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 msgr2=0x7f1fc00a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.771+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 0x7f1fc00a3d70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fa0 tx=0x7f1fd0009040 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.771+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 msgr2=0x7f1fd8196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.771+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd8196fd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8000c00 tx=0x7f1fc8005740 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 shutdown_connections 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1fc00a18b0 0x7f1fc00a3d70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1fd819b970 0x7f1fd819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fd81010e0 0x7f1fd8196fd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 --2- 172.21.15.5:0/3076031962 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1fd8106200 0x7f1fd8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 >> 172.21.15.5:0/3076031962 conn(0x7f1fd80fc980 msgr2=0x7f1fd80fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 shutdown_connections 2023-09-23T04:47:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:12.772+0000 7f1fdd1ff700 1 -- 172.21.15.5:0/3076031962 wait complete. 2023-09-23T04:47:12.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:47:12.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:12 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/973041559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:12 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/973041559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 -- 172.21.15.5:0/214536524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 msgr2=0x7f97a010d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 --2- 172.21.15.5:0/214536524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a010d340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9794009990 tx=0x7f9794009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 -- 172.21.15.5:0/214536524 shutdown_connections 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 --2- 172.21.15.5:0/214536524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 0x7f97a010fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 --2- 172.21.15.5:0/214536524 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 0x7f97a00684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 --2- 172.21.15.5:0/214536524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a010d340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 -- 172.21.15.5:0/214536524 >> 172.21.15.5:0/214536524 conn(0x7f97a0076810 msgr2=0x7f97a0076c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 -- 172.21.15.5:0/214536524 shutdown_connections 2023-09-23T04:47:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.149+0000 7f97a579d700 1 -- 172.21.15.5:0/214536524 wait complete. 2023-09-23T04:47:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.151+0000 7f97a579d700 1 Processor -- start 2023-09-23T04:47:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.151+0000 7f97a579d700 1 -- start start 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 0x7f97a0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 0x7f97a019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a019de90 con 0x7f97a0068a10 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a019e000 con 0x7f97a010d880 2023-09-23T04:47:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a579d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a019e1c0 con 0x7f97a00680f0 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f979f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f97a4f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 0x7f97a019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f979f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52198/0 (socket says 172.21.15.5:52198) 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f979f7fe700 1 -- 172.21.15.5:0/3830185819 learned_addr learned my addr 172.21.15.5:0/3830185819 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.152+0000 7f979ffff700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 0x7f97a0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 -- 172.21.15.5:0/3830185819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 msgr2=0x7f97a0198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 0x7f97a0198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 -- 172.21.15.5:0/3830185819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 msgr2=0x7f97a019d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 0x7f97a019d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 -- 172.21.15.5:0/3830185819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940095f0 con 0x7f97a0068a10 2023-09-23T04:47:13.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.153+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9794005910 tx=0x7f9794005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.154+0000 7f979d7fa700 1 -- 172.21.15.5:0/3830185819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979401d070 con 0x7f97a0068a10 2023-09-23T04:47:13.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.154+0000 7f979d7fa700 1 -- 172.21.15.5:0/3830185819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9794022470 con 0x7f97a0068a10 2023-09-23T04:47:13.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.154+0000 7f979d7fa700 1 -- 172.21.15.5:0/3830185819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794017650 con 0x7f97a0068a10 2023-09-23T04:47:13.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.154+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a019e440 con 0x7f97a0068a10 2023-09-23T04:47:13.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.154+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a01020c0 con 0x7f97a0068a10 2023-09-23T04:47:13.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.160+0000 7f979d7fa700 1 -- 172.21.15.5:0/3830185819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f979400fb00 con 0x7f97a0068a10 2023-09-23T04:47:13.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.160+0000 7f97ba23bb80 1 Processor -- start 2023-09-23T04:47:13.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 -- start start 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a61496e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614971130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6148cc460 con 0x55a614963e90 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6148bdc20 con 0x55a61496ed40 2023-09-23T04:47:13.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.161+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6148bb540 con 0x55a614963570 2023-09-23T04:47:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52486/0 (socket says 172.21.15.5:52486) 2023-09-23T04:47:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 -- 172.21.15.5:0/1775134585 learned_addr learned my addr 172.21.15.5:0/1775134585 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 -- 172.21.15.5:0/1775134585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 msgr2=0x55a614971130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T04:47:13.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614971130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 -- 172.21.15.5:0/1775134585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 msgr2=0x55a61496e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a61496e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.162+0000 7f97a4f9c700 1 -- 172.21.15.5:0/1775134585 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6149198d0 con 0x55a614963570 2023-09-23T04:47:13.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.163+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9798011600 tx=0x7f9798009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16f850a9350e66b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.163+0000 7f979d7fa700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 0x7f97880a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.164+0000 7f979d7fa700 1 -- 172.21.15.5:0/3830185819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f97940ced70 con 0x7f97a0068a10 2023-09-23T04:47:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.164+0000 7f97867fc700 1 -- 172.21.15.5:0/1775134585 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800e040 con 0x55a614963570 2023-09-23T04:47:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.164+0000 7f97867fc700 1 -- 172.21.15.5:0/1775134585 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f979800f040 con 0x55a614963570 2023-09-23T04:47:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.164+0000 7f97867fc700 1 -- 172.21.15.5:0/1775134585 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798004870 con 0x55a614963570 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.164+0000 7f979ffff700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 0x7f97880a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 -- 172.21.15.5:0/1775134585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 msgr2=0x55a614963950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9798011600 tx=0x7f9798009920 comp rx=0 tx=0).stop 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f979ffff700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 0x7f97880a3df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9790014fd0 tx=0x7f9790014d80 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 -- 172.21.15.5:0/1775134585 shutdown_connections 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614971130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a614963950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/1775134585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a61496e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 -- 172.21.15.5:0/1775134585 >> 172.21.15.5:0/1775134585 conn(0x55a6148d8260 msgr2=0x55a61495f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 -- 172.21.15.5:0/1775134585 shutdown_connections 2023-09-23T04:47:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.165+0000 7f97ba23bb80 1 -- 172.21.15.5:0/1775134585 wait complete. 2023-09-23T04:47:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.166+0000 7f97ba23bb80 1 -- start start 2023-09-23T04:47:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.166+0000 7f97ba23bb80 1 Processor -- start 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963e90 0x55a61496bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61496ed40 0x55a61496c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6148b6e30 con 0x55a61496ed40 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a614853710 con 0x55a614963570 2023-09-23T04:47:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6148533b0 con 0x55a614963e90 2023-09-23T04:47:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.167+0000 7f97ba23bb80 10 client.0 _ll_register_callbacks cb 0x55a6146b0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52834/0 (socket says 172.21.15.5:52834) 2023-09-23T04:47:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 -- 172.21.15.5:0/3907803942 learned_addr learned my addr 172.21.15.5:0/3907803942 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61496ed40 0x55a61496c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963e90 0x55a61496bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 -- 172.21.15.5:0/3907803942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963e90 msgr2=0x55a61496bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963e90 0x55a61496bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 -- 172.21.15.5:0/3907803942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61496ed40 msgr2=0x55a61496c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61496ed40 0x55a61496c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.168+0000 7f979ffff700 1 -- 172.21.15.5:0/3907803942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97980112b0 con 0x55a614963570 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f979ffff700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9790022cf0 tx=0x7f97900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22ee6de21b5add8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f977f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:13.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790027070 con 0x55a614963570 2023-09-23T04:47:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6149574a0 con 0x55a614963570 2023-09-23T04:47:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9790022dd0 con 0x55a614963570 2023-09-23T04:47:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9790020ad0 con 0x55a614963570 2023-09-23T04:47:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f97ba23bb80 10 client.25871 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.169+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a6146e1de0 con 0x55a614963570 2023-09-23T04:47:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6149574a0 con 0x55a614963570 2023-09-23T04:47:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f979002b320 con 0x55a614963570 2023-09-23T04:47:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 10 client.25871 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 10 client.25871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:47:13.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 20 client.25871 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a61497e960 0x55a614980d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 4 client.25871 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.170+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6148afe10 con 0x55a61497e960 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.171+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a61497e960 0x55a614980d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.171+0000 7f979f7fe700 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a61497e960 0x55a614980d80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=234f60c75378760a server_cookie=e1a426f7396761fc in_seq=0 out_seq=0 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.171+0000 7f977f7fe700 10 client.25871 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:13.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.172+0000 7f977f7fe700 1 -- 172.21.15.5:0/3907803942 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a6148afe10 con 0x55a61497e960 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.172+0000 7f977f7fe700 10 client.25871 handle_command_reply: tid=0 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.172+0000 7f97ba23bb80 1 client.25871 shutdown 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 msgr2=0x55a61496bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9790022cf0 tx=0x7f97900047c0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 shutdown_connections 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a61497e960 0x55a614980d80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61496ed40 0x55a61496c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a61496bab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/3907803942 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963e90 0x55a61496bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 >> 172.21.15.5:0/3907803942 conn(0x55a6148d8260 msgr2=0x55a61495f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f977f7fe700 0 client.25871 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 shutdown_connections 2023-09-23T04:47:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 1 -- 172.21.15.5:0/3907803942 wait complete. 2023-09-23T04:47:13.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.173+0000 7f97ba23bb80 20 client.25871 trim_cache size 0 max 16384 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.199+0000 7f97ba23bb80 1 Processor -- start 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 -- start start 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614968da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a614737920 con 0x55a614963e90 2023-09-23T04:47:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a614956fe0 con 0x55a61496ed40 2023-09-23T04:47:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.200+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a61496eb40 con 0x55a614963570 2023-09-23T04:47:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52504/0 (socket says 172.21.15.5:52504) 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 -- 172.21.15.5:0/2343884843 learned_addr learned my addr 172.21.15.5:0/2343884843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614968da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 -- 172.21.15.5:0/2343884843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 msgr2=0x55a614968da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614968da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 -- 172.21.15.5:0/2343884843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 msgr2=0x55a6149758f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 -- 172.21.15.5:0/2343884843 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a61495e3f0 con 0x55a614963570 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.201+0000 7f979ffff700 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9790000c00 tx=0x7f97900224e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a36b8955468746 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.202+0000 7f9786ffd700 1 -- 172.21.15.5:0/2343884843 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790027070 con 0x55a614963570 2023-09-23T04:47:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.202+0000 7f9786ffd700 1 -- 172.21.15.5:0/2343884843 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f979002b320 con 0x55a614963570 2023-09-23T04:47:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.202+0000 7f9786ffd700 1 -- 172.21.15.5:0/2343884843 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979003d440 con 0x55a614963570 2023-09-23T04:47:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 -- 172.21.15.5:0/2343884843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 msgr2=0x55a6149664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9790000c00 tx=0x7f97900224e0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 -- 172.21.15.5:0/2343884843 shutdown_connections 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a61496ed40 0x55a614968da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a614963570 0x55a6149664e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/2343884843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 -- 172.21.15.5:0/2343884843 >> 172.21.15.5:0/2343884843 conn(0x55a6148d8260 msgr2=0x55a61496c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 -- 172.21.15.5:0/2343884843 shutdown_connections 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.203+0000 7f97ba23bb80 1 -- 172.21.15.5:0/2343884843 wait complete. 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.204+0000 7f97ba23bb80 1 -- start start 2023-09-23T04:47:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.204+0000 7f97ba23bb80 1 Processor -- start 2023-09-23T04:47:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a61496ed40 0x55a6149ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6149cacd0 con 0x55a614963e90 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6149cae40 con 0x55a614963570 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6149cb000 con 0x55a61496ed40 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f97ba23bb80 10 client.0 _ll_register_callbacks cb 0x7f97900228d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f979ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f979ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52860/0 (socket says 172.21.15.5:52860) 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f979ffff700 1 -- 172.21.15.5:0/340643864 learned_addr learned my addr 172.21.15.5:0/340643864 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.205+0000 7f979f7fe700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f97a4f9c700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a61496ed40 0x55a6149ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 -- 172.21.15.5:0/340643864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a61496ed40 msgr2=0x55a6149ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a61496ed40 0x55a6149ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 -- 172.21.15.5:0/340643864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 msgr2=0x55a6149cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 -- 172.21.15.5:0/340643864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6146e1de0 con 0x55a614963570 2023-09-23T04:47:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f979ffff700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f979002b750 tx=0x7f9790020c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16afa71e714b433d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.206+0000 7f977f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790027070 con 0x55a614963570 2023-09-23T04:47:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9790020e40 con 0x55a614963570 2023-09-23T04:47:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a614965150 con 0x55a614963570 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f979002b490 con 0x55a614963570 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f97ba23bb80 10 client.25877 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a6146cac60 con 0x55a614963570 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.207+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f979004f080 con 0x55a614963570 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9790044070 con 0x55a614963570 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 10 client.25877 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 10 client.25877 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 20 client.25877 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a6149d36a0 0x55a6149d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 4 client.25877 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.208+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f979800f040 con 0x55a6149d36a0 2023-09-23T04:47:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.209+0000 7f979f7fe700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a6149d36a0 0x55a6149d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.209+0000 7f979f7fe700 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a6149d36a0 0x55a6149d5a90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ad92d79e50efda5 server_cookie=475518a7aa3826af in_seq=0 out_seq=0 2023-09-23T04:47:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.209+0000 7f977f7fe700 10 client.25877 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f977f7fe700 1 -- 172.21.15.5:0/340643864 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f979800f040 con 0x55a6149d36a0 2023-09-23T04:47:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f977f7fe700 10 client.25877 handle_command_reply: tid=0 2023-09-23T04:47:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 client.25877 shutdown 2023-09-23T04:47:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 msgr2=0x55a6149cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f979002b750 tx=0x7f9790020c50 comp rx=0 tx=0).stop 2023-09-23T04:47:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 shutdown_connections 2023-09-23T04:47:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55a6149d36a0 0x55a6149d5a90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a61496ed40 0x55a6149ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a614963570 0x55a6149cba00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 --2- 172.21.15.5:0/340643864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614963e90 0x55a6149cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 >> 172.21.15.5:0/340643864 conn(0x55a6148d8260 msgr2=0x55a61496d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f977f7fe700 0 client.25877 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 shutdown_connections 2023-09-23T04:47:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.210+0000 7f97ba23bb80 1 -- 172.21.15.5:0/340643864 wait complete. 2023-09-23T04:47:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.211+0000 7f97ba23bb80 20 client.25877 trim_cache size 0 max 16384 2023-09-23T04:47:13.214 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.211+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 msgr2=0x7f97880a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.211+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 0x7f97880a3df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9790014fd0 tx=0x7f9790014d80 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.211+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 msgr2=0x7f97a01993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.211+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9794005910 tx=0x7f9794005720 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 shutdown_connections 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f97880a1930 0x7f97880a3df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97a010d880 0x7f97a019d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f97a00680f0 0x7f97a0198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 --2- 172.21.15.5:0/3830185819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0068a10 0x7f97a01993f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 >> 172.21.15.5:0/3830185819 conn(0x7f97a0076810 msgr2=0x7f97a010ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 shutdown_connections 2023-09-23T04:47:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.212+0000 7f97a579d700 1 -- 172.21.15.5:0/3830185819 wait complete. 2023-09-23T04:47:13.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.455+0000 7fa23a5ff700 1 -- 172.21.15.5:0/2293153594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2341029e0 msgr2=0x7fa23410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.455+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/2293153594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2341029e0 0x7fa23410aec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa228009990 tx=0x7fa228009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 -- 172.21.15.5:0/2293153594 shutdown_connections 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/2293153594 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa23410b490 0x7fa23410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/2293153594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/2293153594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2341029e0 0x7fa23410aec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 -- 172.21.15.5:0/2293153594 >> 172.21.15.5:0/2293153594 conn(0x7fa234076f30 msgr2=0x7fa234077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 -- 172.21.15.5:0/2293153594 shutdown_connections 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.456+0000 7fa23a5ff700 1 -- 172.21.15.5:0/2293153594 wait complete. 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.457+0000 7fa23a5ff700 1 Processor -- start 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 -- start start 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 0x7fa23419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419c080 con 0x7fa2341020c0 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419c1f0 con 0x7fa23410b490 2023-09-23T04:47:13.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.458+0000 7fa23a5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419c3b0 con 0x7fa2341029e0 2023-09-23T04:47:13.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52524/0 (socket says 172.21.15.5:52524) 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa2337fe700 1 -- 172.21.15.5:0/4134786219 learned_addr learned my addr 172.21.15.5:0/4134786219 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa238b9c700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 0x7fa23419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 -- 172.21.15.5:0/4134786219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 msgr2=0x7fa2341975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 -- 172.21.15.5:0/4134786219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 msgr2=0x7fa23419ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 0x7fa23419ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa233fff700 1 -- 172.21.15.5:0/4134786219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280095f0 con 0x7fa2341020c0 2023-09-23T04:47:13.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.459+0000 7fa238b9c700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 0x7fa23419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa233fff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341970a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9f0 tx=0x7fa22400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa2317fa700 1 -- 172.21.15.5:0/4134786219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400c9c0 con 0x7fa2341020c0 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa2317fa700 1 -- 172.21.15.5:0/4134786219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa224010430 con 0x7fa2341020c0 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23419c690 con 0x7fa2341020c0 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa2317fa700 1 -- 172.21.15.5:0/4134786219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224018680 con 0x7fa2341020c0 2023-09-23T04:47:13.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.460+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340ffe20 con 0x7fa2341020c0 2023-09-23T04:47:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.465+0000 7fa24dedeb80 1 Processor -- start 2023-09-23T04:47:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.465+0000 7fa24dedeb80 1 -- start start 2023-09-23T04:47:13.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fad20 0x555701505690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555701505bd0 0x555701507fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701463460 con 0x5557014fad20 2023-09-23T04:47:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701454c20 con 0x5557014fa400 2023-09-23T04:47:13.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701452540 con 0x555701505bd0 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa238b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa238b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52894/0 (socket says 172.21.15.5:52894) 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa238b9c700 1 -- 172.21.15.5:0/337126034 learned_addr learned my addr 172.21.15.5:0/337126034 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa2317fa700 1 -- 172.21.15.5:0/4134786219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7fa224010860 con 0x7fa2341020c0 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.466+0000 7fa2337fe700 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555701505bd0 0x555701507fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.467+0000 7fa238b9c700 1 -- 172.21.15.5:0/337126034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555701505bd0 msgr2=0x555701507fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.467+0000 7fa238b9c700 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555701505bd0 0x555701507fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.467+0000 7fa238b9c700 1 -- 172.21.15.5:0/337126034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fad20 msgr2=0x555701505690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:47:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.468+0000 7fa2317fa700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 0x7fa21c0a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.468+0000 7fa233fff700 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fad20 0x555701505690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.468+0000 7fa238b9c700 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fad20 0x555701505690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.469+0000 7fa238b9c700 1 -- 172.21.15.5:0/337126034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557014b08d0 con 0x5557014fa400 2023-09-23T04:47:13.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.469+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 0x7fa21c0a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.469+0000 7fa2317fa700 1 -- 172.21.15.5:0/4134786219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa224014070 con 0x7fa2341020c0 2023-09-23T04:47:13.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.470+0000 7fa238b9c700 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00a870 tx=0x7fa22c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5e036ba4eeb5f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.470+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 0x7fa21c0a3e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2341986b0 tx=0x7fa228005ab0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.470+0000 7fa21a7fc700 1 -- 172.21.15.5:0/337126034 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c010040 con 0x5557014fa400 2023-09-23T04:47:13.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.470+0000 7fa21a7fc700 1 -- 172.21.15.5:0/337126034 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22c014b80 con 0x5557014fa400 2023-09-23T04:47:13.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.470+0000 7fa21a7fc700 1 -- 172.21.15.5:0/337126034 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c013b20 con 0x5557014fa400 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 -- 172.21.15.5:0/337126034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 msgr2=0x5557014fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00a870 tx=0x7fa22c009650 comp rx=0 tx=0).stop 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 -- 172.21.15.5:0/337126034 shutdown_connections 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555701505bd0 0x555701507fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x5557014fa7e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/337126034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fad20 0x555701505690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 -- 172.21.15.5:0/337126034 >> 172.21.15.5:0/337126034 conn(0x55570146f260 msgr2=0x5557014f6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 -- 172.21.15.5:0/337126034 shutdown_connections 2023-09-23T04:47:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.471+0000 7fa24dedeb80 1 -- 172.21.15.5:0/337126034 wait complete. 2023-09-23T04:47:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 -- start start 2023-09-23T04:47:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 Processor -- start 2023-09-23T04:47:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701502ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701503410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.472+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55570144de30 con 0x555701505bd0 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557013ea710 con 0x5557014fa400 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557013ea3b0 con 0x5557014fad20 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa24dedeb80 10 client.0 _ll_register_callbacks cb 0x555701247800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52908/0 (socket says 172.21.15.5:52908) 2023-09-23T04:47:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 -- 172.21.15.5:0/3785749780 learned_addr learned my addr 172.21.15.5:0/3785749780 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701502ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 -- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 msgr2=0x555701502ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701502ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 -- 172.21.15.5:0/3785749780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 msgr2=0x555701503410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701503410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.473+0000 7fa233fff700 1 -- 172.21.15.5:0/3785749780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c00a4a0 con 0x5557014fa400 2023-09-23T04:47:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.474+0000 7fa233fff700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9c0 tx=0x7fa22409a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4357b9b55b6498fe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.474+0000 7fa2137fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.474+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22409a320 con 0x5557014fa400 2023-09-23T04:47:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.474+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa22409e470 con 0x5557014fa400 2023-09-23T04:47:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.474+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557014ee250 con 0x5557014fa400 2023-09-23T04:47:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240a26b0 con 0x5557014fa400 2023-09-23T04:47:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 10 client.25889 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555701278de0 con 0x5557014fa400 2023-09-23T04:47:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557014ee250 con 0x5557014fa400 2023-09-23T04:47:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa22400cb20 con 0x5557014fa400 2023-09-23T04:47:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 10 client.25889 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 10 client.25889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:47:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 20 client.25889 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55570151df20 0x555701520310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 4 client.25889 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.475+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555701446e10 con 0x55570151df20 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.476+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55570151df20 0x555701520310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.476+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55570151df20 0x555701520310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2113c5ecadffe117 server_cookie=c6d963dbb8f38c68 in_seq=0 out_seq=0 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.476+0000 7fa2137fe700 10 client.25889 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.477+0000 7fa2137fe700 1 -- 172.21.15.5:0/3785749780 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555701446e10 con 0x55570151df20 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.477+0000 7fa2137fe700 10 client.25889 handle_command_reply: tid=0 2023-09-23T04:47:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.477+0000 7fa24dedeb80 1 client.25889 shutdown 2023-09-23T04:47:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 msgr2=0x555701502990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9c0 tx=0x7fa22409a860 comp rx=0 tx=0).stop 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 shutdown_connections 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701503410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55570151df20 0x555701520310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fa400 0x555701502990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3785749780 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701502ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 >> 172.21.15.5:0/3785749780 conn(0x55570146f260 msgr2=0x5557014f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa2137fe700 0 client.25889 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 shutdown_connections 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3785749780 wait complete. 2023-09-23T04:47:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.478+0000 7fa24dedeb80 20 client.25889 trim_cache size 0 max 16384 2023-09-23T04:47:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 Processor -- start 2023-09-23T04:47:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 -- start start 2023-09-23T04:47:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fa400 0x5557014fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555701505bd0 0x5557014fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557012ce920 con 0x5557014fa400 2023-09-23T04:47:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701515510 con 0x555701505bd0 2023-09-23T04:47:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.492+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701500f60 con 0x5557014fad20 2023-09-23T04:47:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa238b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555701505bd0 0x5557014fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52550/0 (socket says 172.21.15.5:52550) 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 -- 172.21.15.5:0/4294667071 learned_addr learned my addr 172.21.15.5:0/4294667071 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa233fff700 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fa400 0x5557014fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 -- 172.21.15.5:0/4294667071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555701505bd0 msgr2=0x5557014fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555701505bd0 0x5557014fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 -- 172.21.15.5:0/4294667071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fa400 msgr2=0x5557014fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fa400 0x5557014fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.493+0000 7fa2337fe700 1 -- 172.21.15.5:0/4294667071 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555701261c60 con 0x5557014fad20 2023-09-23T04:47:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.494+0000 7fa2337fe700 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa228005b40 tx=0x7fa228022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=707bb41250e94079 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.494+0000 7fa21affd700 1 -- 172.21.15.5:0/4294667071 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22801d040 con 0x5557014fad20 2023-09-23T04:47:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.494+0000 7fa21affd700 1 -- 172.21.15.5:0/4294667071 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22801e040 con 0x5557014fad20 2023-09-23T04:47:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.494+0000 7fa21affd700 1 -- 172.21.15.5:0/4294667071 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800f800 con 0x5557014fad20 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 -- 172.21.15.5:0/4294667071 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 msgr2=0x555701514f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa228005b40 tx=0x7fa228022000 comp rx=0 tx=0).stop 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 -- 172.21.15.5:0/4294667071 shutdown_connections 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555701505bd0 0x5557014fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557014fa400 0x5557014fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/4294667071 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fad20 0x555701514f70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 -- 172.21.15.5:0/4294667071 >> 172.21.15.5:0/4294667071 conn(0x55570146f260 msgr2=0x555701503190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 -- 172.21.15.5:0/4294667071 shutdown_connections 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.495+0000 7fa24dedeb80 1 -- 172.21.15.5:0/4294667071 wait complete. 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 -- start start 2023-09-23T04:47:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 Processor -- start 2023-09-23T04:47:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 0x5557015661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701570d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701566910 con 0x555701505bd0 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701566a80 con 0x5557014fad20 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.496+0000 7fa24dedeb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555701571240 con 0x5557014fa400 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa24dedeb80 10 client.0 _ll_register_callbacks cb 0x7fa22409a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52552/0 (socket says 172.21.15.5:52552) 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 -- 172.21.15.5:0/3894825572 learned_addr learned my addr 172.21.15.5:0/3894825572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 0x5557015661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa238b9c700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701570d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 -- 172.21.15.5:0/3894825572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 msgr2=0x5557015661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 0x5557015661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 -- 172.21.15.5:0/3894825572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 msgr2=0x555701570d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701570d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.497+0000 7fa233fff700 1 -- 172.21.15.5:0/3894825572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23410e5d0 con 0x5557014fa400 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 0x5557015661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa238b9c700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701570d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa233fff700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa224010c10 tx=0x7fa224009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefc23861b139e02 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa2137fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240cb0b0 con 0x5557014fa400 2023-09-23T04:47:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa22c014b80 con 0x5557014fa400 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa22409e470 con 0x5557014fa400 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240cb210 con 0x5557014fa400 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa24dedeb80 10 client.25942 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.498+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55570129f250 con 0x5557014fa400 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22c014b80 con 0x5557014fa400 2023-09-23T04:47:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa22409f1b0 con 0x5557014fa400 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 10 client.25942 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 10 client.25942 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 20 client.25942 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x555701572620 0x555701574a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 4 client.25942 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.499+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555701446b30 con 0x555701572620 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x555701572620 0x555701574a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa2337fe700 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x555701572620 0x555701574a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65daf19a7aed96e2 server_cookie=491c6185a81cd2ae in_seq=0 out_seq=0 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa2137fe700 10 client.25942 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa2137fe700 1 -- 172.21.15.5:0/3894825572 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555701446b30 con 0x555701572620 2023-09-23T04:47:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa2137fe700 10 client.25942 handle_command_reply: tid=0 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.500+0000 7fa24dedeb80 1 client.25942 shutdown 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 msgr2=0x555701565c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa224010c10 tx=0x7fa224009790 comp rx=0 tx=0).stop 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 shutdown_connections 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555701505bd0 0x555701570d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5557014fa400 0x555701565c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x555701572620 0x555701574a10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 --2- 172.21.15.5:0/3894825572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557014fad20 0x5557015661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 >> 172.21.15.5:0/3894825572 conn(0x55570146f260 msgr2=0x5557014fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa2137fe700 0 client.25942 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 shutdown_connections 2023-09-23T04:47:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 1 -- 172.21.15.5:0/3894825572 wait complete. 2023-09-23T04:47:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.501+0000 7fa24dedeb80 20 client.25942 trim_cache size 0 max 16384 2023-09-23T04:47:13.504 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 msgr2=0x7fa21c0a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 0x7fa21c0a3e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa2341986b0 tx=0x7fa228005ab0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 msgr2=0x7fa2341970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341970a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e9f0 tx=0x7fa22400ed00 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 shutdown_connections 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa21c0a1980 0x7fa21c0a3e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa23410b490 0x7fa23419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2341020c0 0x7fa2341970a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 --2- 172.21.15.5:0/4134786219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2341029e0 0x7fa2341975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 >> 172.21.15.5:0/4134786219 conn(0x7fa234076f30 msgr2=0x7fa2340fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.502+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 shutdown_connections 2023-09-23T04:47:13.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.503+0000 7fa23a5ff700 1 -- 172.21.15.5:0/4134786219 wait complete. 2023-09-23T04:47:13.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:47:13.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.860+0000 7f4dedfff700 1 -- 172.21.15.5:0/1827264149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 msgr2=0x7f4de81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.860+0000 7f4dedfff700 1 --2- 172.21.15.5:0/1827264149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de81014d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009990 tx=0x7f4dd8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.860+0000 7f4dedfff700 1 -- 172.21.15.5:0/1827264149 shutdown_connections 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 --2- 172.21.15.5:0/1827264149 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8106210 0x7f4de810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 --2- 172.21.15.5:0/1827264149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de81014d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 --2- 172.21.15.5:0/1827264149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8101aa0 0x7f4de8105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 -- 172.21.15.5:0/1827264149 >> 172.21.15.5:0/1827264149 conn(0x7f4de80fc9d0 msgr2=0x7f4de80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 -- 172.21.15.5:0/1827264149 shutdown_connections 2023-09-23T04:47:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.861+0000 7f4dedfff700 1 -- 172.21.15.5:0/1827264149 wait complete. 2023-09-23T04:47:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.863+0000 7f4dedfff700 1 Processor -- start 2023-09-23T04:47:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.863+0000 7f4dedfff700 1 -- start start 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 0x7f4de8073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 0x7f4de81a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8075070 con 0x7f4de81010f0 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de819b170 con 0x7f4de8106210 2023-09-23T04:47:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4dedfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de819b330 con 0x7f4de8073ba0 2023-09-23T04:47:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 0x7f4de8073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4de6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 0x7f4de8073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52948/0 (socket says 172.21.15.5:52948) 2023-09-23T04:47:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4de6ffd700 1 -- 172.21.15.5:0/3236142483 learned_addr learned my addr 172.21.15.5:0/3236142483 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.864+0000 7f4de77fe700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 0x7f4de81a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 -- 172.21.15.5:0/3236142483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 msgr2=0x7f4de8073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 0x7f4de8073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 -- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 msgr2=0x7f4de8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de8073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de7fff700 1 -- 172.21.15.5:0/3236142483 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80095f0 con 0x7f4de8073ba0 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.865+0000 7f4de77fe700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 0x7f4de81a99b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4de000efc0 tx=0x7f4de000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4de4ff9700 1 -- 172.21.15.5:0/3236142483 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0018070 con 0x7f4de8073ba0 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de819b5b0 con 0x7f4de8073ba0 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4de4ff9700 1 -- 172.21.15.5:0/3236142483 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de000f040 con 0x7f4de8073ba0 2023-09-23T04:47:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4de4ff9700 1 -- 172.21.15.5:0/3236142483 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000cd40 con 0x7f4de8073ba0 2023-09-23T04:47:13.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.866+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de819bac0 con 0x7f4de8073ba0 2023-09-23T04:47:13.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.871+0000 7f4de4ff9700 1 -- 172.21.15.5:0/3236142483 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f4de0003a00 con 0x7f4de8073ba0 2023-09-23T04:47:13.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.874+0000 7f4e019afb80 1 Processor -- start 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.874+0000 7f4de4ff9700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 0x7f4dd00a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.874+0000 7f4e019afb80 1 -- start start 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4de4ff9700 1 -- 172.21.15.5:0/3236142483 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f4de00cd4a0 con 0x7f4de8073ba0 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c237690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c239fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c195460 con 0x55881c22c400 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c186c20 con 0x55881c22cd20 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4e019afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c184540 con 0x55881c237bd0 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.875+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52346/0 (socket says 172.21.15.5:52346) 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 -- 172.21.15.5:0/96327579 learned_addr learned my addr 172.21.15.5:0/96327579 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c237690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 -- 172.21.15.5:0/96327579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 msgr2=0x55881c239fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T04:47:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c239fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 -- 172.21.15.5:0/96327579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 msgr2=0x55881c237690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c237690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.876+0000 7f4de77fe700 1 -- 172.21.15.5:0/96327579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881c1e28d0 con 0x55881c22c400 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.877+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 0x7f4dd00a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.877+0000 7f4de77fe700 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005bc0 tx=0x7f4dd8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d402d004b96b07d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.877+0000 7f4dcdffb700 1 -- 172.21.15.5:0/96327579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd801d070 con 0x55881c22c400 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.877+0000 7f4dcdffb700 1 -- 172.21.15.5:0/96327579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd801e040 con 0x55881c22c400 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.877+0000 7f4dcdffb700 1 -- 172.21.15.5:0/96327579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800f460 con 0x55881c22c400 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.878+0000 7f4e019afb80 1 -- 172.21.15.5:0/96327579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 msgr2=0x55881c22c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.878+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 0x7f4dd00a3d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4de8074910 tx=0x7f4ddc00a3b0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.878+0000 7f4e019afb80 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005bc0 tx=0x7f4dd8005bf0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 -- 172.21.15.5:0/96327579 shutdown_connections 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c239fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22c7e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 --2- 172.21.15.5:0/96327579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c237690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 -- 172.21.15.5:0/96327579 >> 172.21.15.5:0/96327579 conn(0x55881c1a1260 msgr2=0x55881c228790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 -- 172.21.15.5:0/96327579 shutdown_connections 2023-09-23T04:47:13.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.879+0000 7f4e019afb80 1 -- 172.21.15.5:0/96327579 wait complete. 2023-09-23T04:47:13.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.880+0000 7f4e019afb80 1 -- start start 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.880+0000 7f4e019afb80 1 Processor -- start 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c234920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c234e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c17fe30 con 0x55881c22c400 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c11c710 con 0x55881c22cd20 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c11c3b0 con 0x55881c237bd0 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.881+0000 7f4e019afb80 10 client.0 _ll_register_callbacks cb 0x55881bf79800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52590/0 (socket says 172.21.15.5:52590) 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 -- 172.21.15.5:0/3936188457 learned_addr learned my addr 172.21.15.5:0/3936188457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c234e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de77fe700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c234920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 -- 172.21.15.5:0/3936188457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 msgr2=0x55881c234e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c234e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 -- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 msgr2=0x55881c234920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c234920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.882+0000 7f4de7fff700 1 -- 172.21.15.5:0/3936188457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de810e600 con 0x55881c237bd0 2023-09-23T04:47:13.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4de7fff700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4de0099da0 tx=0x7f4de00999d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb700a1923c9821 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4dc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0099500 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55881c220250 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f4de0004ad0 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4de001b320 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4e019afb80 10 client.25951 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.883+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55881bfaade0 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.884+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de00b8020 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4de00a1c80 con 0x55881c237bd0 2023-09-23T04:47:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 10 client.25951 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 20 client.25951 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c247730 0x55881c249b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 4 client.25951 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55881c178e10 con 0x55881c247730 2023-09-23T04:47:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.885+0000 7f4de77fe700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c247730 0x55881c249b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.886+0000 7f4de77fe700 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c247730 0x55881c249b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e964d4e21019329 server_cookie=88d573e0c801b84b in_seq=0 out_seq=0 2023-09-23T04:47:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.886+0000 7f4dc6ffd700 10 client.25951 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.887+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3936188457 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55881c178e10 con 0x55881c247730 2023-09-23T04:47:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.887+0000 7f4dc6ffd700 10 client.25951 handle_command_reply: tid=0 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.887+0000 7f4e019afb80 1 client.25951 shutdown 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 msgr2=0x55881c2353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4de0099da0 tx=0x7f4de00999d0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 shutdown_connections 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c247730 0x55881c249b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c2353a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c234920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4dc6ffd700 0 client.25951 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 --2- 172.21.15.5:0/3936188457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c234e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 >> 172.21.15.5:0/3936188457 conn(0x55881c1a1260 msgr2=0x55881c228380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 shutdown_connections 2023-09-23T04:47:13.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 1 -- 172.21.15.5:0/3936188457 wait complete. 2023-09-23T04:47:13.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.888+0000 7f4e019afb80 20 client.25951 trim_cache size 0 max 16384 2023-09-23T04:47:13.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:13 smithi005 ceph-mon[97638]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-09-23T04:47:13.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:13 smithi118 ceph-mon[134297]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-09-23T04:47:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 Processor -- start 2023-09-23T04:47:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 -- start start 2023-09-23T04:47:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c231bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c000920 con 0x55881c22c400 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c229bc0 con 0x55881c22cd20 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.915+0000 7f4e019afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c22f7e0 con 0x55881c237bd0 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.916+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.916+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52368/0 (socket says 172.21.15.5:52368) 2023-09-23T04:47:13.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.916+0000 7f4de77fe700 1 -- 172.21.15.5:0/2158306388 learned_addr learned my addr 172.21.15.5:0/2158306388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.916+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.916+0000 7f4de7fff700 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c231bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 -- 172.21.15.5:0/2158306388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 msgr2=0x55881c231bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c231bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 -- 172.21.15.5:0/2158306388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 msgr2=0x55881c23e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 -- 172.21.15.5:0/2158306388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881bf93c60 con 0x55881c22c400 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009990 tx=0x7f4dd801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49e736deae4928 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4dce7fc700 1 -- 172.21.15.5:0/2158306388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd801d040 con 0x55881c22c400 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4dce7fc700 1 -- 172.21.15.5:0/2158306388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd8034d40 con 0x55881c22c400 2023-09-23T04:47:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.917+0000 7f4dce7fc700 1 -- 172.21.15.5:0/2158306388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800faa0 con 0x55881c22c400 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.918+0000 7f4e019afb80 1 -- 172.21.15.5:0/2158306388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 msgr2=0x55881c22f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.918+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009990 tx=0x7f4dd801a040 comp rx=0 tx=0).stop 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 -- 172.21.15.5:0/2158306388 shutdown_connections 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c231bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c22f2a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2158306388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 -- 172.21.15.5:0/2158306388 >> 172.21.15.5:0/2158306388 conn(0x55881c1a1260 msgr2=0x55881c22fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 -- 172.21.15.5:0/2158306388 shutdown_connections 2023-09-23T04:47:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 -- 172.21.15.5:0/2158306388 wait complete. 2023-09-23T04:47:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.919+0000 7f4e019afb80 1 -- start start 2023-09-23T04:47:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.920+0000 7f4e019afb80 1 Processor -- start 2023-09-23T04:47:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.920+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.920+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4e019afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c29a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4e019afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c23d550 con 0x55881c22c400 2023-09-23T04:47:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4e019afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c29aad0 con 0x55881c22cd20 2023-09-23T04:47:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4e019afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881c29ac90 con 0x55881c237bd0 2023-09-23T04:47:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4e019afb80 10 client.0 _ll_register_callbacks cb 0x7f4de0096150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52374/0 (socket says 172.21.15.5:52374) 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4de77fe700 1 -- 172.21.15.5:0/2409184359 learned_addr learned my addr 172.21.15.5:0/2409184359 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.921+0000 7f4de7fff700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c29a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 -- 172.21.15.5:0/2409184359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 msgr2=0x55881c29a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c29a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 -- 172.21.15.5:0/2409184359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 msgr2=0x55881c23cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 -- 172.21.15.5:0/2409184359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55881c1e28d0 con 0x55881c22c400 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4de77fe700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bf30 tx=0x7f4dd8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489fb965ba0bd8ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4dc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.922+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd801d040 con 0x55881c22c400 2023-09-23T04:47:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.923+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f4dd8031040 con 0x55881c22c400 2023-09-23T04:47:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.923+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dd801e040 con 0x55881c22c400 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.923+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd8034a80 con 0x55881c22c400 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.923+0000 7f4e019afb80 10 client.15921 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.923+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55881bfd1250 con 0x55881c22c400 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd801e040 con 0x55881c22c400 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4dd803d7a0 con 0x55881c22c400 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 20 client.15921 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c29bf00 0x55881c29e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 4 client.15921 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.924+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55881c178b30 con 0x55881c29bf00 2023-09-23T04:47:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.925+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c29bf00 0x55881c29e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.925+0000 7f4de6ffd700 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c29bf00 0x55881c29e2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797ea883eeda0706 server_cookie=a62cdd1cfe207907 in_seq=0 out_seq=0 2023-09-23T04:47:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.925+0000 7f4dc6ffd700 10 client.15921 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/2409184359 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55881c178b30 con 0x55881c29bf00 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4dc6ffd700 10 client.15921 handle_command_reply: tid=0 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 client.15921 shutdown 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 msgr2=0x55881c23c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800bf30 tx=0x7f4dd8004710 comp rx=0 tx=0).stop 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 shutdown_connections 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55881c29bf00 0x55881c29e2f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55881c237bd0 0x55881c29a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55881c22c400 0x55881c23c890 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 --2- 172.21.15.5:0/2409184359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55881c22cd20 0x55881c23cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 >> 172.21.15.5:0/2409184359 conn(0x55881c1a1260 msgr2=0x55881c2348c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4dc6ffd700 0 client.15921 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.926+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 shutdown_connections 2023-09-23T04:47:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4e019afb80 1 -- 172.21.15.5:0/2409184359 wait complete. 2023-09-23T04:47:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4e019afb80 20 client.15921 trim_cache size 0 max 16384 2023-09-23T04:47:13.929 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 msgr2=0x7f4dd00a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 0x7f4dd00a3d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4de8074910 tx=0x7f4ddc00a3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 msgr2=0x7f4de81a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.927+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 0x7f4de81a99b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4de000efc0 tx=0x7f4de000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 shutdown_connections 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4dd00a18b0 0x7f4dd00a3d70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4de8073ba0 0x7f4de81a99b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de81010f0 0x7f4de8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 --2- 172.21.15.5:0/3236142483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4de8106210 0x7f4de8073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 >> 172.21.15.5:0/3236142483 conn(0x7f4de80fc9d0 msgr2=0x7f4de80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 shutdown_connections 2023-09-23T04:47:13.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:13.928+0000 7f4dedfff700 1 -- 172.21.15.5:0/3236142483 wait complete. 2023-09-23T04:47:13.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:13 smithi069 ceph-mon[133931]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2023-09-23T04:47:14.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:47:14.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:47:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.311+0000 7f59601ff700 1 -- 172.21.15.5:0/3141565062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073250 msgr2=0x7f5958073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.311+0000 7f59601ff700 1 --2- 172.21.15.5:0/3141565062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073250 0x7f5958073630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59540098f0 tx=0x7f5954009c00 comp rx=0 tx=0).stop 2023-09-23T04:47:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 -- 172.21.15.5:0/3141565062 shutdown_connections 2023-09-23T04:47:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 --2- 172.21.15.5:0/3141565062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595810fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 --2- 172.21.15.5:0/3141565062 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073250 0x7f5958073630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 --2- 172.21.15.5:0/3141565062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073b70 0x7f595810d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 -- 172.21.15.5:0/3141565062 >> 172.21.15.5:0/3141565062 conn(0x7f59580fc9d0 msgr2=0x7f59580fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 -- 172.21.15.5:0/3141565062 shutdown_connections 2023-09-23T04:47:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.312+0000 7f59601ff700 1 -- 172.21.15.5:0/3141565062 wait complete. 2023-09-23T04:47:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.313+0000 7f59601ff700 1 Processor -- start 2023-09-23T04:47:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 -- start start 2023-09-23T04:47:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073250 0x7f59581992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e2a0 con 0x7f595810d6c0 2023-09-23T04:47:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e410 con 0x7f5958073250 2023-09-23T04:47:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.314+0000 7f59601ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e5d0 con 0x7f5958073b70 2023-09-23T04:47:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52614/0 (socket says 172.21.15.5:52614) 2023-09-23T04:47:15.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 -- 172.21.15.5:0/3344729389 learned_addr learned my addr 172.21.15.5:0/3344729389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595e79c700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595df9b700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073250 0x7f59581992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 -- 172.21.15.5:0/3344729389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073250 msgr2=0x7f59581992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073250 0x7f59581992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 -- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 msgr2=0x7f595819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595d79a700 1 -- 172.21.15.5:0/3344729389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954009520 con 0x7f5958073b70 2023-09-23T04:47:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.315+0000 7f595e79c700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f595d79a700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f594800e9c0 tx=0x7f594800ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f594effd700 1 -- 172.21.15.5:0/3344729389 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800cbd0 con 0x7f5958073b70 2023-09-23T04:47:15.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f594effd700 1 -- 172.21.15.5:0/3344729389 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594800cd30 con 0x7f5958073b70 2023-09-23T04:47:15.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595819e8b0 con 0x7f5958073b70 2023-09-23T04:47:15.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f594effd700 1 -- 172.21.15.5:0/3344729389 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948005580 con 0x7f5958073b70 2023-09-23T04:47:15.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.316+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a60f0 con 0x7f5958073b70 2023-09-23T04:47:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.322+0000 7f594effd700 1 -- 172.21.15.5:0/3344729389 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f5948004750 con 0x7f5958073b70 2023-09-23T04:47:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.323+0000 7f5973a66b80 1 Processor -- start 2023-09-23T04:47:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.323+0000 7f5973a66b80 1 -- start start 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.323+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b48c480 0x55654b48c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.323+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b497710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b49a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f5973a66b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b3f54e0 con 0x55654b48cda0 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f5973a66b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b3e6ca0 con 0x55654b497c50 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f5973a66b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b3e45c0 con 0x55654b48c480 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b497710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b497710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52398/0 (socket says 172.21.15.5:52398) 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f595df9b700 1 -- 172.21.15.5:0/3035186189 learned_addr learned my addr 172.21.15.5:0/3035186189 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f595d79a700 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b49a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.324+0000 7f595e79c700 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b48c480 0x55654b48c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 -- 172.21.15.5:0/3035186189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b48c480 msgr2=0x55654b48c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b48c480 0x55654b48c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 -- 172.21.15.5:0/3035186189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 msgr2=0x55654b497710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b497710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 -- 172.21.15.5:0/3035186189 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55654b442950 con 0x55654b497c50 2023-09-23T04:47:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f595d79a700 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b49a040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59480cada0 tx=0x7f59480d1a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ada74ccccf7b91 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.325+0000 7f5943fff700 1 -- 172.21.15.5:0/3035186189 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480d86d0 con 0x55654b497c50 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.326+0000 7f5943fff700 1 -- 172.21.15.5:0/3035186189 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59480d8cf0 con 0x55654b497c50 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.326+0000 7f5943fff700 1 -- 172.21.15.5:0/3035186189 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480daba0 con 0x55654b497c50 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.326+0000 7f594effd700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 0x7f59440a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.326+0000 7f594effd700 1 -- 172.21.15.5:0/3344729389 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f5948014070 con 0x7f5958073b70 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.326+0000 7f595e79c700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 0x7f59440a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 -- 172.21.15.5:0/3035186189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 msgr2=0x55654b49a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b49a040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59480cada0 tx=0x7f59480d1a10 comp rx=0 tx=0).stop 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 -- 172.21.15.5:0/3035186189 shutdown_connections 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b49a040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b48c480 0x55654b48c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3035186189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b497710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 -- 172.21.15.5:0/3035186189 >> 172.21.15.5:0/3035186189 conn(0x55654b4012e0 msgr2=0x55654b488810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 -- 172.21.15.5:0/3035186189 shutdown_connections 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f595e79c700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 0x7f59440a3d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59500094a0 tx=0x7f5950009040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.327+0000 7f5973a66b80 1 -- 172.21.15.5:0/3035186189 wait complete. 2023-09-23T04:47:15.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.328+0000 7f5973a66b80 1 -- start start 2023-09-23T04:47:15.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.328+0000 7f5973a66b80 1 Processor -- start 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b48cda0 0x55654b494f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b495460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b3dfeb0 con 0x55654b48c480 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b37c790 con 0x55654b48cda0 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b37c430 con 0x55654b497c50 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.329+0000 7f5973a66b80 10 client.0 _ll_register_callbacks cb 0x55654b1d9880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.330+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.330+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52406/0 (socket says 172.21.15.5:52406) 2023-09-23T04:47:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.330+0000 7f595df9b700 1 -- 172.21.15.5:0/2564087212 learned_addr learned my addr 172.21.15.5:0/2564087212 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.330+0000 7f595d79a700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b48cda0 0x55654b494f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.330+0000 7f595e79c700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b495460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 -- 172.21.15.5:0/2564087212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 msgr2=0x55654b495460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b495460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 -- 172.21.15.5:0/2564087212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b48cda0 msgr2=0x55654b494f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b48cda0 0x55654b494f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 -- 172.21.15.5:0/2564087212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a60f0 con 0x55654b48c480 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f595df9b700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5954009230 tx=0x7f595400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33da890b514d3850 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.331+0000 7f59417fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595401c050 con 0x55654b48c480 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f595400bbc0 con 0x55654b48c480 2023-09-23T04:47:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55654b4802d0 con 0x55654b48c480 2023-09-23T04:47:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f595400f940 con 0x55654b48c480 2023-09-23T04:47:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f5973a66b80 10 client.15933 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.332+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55654b20ae60 con 0x55654b48c480 2023-09-23T04:47:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55654b4802d0 con 0x55654b48c480 2023-09-23T04:47:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5954015020 con 0x55654b48c480 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 20 client.15933 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b4affe0 0x55654b4b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 4 client.15933 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.333+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55654b3d8e90 con 0x55654b4affe0 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.334+0000 7f595d79a700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b4affe0 0x55654b4b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.335+0000 7f595d79a700 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b4affe0 0x55654b4b2420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2108e281987a631e server_cookie=b0c0b757b5898849 in_seq=0 out_seq=0 2023-09-23T04:47:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.335+0000 7f59417fa700 10 client.15933 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f59417fa700 1 -- 172.21.15.5:0/2564087212 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55654b3d8e90 con 0x55654b4affe0 2023-09-23T04:47:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f59417fa700 10 client.15933 handle_command_reply: tid=0 2023-09-23T04:47:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 client.15933 shutdown 2023-09-23T04:47:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 msgr2=0x55654b4949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5954009230 tx=0x7f595400b790 comp rx=0 tx=0).stop 2023-09-23T04:47:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 shutdown_connections 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b4affe0 0x55654b4b2420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b495460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48c480 0x55654b4949e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f59417fa700 0 client.15933 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2564087212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b48cda0 0x55654b494f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.336+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 >> 172.21.15.5:0/2564087212 conn(0x55654b4012e0 msgr2=0x55654b4883a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.337+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 shutdown_connections 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.337+0000 7f5973a66b80 1 -- 172.21.15.5:0/2564087212 wait complete. 2023-09-23T04:47:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.337+0000 7f5973a66b80 20 client.15933 trim_cache size 0 max 16384 2023-09-23T04:47:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.371+0000 7f5973a66b80 1 Processor -- start 2023-09-23T04:47:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 -- start start 2023-09-23T04:47:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 0x55654b493690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b2609a0 con 0x55654b48cda0 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b484e10 con 0x55654b497300 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.372+0000 7f5973a66b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b497b10 con 0x55654b497c50 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52634/0 (socket says 172.21.15.5:52634) 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 -- 172.21.15.5:0/3586699730 learned_addr learned my addr 172.21.15.5:0/3586699730 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595e79c700 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 0x55654b493690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 -- 172.21.15.5:0/3586699730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 msgr2=0x55654b493690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 0x55654b493690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 -- 172.21.15.5:0/3586699730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 msgr2=0x55654b4a6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4a6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595d79a700 1 -- 172.21.15.5:0/3586699730 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55654b484500 con 0x55654b497c50 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.373+0000 7f595e79c700 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 0x55654b493690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.374+0000 7f595d79a700 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f594809a1f0 tx=0x7f5948009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f0e8a360af4f41 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.374+0000 7f594cff9700 1 -- 172.21.15.5:0/3586699730 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480af070 con 0x55654b497c50 2023-09-23T04:47:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.374+0000 7f594cff9700 1 -- 172.21.15.5:0/3586699730 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5948096830 con 0x55654b497c50 2023-09-23T04:47:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.374+0000 7f594cff9700 1 -- 172.21.15.5:0/3586699730 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594809a030 con 0x55654b497c50 2023-09-23T04:47:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 -- 172.21.15.5:0/3586699730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 msgr2=0x55654b4952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f594809a1f0 tx=0x7f5948009740 comp rx=0 tx=0).stop 2023-09-23T04:47:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 -- 172.21.15.5:0/3586699730 shutdown_connections 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497300 0x55654b493690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4a6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 --2- 172.21.15.5:0/3586699730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497c50 0x55654b4952c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 -- 172.21.15.5:0/3586699730 >> 172.21.15.5:0/3586699730 conn(0x55654b4012e0 msgr2=0x55654b492f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 -- 172.21.15.5:0/3586699730 shutdown_connections 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.375+0000 7f5973a66b80 1 -- 172.21.15.5:0/3586699730 wait complete. 2023-09-23T04:47:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.376+0000 7f5973a66b80 1 -- start start 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.376+0000 7f5973a66b80 1 Processor -- start 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 0x55654b4fee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b4a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b4a6930 con 0x55654b48cda0 2023-09-23T04:47:15.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b4a6aa0 con 0x55654b497c50 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55654b4a6c60 con 0x55654b497300 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f5973a66b80 10 client.0 _ll_register_callbacks cb 0x7f5954009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f595df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52428/0 (socket says 172.21.15.5:52428) 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f595df9b700 1 -- 172.21.15.5:0/2321574562 learned_addr learned my addr 172.21.15.5:0/2321574562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.377+0000 7f595d79a700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 0x55654b4fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595e79c700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b4a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 -- 172.21.15.5:0/2321574562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 msgr2=0x55654b4a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b4a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 -- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 msgr2=0x55654b4fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 -- 172.21.15.5:0/2321574562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595001a740 con 0x55654b497300 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595df9b700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f595d79a700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 0x55654b4fee30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55654b490a80 tx=0x7f59480cadc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd1476e420ead3a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.378+0000 7f59417fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:15.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480af070 con 0x55654b497300 2023-09-23T04:47:15.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f59480d8040 con 0x55654b497300 2023-09-23T04:47:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59480d8cf0 con 0x55654b497300 2023-09-23T04:47:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59480d1920 con 0x55654b497300 2023-09-23T04:47:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f5973a66b80 10 client.25975 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55654b442950 con 0x55654b497300 2023-09-23T04:47:15.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.379+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59480d8cf0 con 0x55654b497300 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.380+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5948096c00 con 0x55654b497300 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 20 client.25975 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b506300 0x55654b5086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 4 client.25975 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55654b3d8bb0 con 0x55654b506300 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.381+0000 7f595df9b700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b506300 0x55654b5086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.382+0000 7f595df9b700 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b506300 0x55654b5086f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a969b4120fc76e5 server_cookie=b683924495e41db4 in_seq=0 out_seq=0 2023-09-23T04:47:15.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.382+0000 7f59417fa700 10 client.25975 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.383+0000 7f59417fa700 1 -- 172.21.15.5:0/2321574562 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55654b3d8bb0 con 0x55654b506300 2023-09-23T04:47:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.383+0000 7f59417fa700 10 client.25975 handle_command_reply: tid=0 2023-09-23T04:47:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.384+0000 7f5973a66b80 1 client.25975 shutdown 2023-09-23T04:47:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.384+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 msgr2=0x55654b4fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.384+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 0x55654b4fee30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55654b490a80 tx=0x7f59480cadc0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 shutdown_connections 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55654b506300 0x55654b5086f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55654b497c50 0x55654b4a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55654b48cda0 0x55654b4fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 --2- 172.21.15.5:0/2321574562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55654b497300 0x55654b4fee30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 >> 172.21.15.5:0/2321574562 conn(0x55654b4012e0 msgr2=0x55654b496b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f59417fa700 0 client.25975 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 shutdown_connections 2023-09-23T04:47:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.385+0000 7f5973a66b80 1 -- 172.21.15.5:0/2321574562 wait complete. 2023-09-23T04:47:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.386+0000 7f5973a66b80 20 client.25975 trim_cache size 0 max 16384 2023-09-23T04:47:15.386 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a1dbae61-b84d-4f38-a1cb-58502b213484", 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.387+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 msgr2=0x7f59440a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.387+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 0x7f59440a3d70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59500094a0 tx=0x7f5950009040 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.387+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 msgr2=0x7f5958199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.387+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f594800e9c0 tx=0x7f594800ed80 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 shutdown_connections 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f59440a18b0 0x7f59440a3d70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595810d6c0 0x7f595819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5958073250 0x7f59581992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 --2- 172.21.15.5:0/3344729389 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5958073b70 0x7f5958199800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 >> 172.21.15.5:0/3344729389 conn(0x7f59580fc9d0 msgr2=0x7f595810f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 shutdown_connections 2023-09-23T04:47:15.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.388+0000 7f59601ff700 1 -- 172.21.15.5:0/3344729389 wait complete. 2023-09-23T04:47:15.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:47:15.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:15 smithi005 ceph-mon[97638]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2023-09-23T04:47:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 -- 172.21.15.5:0/2233398104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 msgr2=0x7f03d010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 --2- 172.21.15.5:0/2233398104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d010dd90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009f90 tx=0x7f03c800b330 comp rx=0 tx=0).stop 2023-09-23T04:47:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 -- 172.21.15.5:0/2233398104 shutdown_connections 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 --2- 172.21.15.5:0/2233398104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d010dd90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 --2- 172.21.15.5:0/2233398104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d0069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 --2- 172.21.15.5:0/2233398104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0069a50 0x7f03d0105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 -- 172.21.15.5:0/2233398104 >> 172.21.15.5:0/2233398104 conn(0x7f03d0077ca0 msgr2=0x7f03d00780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 -- 172.21.15.5:0/2233398104 shutdown_connections 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.771+0000 7f03d7fff700 1 -- 172.21.15.5:0/2233398104 wait complete. 2023-09-23T04:47:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.773+0000 7f03d7fff700 1 Processor -- start 2023-09-23T04:47:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.773+0000 7f03d7fff700 1 -- start start 2023-09-23T04:47:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d0100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 0x7f03d01a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01a56c0 con 0x7f03d00690a0 2023-09-23T04:47:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01a5830 con 0x7f03d0106210 2023-09-23T04:47:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d7fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01a59f0 con 0x7f03d0100a70 2023-09-23T04:47:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52440/0 (socket says 172.21.15.5:52440) 2023-09-23T04:47:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d5d9b700 1 -- 172.21.15.5:0/969558131 learned_addr learned my addr 172.21.15.5:0/969558131 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.774+0000 7f03d659c700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 0x7f03d01a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d559a700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d0100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 -- 172.21.15.5:0/969558131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 msgr2=0x7f03d01a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 0x7f03d01a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 -- 172.21.15.5:0/969558131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 msgr2=0x7f03d0100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d0100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 -- 172.21.15.5:0/969558131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8009c40 con 0x7f03d00690a0 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d659c700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 0x7f03d01a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.775+0000 7f03d5d9b700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9c0 tx=0x7f03cc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.776+0000 7f03c6ffd700 1 -- 172.21.15.5:0/969558131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00cbd0 con 0x7f03d00690a0 2023-09-23T04:47:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.776+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d01a5cd0 con 0x7f03d00690a0 2023-09-23T04:47:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.776+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d01a61d0 con 0x7f03d00690a0 2023-09-23T04:47:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.776+0000 7f03c6ffd700 1 -- 172.21.15.5:0/969558131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03cc00cd30 con 0x7f03d00690a0 2023-09-23T04:47:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.776+0000 7f03c6ffd700 1 -- 172.21.15.5:0/969558131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc005590 con 0x7f03d00690a0 2023-09-23T04:47:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.782+0000 7f03c6ffd700 1 -- 172.21.15.5:0/969558131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f03cc004750 con 0x7f03d00690a0 2023-09-23T04:47:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.782+0000 7f03eb898b80 1 Processor -- start 2023-09-23T04:47:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 -- start start 2023-09-23T04:47:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea2bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e987460 con 0x55d03ea29bd0 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e978c20 con 0x55d03ea1ed20 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.783+0000 7f03eb898b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e976540 con 0x55d03ea1e400 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52664/0 (socket says 172.21.15.5:52664) 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 -- 172.21.15.5:0/3741355545 learned_addr learned my addr 172.21.15.5:0/3741355545 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d559a700 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea2bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 -- 172.21.15.5:0/3741355545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 msgr2=0x55d03ea29690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea29690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 -- 172.21.15.5:0/3741355545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 msgr2=0x55d03ea2bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea2bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d659c700 1 -- 172.21.15.5:0/3741355545 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03e9d48d0 con 0x55d03ea1e400 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.784+0000 7f03d559a700 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.785+0000 7f03c6ffd700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 0x7f03bc0a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.785+0000 7f03c6ffd700 1 -- 172.21.15.5:0/969558131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f03cc014070 con 0x7f03d00690a0 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.785+0000 7f03d659c700 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03c8007ca0 tx=0x7f03c8007cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ba5875c52248f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.786+0000 7f03d5d9b700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 0x7f03bc0a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.786+0000 7f03bbfff700 1 -- 172.21.15.5:0/3741355545 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800e070 con 0x55d03ea1e400 2023-09-23T04:47:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.786+0000 7f03bbfff700 1 -- 172.21.15.5:0/3741355545 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c800b9f0 con 0x55d03ea1e400 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.786+0000 7f03bbfff700 1 -- 172.21.15.5:0/3741355545 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c802c440 con 0x55d03ea1e400 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 -- 172.21.15.5:0/3741355545 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 msgr2=0x55d03ea1e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03c8007ca0 tx=0x7f03c8007cd0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03d5d9b700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 0x7f03bc0a3df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0cad20 tx=0x7f03cc00cf10 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 -- 172.21.15.5:0/3741355545 shutdown_connections 2023-09-23T04:47:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1e400 0x55d03ea1e7e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3741355545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea29690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 -- 172.21.15.5:0/3741355545 >> 172.21.15.5:0/3741355545 conn(0x55d03e993260 msgr2=0x55d03ea1a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 -- 172.21.15.5:0/3741355545 shutdown_connections 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.787+0000 7f03eb898b80 1 -- 172.21.15.5:0/3741355545 wait complete. 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.788+0000 7f03eb898b80 1 -- start start 2023-09-23T04:47:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.788+0000 7f03eb898b80 1 Processor -- start 2023-09-23T04:47:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea26930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e971e30 con 0x55d03ea1e400 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e90e710 con 0x55d03ea1ed20 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e90e3b0 con 0x55d03ea29bd0 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.789+0000 7f03eb898b80 10 client.0 _ll_register_callbacks cb 0x55d03e76b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.790+0000 7f03d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.790+0000 7f03d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:52676/0 (socket says 172.21.15.5:52676) 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.790+0000 7f03d659c700 1 -- 172.21.15.5:0/2058653580 learned_addr learned my addr 172.21.15.5:0/2058653580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.790+0000 7f03d559a700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.790+0000 7f03d659c700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea26930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 -- 172.21.15.5:0/2058653580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 msgr2=0x55d03ea26e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea26e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 -- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 msgr2=0x55d03ea26930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea26930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 -- 172.21.15.5:0/2058653580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d010e600 con 0x55d03ea29bd0 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03d659c700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009040 tx=0x7f03c8007530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbd25284eb1dbe8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.791+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800e040 con 0x55d03ea29bd0 2023-09-23T04:47:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d03ea12250 con 0x55d03ea29bd0 2023-09-23T04:47:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f03c8035460 con 0x55d03ea29bd0 2023-09-23T04:47:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c8035620 con 0x55d03ea29bd0 2023-09-23T04:47:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03eb898b80 10 client.25987 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d03e79cde0 con 0x55d03ea29bd0 2023-09-23T04:47:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.792+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d03ea12250 con 0x55d03ea29bd0 2023-09-23T04:47:15.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f03c800b050 con 0x55d03ea29bd0 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 20 client.25987 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea41c80 0x55d03ea440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 4 client.25987 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d03e96ae10 con 0x55d03ea41c80 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.794+0000 7f03d559a700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea41c80 0x55d03ea440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.795+0000 7f03d559a700 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea41c80 0x55d03ea440a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4be16d584cef0d server_cookie=278bc9bbf6a43035 in_seq=0 out_seq=0 2023-09-23T04:47:15.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.795+0000 7f03b97fa700 10 client.25987 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.796+0000 7f03b97fa700 1 -- 172.21.15.5:0/2058653580 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d03e96ae10 con 0x55d03ea41c80 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.796+0000 7f03b97fa700 10 client.25987 handle_command_reply: tid=0 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.796+0000 7f03eb898b80 1 client.25987 shutdown 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.796+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 msgr2=0x55d03ea273b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.796+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009040 tx=0x7f03c8007530 comp rx=0 tx=0).stop 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 shutdown_connections 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea41c80 0x55d03ea440a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea273b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea26930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 --2- 172.21.15.5:0/2058653580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea26e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 >> 172.21.15.5:0/2058653580 conn(0x55d03e993260 msgr2=0x55d03ea1a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03b97fa700 0 client.25987 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 shutdown_connections 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 1 -- 172.21.15.5:0/2058653580 wait complete. 2023-09-23T04:47:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.797+0000 7f03eb898b80 20 client.25987 trim_cache size 0 max 16384 2023-09-23T04:47:15.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.832+0000 7f03eb898b80 1 Processor -- start 2023-09-23T04:47:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.832+0000 7f03eb898b80 1 -- start start 2023-09-23T04:47:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1ed20 0x55d03ea27230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03e7f2920 con 0x55d03ea29bd0 2023-09-23T04:47:15.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ea1f4d0 con 0x55d03ea1e400 2023-09-23T04:47:15.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03eb898b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ea1b5b0 con 0x55d03ea1ed20 2023-09-23T04:47:15.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03d659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03d659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53086/0 (socket says 172.21.15.5:53086) 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03d659c700 1 -- 172.21.15.5:0/579345529 learned_addr learned my addr 172.21.15.5:0/579345529 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03d559a700 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1ed20 0x55d03ea27230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.833+0000 7f03d659c700 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 -- 172.21.15.5:0/579345529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1ed20 msgr2=0x55d03ea27230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1ed20 0x55d03ea27230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 -- 172.21.15.5:0/579345529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 msgr2=0x55d03ea23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 -- 172.21.15.5:0/579345529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d03e785c60 con 0x55d03ea1e400 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03d659c700 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03c80060e0 tx=0x7f03c800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb927b6d6db93d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03c4ff9700 1 -- 172.21.15.5:0/579345529 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800b0b0 con 0x55d03ea1e400 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03c4ff9700 1 -- 172.21.15.5:0/579345529 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c803c070 con 0x55d03ea1e400 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.834+0000 7f03c4ff9700 1 -- 172.21.15.5:0/579345529 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c802c440 con 0x55d03ea1e400 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 -- 172.21.15.5:0/579345529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 msgr2=0x55d03ea30720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03c80060e0 tx=0x7f03c800b040 comp rx=0 tx=0).stop 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 -- 172.21.15.5:0/579345529 shutdown_connections 2023-09-23T04:47:15.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea29bd0 0x55d03ea23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1e400 0x55d03ea30720 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 --2- 172.21.15.5:0/579345529 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea1ed20 0x55d03ea27230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.835+0000 7f03eb898b80 1 -- 172.21.15.5:0/579345529 >> 172.21.15.5:0/579345529 conn(0x55d03e993260 msgr2=0x55d03ea26b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.836+0000 7f03eb898b80 1 -- 172.21.15.5:0/579345529 shutdown_connections 2023-09-23T04:47:15.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.836+0000 7f03eb898b80 1 -- 172.21.15.5:0/579345529 wait complete. 2023-09-23T04:47:15.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.836+0000 7f03eb898b80 1 -- start start 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 Processor -- start 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea8eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ea8f000 con 0x55d03ea1e400 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ea8f170 con 0x55d03ea1ed20 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d03ea8f330 con 0x55d03ea29bd0 2023-09-23T04:47:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.837+0000 7f03eb898b80 10 client.0 _ll_register_callbacks cb 0x7f03c8007920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52478/0 (socket says 172.21.15.5:52478) 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 -- 172.21.15.5:0/3293553677 learned_addr learned my addr 172.21.15.5:0/3293553677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea8eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d559a700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 -- 172.21.15.5:0/3293553677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 msgr2=0x55d03ea8eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea8eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 -- 172.21.15.5:0/3293553677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 msgr2=0x55d03ea8a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea8a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.838+0000 7f03d659c700 1 -- 172.21.15.5:0/3293553677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d03e79cde0 con 0x55d03ea1e400 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.839+0000 7f03d659c700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03c80061e0 tx=0x7f03c8006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc2dcb50746961 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.839+0000 7f03b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.839+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80358d0 con 0x55d03ea1e400 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.839+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f03c8035a30 con 0x55d03ea1e400 2023-09-23T04:47:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.839+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c800b9f0 con 0x55d03ea1e400 2023-09-23T04:47:15.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.840+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c8050460 con 0x55d03ea1e400 2023-09-23T04:47:15.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.840+0000 7f03eb898b80 10 client.15957 fetch_fsmap learned FSMap version 27 2023-09-23T04:47:15.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.840+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d03e7c3250 con 0x55d03ea1e400 2023-09-23T04:47:15.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.840+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03c805a020 con 0x55d03ea1e400 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f03c8041430 con 0x55d03ea1e400 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 20 client.15957 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea966a0 0x55d03ea98a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 4 client.15957 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.841+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d03e96ab30 con 0x55d03ea966a0 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.842+0000 7f03d559a700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea966a0 0x55d03ea98a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.842+0000 7f03d559a700 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea966a0 0x55d03ea98a90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685b9215063aac5c server_cookie=a50593bf0e3a28ac in_seq=0 out_seq=0 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.842+0000 7f03b97fa700 10 client.15957 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03b97fa700 1 -- 172.21.15.5:0/3293553677 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d03e96ab30 con 0x55d03ea966a0 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03b97fa700 10 client.15957 handle_command_reply: tid=0 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 client.15957 shutdown 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 msgr2=0x55d03ea89bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03c80061e0 tx=0x7f03c8006040 comp rx=0 tx=0).stop 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 shutdown_connections 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d03ea966a0 0x55d03ea98a90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d03ea29bd0 0x55d03ea8eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d03ea1e400 0x55d03ea89bf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 --2- 172.21.15.5:0/3293553677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d03ea1ed20 0x55d03ea8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 >> 172.21.15.5:0/3293553677 conn(0x55d03e993260 msgr2=0x55d03ea28240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03b97fa700 0 client.15957 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.843+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 shutdown_connections 2023-09-23T04:47:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.844+0000 7f03eb898b80 1 -- 172.21.15.5:0/3293553677 wait complete. 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.844+0000 7f03eb898b80 20 client.15957 trim_cache size 0 max 16384 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stdout: "a1dbae61-b84d-4f38-a1cb-58502b213484": { 2023-09-23T04:47:15.849 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:47:15.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a1dbae61-b84d-4f38-a1cb-58502b213484", 2023-09-23T04:47:15.850 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:47:15.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:47:15.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:47:15.850 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 msgr2=0x7f03bc0a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 0x7f03bc0a3df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0cad20 tx=0x7f03cc00cf10 comp rx=0 tx=0).stop 2023-09-23T04:47:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 msgr2=0x7f03d00ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9c0 tx=0x7f03cc00ed80 comp rx=0 tx=0).stop 2023-09-23T04:47:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 shutdown_connections 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f03bc0a1930 0x7f03bc0a3df0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f03d0100a70 0x7f03d01a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d00690a0 0x7f03d00ffff0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 --2- 172.21.15.5:0/969558131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03d0106210 0x7f03d0100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 >> 172.21.15.5:0/969558131 conn(0x7f03d0077ca0 msgr2=0x7f03d00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 shutdown_connections 2023-09-23T04:47:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:15.845+0000 7f03d7fff700 1 -- 172.21.15.5:0/969558131 wait complete. 2023-09-23T04:47:15.875 INFO:tasks.cephfs.filesystem:scrub status for tag:a1dbae61-b84d-4f38-a1cb-58502b213484 - {'path': '/', 'tag': 'a1dbae61-b84d-4f38-a1cb-58502b213484', 'options': 'recursive,force'} 2023-09-23T04:47:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:15 smithi069 ceph-mon[133931]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2023-09-23T04:47:16.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:15 smithi118 ceph-mon[134297]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 23 op/s 2023-09-23T04:47:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:16 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:47:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:16 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:16 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:16 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:16 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:47:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:16 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:16 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:16 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:16 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:47:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:16 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:16 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:16 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:17 smithi069 ceph-mon[133931]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-23T04:47:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:17 smithi005 ceph-mon[97638]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-23T04:47:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:17 smithi118 ceph-mon[134297]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-09-23T04:47:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:19 smithi069 ceph-mon[133931]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-23T04:47:20.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:19 smithi005 ceph-mon[97638]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-23T04:47:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:19 smithi118 ceph-mon[134297]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-23T04:47:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:20 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:47:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:20 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:47:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:20 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:47:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:21 smithi069 ceph-mon[133931]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-23T04:47:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:21 smithi005 ceph-mon[97638]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-23T04:47:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:21 smithi118 ceph-mon[134297]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-23T04:47:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:22 smithi005 ceph-mon[97638]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-23T04:47:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:22 smithi118 ceph-mon[134297]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-23T04:47:23.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:22 smithi069 ceph-mon[133931]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-23T04:47:23.335 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 62% (7447/11967) Updating files: 63% (7540/11967) Updating files: 64% (7659/11967) Updating files: 65% (7779/11967) Updating files: 66% (7899/11967) Updating files: 67% (8018/11967) Updating files: 68% (8138/11967) Updating files: 69% (8258/11967) Updating files: 70% (8377/11967) Updating files: 71% (8497/11967) Updating files: 72% (8617/11967) Updating files: 73% (8736/11967) Updating files: 74% (8856/11967) Updating files: 75% (8976/11967) Updating files: 76% (9095/11967) Updating files: 77% (9215/11967) Updating files: 78% (9335/11967) Updating files: 79% (9454/11967) Updating files: 80% (9574/11967) Updating files: 81% (9694/11967) Updating files: 82% (9813/11967) Updating files: 83% (9933/11967) Updating files: 84% (10053/11967) Updating files: 85% (10172/11967) Updating files: 86% (10292/11967) Updating files: 87% (10412/11967) Updating files: 88% (10531/11967) Updating files: 89% (10651/11967) Updating files: 90% (10771/11967) Updating files: 91% (10890/11967) Updating files: 92% (11010/11967) Updating files: 93% (11130/11967) Updating files: 94% (11249/11967) Updating files: 95% (11369/11967) Updating files: 96% (11489/11967) Updating files: 97% (11608/11967) Updating files: 98% (11728/11967) Updating files: 98% (11827/11967) Updating files: 99% (11848/11967) Updating files: 100% (11967/11967) Updating files: 100% (11967/11967), done. 2023-09-23T04:47:23.796 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '862663913a74a99e1c52d50e6a7e59dd6705749a'. 2023-09-23T04:47:23.796 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2023-09-23T04:47:23.797 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr: 2023-09-23T04:47:23.798 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 862663913a7 Merge PR #52623 into wip-vshankar-testing-20230922.144532 2023-09-23T04:47:23.807 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-09-23T04:47:23.863 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-09-23T04:47:23.868 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-23T04:47:23.868 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-09-23T04:47:24.445 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-09-23T04:47:24.524 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-09-23T04:47:24.589 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-09-23T04:47:24.593 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-23T04:47:24.593 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-09-23T04:47:24.657 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-09-23T04:47:24.664 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T04:47:24.664 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-09-23T04:47:24.708 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-09-23T04:47:24.709 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-09-23T04:47:24.709 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-09-23T04:47:24.807 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 1 2023-09-23T04:47:24.816 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-23T04:47:24.816 INFO:tasks.workunit.client.0.smithi005.stdout: 2023-09-23T04:47:24.816 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-23T04:47:24.817 INFO:tasks.workunit.client.0.smithi005.stdout:0 of 1 processes prepared for launch 0 sec 2023-09-23T04:47:24.918 INFO:tasks.workunit.client.0.smithi005.stdout:1 of 1 processes prepared for launch 0 sec 2023-09-23T04:47:24.918 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2023-09-23T04:47:25.918 INFO:tasks.workunit.client.0.smithi005.stdout: 1 2922 33.48 MB/sec warmup 1 sec latency 13.537 ms 2023-09-23T04:47:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:25 smithi069 ceph-mon[133931]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-23T04:47:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:25 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:47:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:25 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:25.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:25 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:25 smithi005 ceph-mon[97638]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-23T04:47:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:25 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:47:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:25 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:26.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:25 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:25 smithi118 ceph-mon[134297]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-23T04:47:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:25 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:47:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:25 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:25 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:26.918 INFO:tasks.workunit.client.0.smithi005.stdout: 1 5918 23.50 MB/sec warmup 2 sec latency 6.737 ms 2023-09-23T04:47:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:47:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:47:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:47:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:47:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:47:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:47:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:47:27.918 INFO:tasks.workunit.client.0.smithi005.stdout: 1 8692 21.46 MB/sec warmup 3 sec latency 5.823 ms 2023-09-23T04:47:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:27 smithi069 ceph-mon[133931]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:47:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:27 smithi005 ceph-mon[97638]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:47:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:27 smithi118 ceph-mon[134297]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2023-09-23T04:47:28.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11521 19.73 MB/sec warmup 4 sec latency 24.101 ms 2023-09-23T04:47:29.918 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14622 19.48 MB/sec warmup 5 sec latency 12.257 ms 2023-09-23T04:47:29.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:29 smithi069 ceph-mon[133931]: pgmap v39: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 369 KiB/s wr, 58 op/s 2023-09-23T04:47:30.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:29 smithi005 ceph-mon[97638]: pgmap v39: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 369 KiB/s wr, 58 op/s 2023-09-23T04:47:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:29 smithi118 ceph-mon[134297]: pgmap v39: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 369 KiB/s wr, 58 op/s 2023-09-23T04:47:30.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 17818 19.18 MB/sec warmup 6 sec latency 14.399 ms 2023-09-23T04:47:31.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18533 16.71 MB/sec warmup 7 sec latency 523.810 ms 2023-09-23T04:47:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:31 smithi069 ceph-mon[133931]: pgmap v40: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 369 KiB/s wr, 54 op/s 2023-09-23T04:47:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:31 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:31 smithi005 ceph-mon[97638]: pgmap v40: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 369 KiB/s wr, 54 op/s 2023-09-23T04:47:32.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:31 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:32.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:31 smithi118 ceph-mon[134297]: pgmap v40: 129 pgs: 129 active+clean; 5.1 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 369 KiB/s wr, 54 op/s 2023-09-23T04:47:32.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:31 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:32.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18533 14.62 MB/sec warmup 8 sec latency 1002.474 ms 2023-09-23T04:47:33.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18533 13.00 MB/sec warmup 9 sec latency 2002.583 ms 2023-09-23T04:47:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:33 smithi069 ceph-mon[133931]: pgmap v41: 129 pgs: 129 active+clean; 7.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 566 KiB/s wr, 80 op/s 2023-09-23T04:47:34.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:33 smithi005 ceph-mon[97638]: pgmap v41: 129 pgs: 129 active+clean; 7.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 566 KiB/s wr, 80 op/s 2023-09-23T04:47:34.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:33 smithi118 ceph-mon[134297]: pgmap v41: 129 pgs: 129 active+clean; 7.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 566 KiB/s wr, 80 op/s 2023-09-23T04:47:34.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18533 11.70 MB/sec warmup 10 sec latency 3002.672 ms 2023-09-23T04:47:35.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18656 10.66 MB/sec warmup 11 sec latency 3215.644 ms 2023-09-23T04:47:35.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:35 smithi069 ceph-mon[133931]: pgmap v42: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 887 KiB/s wr, 125 op/s 2023-09-23T04:47:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:35 smithi005 ceph-mon[97638]: pgmap v42: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 887 KiB/s wr, 125 op/s 2023-09-23T04:47:36.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:35 smithi118 ceph-mon[134297]: pgmap v42: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 887 KiB/s wr, 125 op/s 2023-09-23T04:47:36.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18657 9.77 MB/sec warmup 12 sec latency 998.476 ms 2023-09-23T04:47:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:36 smithi005 ceph-mon[97638]: pgmap v43: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 887 KiB/s wr, 123 op/s 2023-09-23T04:47:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:36 smithi118 ceph-mon[134297]: pgmap v43: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 887 KiB/s wr, 123 op/s 2023-09-23T04:47:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:36 smithi069 ceph-mon[133931]: pgmap v43: 129 pgs: 129 active+clean; 10 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 887 KiB/s wr, 123 op/s 2023-09-23T04:47:37.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18657 9.02 MB/sec warmup 13 sec latency 1030.199 ms 2023-09-23T04:47:38.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18657 8.38 MB/sec warmup 14 sec latency 2030.271 ms 2023-09-23T04:47:39.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18657 7.82 MB/sec warmup 15 sec latency 3030.403 ms 2023-09-23T04:47:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:39 smithi069 ceph-mon[133931]: pgmap v44: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 182 op/s 2023-09-23T04:47:40.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:39 smithi005 ceph-mon[97638]: pgmap v44: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 182 op/s 2023-09-23T04:47:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:39 smithi118 ceph-mon[134297]: pgmap v44: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 182 op/s 2023-09-23T04:47:40.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18657 7.33 MB/sec warmup 16 sec latency 4030.566 ms 2023-09-23T04:47:41.919 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18722 6.91 MB/sec warmup 17 sec latency 4992.189 ms 2023-09-23T04:47:41.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:41 smithi069 ceph-mon[133931]: pgmap v45: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.5 MiB/s wr, 131 op/s 2023-09-23T04:47:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:41 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:42.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:41 smithi005 ceph-mon[97638]: pgmap v45: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.5 MiB/s wr, 131 op/s 2023-09-23T04:47:42.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:41 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:41 smithi118 ceph-mon[134297]: pgmap v45: 129 pgs: 129 active+clean; 28 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 3.5 MiB/s wr, 131 op/s 2023-09-23T04:47:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:41 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:47:42.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21308 7.45 MB/sec warmup 18 sec latency 14.029 ms 2023-09-23T04:47:43.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22238 7.18 MB/sec warmup 19 sec latency 186.639 ms 2023-09-23T04:47:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:43 smithi069 ceph-mon[133931]: pgmap v46: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 3.5 MiB/s wr, 151 op/s 2023-09-23T04:47:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:43 smithi005 ceph-mon[97638]: pgmap v46: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 3.5 MiB/s wr, 151 op/s 2023-09-23T04:47:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:43 smithi118 ceph-mon[134297]: pgmap v46: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 3.5 MiB/s wr, 151 op/s 2023-09-23T04:47:44.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22238 6.82 MB/sec warmup 20 sec latency 1186.730 ms 2023-09-23T04:47:45.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:47:45.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22238 6.50 MB/sec warmup 21 sec latency 2186.796 ms 2023-09-23T04:47:45.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:45 smithi005 ceph-mon[97638]: pgmap v47: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 3.3 MiB/s wr, 127 op/s 2023-09-23T04:47:45.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:45 smithi069 ceph-mon[133931]: pgmap v47: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 3.3 MiB/s wr, 127 op/s 2023-09-23T04:47:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 -- 172.21.15.5:0/1536732574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24601010d0 msgr2=0x7f24601014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 --2- 172.21.15.5:0/1536732574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24601010d0 0x7f24601014b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2448009990 tx=0x7f2448009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 -- 172.21.15.5:0/1536732574 shutdown_connections 2023-09-23T04:47:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 --2- 172.21.15.5:0/1536732574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 0x7f246010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 --2- 172.21.15.5:0/1536732574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24601010d0 0x7f24601014b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 --2- 172.21.15.5:0/1536732574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2460101a80 0x7f2460105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 -- 172.21.15.5:0/1536732574 >> 172.21.15.5:0/1536732574 conn(0x7f24600fc990 msgr2=0x7f24600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 -- 172.21.15.5:0/1536732574 shutdown_connections 2023-09-23T04:47:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.114+0000 7f24665ff700 1 -- 172.21.15.5:0/1536732574 wait complete. 2023-09-23T04:47:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.115+0000 7f24665ff700 1 Processor -- start 2023-09-23T04:47:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.115+0000 7f24665ff700 1 -- start start 2023-09-23T04:47:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 0x7f2460199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246019dae0 0x7f246019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246019e560 con 0x7f24601010d0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246019e6d0 con 0x7f246019dae0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f24665ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246019e890 con 0x7f24601061f0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55102/0 (socket says 172.21.15.5:55102) 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 -- 172.21.15.5:0/4137870566 learned_addr learned my addr 172.21.15.5:0/4137870566 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f2457fff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 0x7f2460199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f2464b9c700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246019dae0 0x7f246019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 -- 172.21.15.5:0/4137870566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 msgr2=0x7f2460199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 0x7f2460199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 -- 172.21.15.5:0/4137870566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246019dae0 msgr2=0x7f246019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246019dae0 0x7f246019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.116+0000 7f245ffff700 1 -- 172.21.15.5:0/4137870566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480095f0 con 0x7f24601010d0 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f245ffff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2448004730 tx=0x7f2448004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f245dffb700 1 -- 172.21.15.5:0/4137870566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244801d070 con 0x7f24601010d0 2023-09-23T04:47:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24601a5d90 con 0x7f24601010d0 2023-09-23T04:47:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f245dffb700 1 -- 172.21.15.5:0/4137870566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2448005a00 con 0x7f24601010d0 2023-09-23T04:47:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f245dffb700 1 -- 172.21.15.5:0/4137870566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480053f0 con 0x7f24601010d0 2023-09-23T04:47:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.117+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24601a6200 con 0x7f24601010d0 2023-09-23T04:47:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.120+0000 7f245dffb700 1 -- 172.21.15.5:0/4137870566 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f2448005550 con 0x7f24601010d0 2023-09-23T04:47:46.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f245dffb700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 0x7f244c0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f2479e50b80 1 Processor -- start 2023-09-23T04:47:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f245dffb700 1 -- 172.21.15.5:0/4137870566 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f24480cea80 con 0x7f24601010d0 2023-09-23T04:47:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f2457fff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 0x7f244c0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f2479e50b80 1 -- start start 2023-09-23T04:47:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.122+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec85690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bec87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2479e50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bebe3460 con 0x5592bec85bd0 2023-09-23T04:47:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2479e50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bebd4c20 con 0x5592bec7ad20 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2479e50b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bebd2540 con 0x5592bec7a400 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44468/0 (socket says 172.21.15.5:44468) 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 -- 172.21.15.5:0/2286807593 learned_addr learned my addr 172.21.15.5:0/2286807593 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2457fff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 0x7f244c0a3cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f246019a750 tx=0x7f2450009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2464b9c700 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bec87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f2457fff700 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec85690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 -- 172.21.15.5:0/2286807593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 msgr2=0x5592bec85690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec85690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 -- 172.21.15.5:0/2286807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 msgr2=0x5592bec87fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bec87fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 -- 172.21.15.5:0/2286807593 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592bec308d0 con 0x5592bec7a400 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.123+0000 7f245ffff700 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f244809b210 tx=0x7f244809b3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f597ae059c0d564 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f24567fc700 1 -- 172.21.15.5:0/2286807593 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244809bc10 con 0x5592bec7a400 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f24567fc700 1 -- 172.21.15.5:0/2286807593 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244809b510 con 0x5592bec7a400 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f24567fc700 1 -- 172.21.15.5:0/2286807593 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480a35e0 con 0x5592bec7a400 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 -- 172.21.15.5:0/2286807593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 msgr2=0x5592bec7a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f244809b210 tx=0x7f244809b3d0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 -- 172.21.15.5:0/2286807593 shutdown_connections 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bec87fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec7a7e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 --2- 172.21.15.5:0/2286807593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec85690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.124+0000 7f2479e50b80 1 -- 172.21.15.5:0/2286807593 >> 172.21.15.5:0/2286807593 conn(0x5592bebef260 msgr2=0x5592bec76790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.125+0000 7f2479e50b80 1 -- 172.21.15.5:0/2286807593 shutdown_connections 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.125+0000 7f2479e50b80 1 -- 172.21.15.5:0/2286807593 wait complete. 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.125+0000 7f2479e50b80 1 -- start start 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.125+0000 7f2479e50b80 1 Processor -- start 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 0x5592bec864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec85bd0 0x5592bec86a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bebcde30 con 0x5592bec7ad20 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592beb6a710 con 0x5592bec85bd0 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592beb6a3b0 con 0x5592bec7a400 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2479e50b80 10 client.0 _ll_register_callbacks cb 0x5592be9c7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2457fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 0x5592bec864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f245ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f245ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44480/0 (socket says 172.21.15.5:44480) 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f245ffff700 1 -- 172.21.15.5:0/1707582080 learned_addr learned my addr 172.21.15.5:0/1707582080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.126+0000 7f2464b9c700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec85bd0 0x5592bec86a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 -- 172.21.15.5:0/1707582080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec85bd0 msgr2=0x5592bec86a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec85bd0 0x5592bec86a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 -- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 msgr2=0x5592bec864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 0x5592bec864f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 -- 172.21.15.5:0/1707582080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24601a6200 con 0x5592bec7a400 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f2457fff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 0x5592bec864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f245ffff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f244809b4a0 tx=0x7f24480a3700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ca458e241174ad server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f243f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244809f660 con 0x5592bec7a400 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592bec6e250 con 0x5592bec7a400 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f24480a3810 con 0x5592bec7a400 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24480998b0 con 0x5592bec7a400 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f2479e50b80 10 client.26005 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.127+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5592be9f8de0 con 0x5592bec7a400 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2448080020 con 0x5592bec7a400 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f244809f120 con 0x5592bec7a400 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 10 client.26005 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 20 client.26005 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bec94f10 0x5592bec97330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 4 client.26005 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592bebc6e10 con 0x5592bec94f10 2023-09-23T04:47:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.128+0000 7f2457fff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bec94f10 0x5592bec97330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.129+0000 7f2457fff700 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bec94f10 0x5592bec97330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9826dc9dde4e5de server_cookie=de173fe7986fa55d in_seq=0 out_seq=0 2023-09-23T04:47:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.129+0000 7f243f7fe700 10 client.26005 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.129+0000 7f243f7fe700 1 -- 172.21.15.5:0/1707582080 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5592bebc6e10 con 0x5592bec94f10 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.129+0000 7f243f7fe700 10 client.26005 handle_command_reply: tid=0 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.129+0000 7f2479e50b80 1 client.26005 shutdown 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 msgr2=0x5592bec85fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f244809b4a0 tx=0x7f24480a3700 comp rx=0 tx=0).stop 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 shutdown_connections 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bec94f10 0x5592bec97330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec85bd0 0x5592bec86a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7a400 0x5592bec85fb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 --2- 172.21.15.5:0/1707582080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7ad20 0x5592bec864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f243f7fe700 0 client.26005 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 >> 172.21.15.5:0/1707582080 conn(0x5592bebef260 msgr2=0x5592bec76320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 shutdown_connections 2023-09-23T04:47:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 1 -- 172.21.15.5:0/1707582080 wait complete. 2023-09-23T04:47:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.130+0000 7f2479e50b80 20 client.26005 trim_cache size 0 max 16384 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 Processor -- start 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 -- start start 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec80550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec85bd0 0x5592bec7e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bea4e920 con 0x5592bec7a400 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bec7bbf0 con 0x5592bec7ad20 2023-09-23T04:47:46.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2479e50b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bec83270 con 0x5592bec85bd0 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f245ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f245ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55134/0 (socket says 172.21.15.5:55134) 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f245ffff700 1 -- 172.21.15.5:0/952683348 learned_addr learned my addr 172.21.15.5:0/952683348 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2457fff700 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec80550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.148+0000 7f2464b9c700 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec85bd0 0x5592bec7e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 -- 172.21.15.5:0/952683348 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec85bd0 msgr2=0x5592bec7e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec85bd0 0x5592bec7e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 -- 172.21.15.5:0/952683348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 msgr2=0x5592bec80550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec80550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 -- 172.21.15.5:0/952683348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592be9e1c60 con 0x5592bec7a400 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f245ffff700 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f244809b210 tx=0x7f244800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b80ad676c23dba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f2456ffd700 1 -- 172.21.15.5:0/952683348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480a1420 con 0x5592bec7a400 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f2456ffd700 1 -- 172.21.15.5:0/952683348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2448080070 con 0x5592bec7a400 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.149+0000 7f2456ffd700 1 -- 172.21.15.5:0/952683348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480a1680 con 0x5592bec7a400 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- 172.21.15.5:0/952683348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 msgr2=0x5592bec80010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f244809b210 tx=0x7f244800b490 comp rx=0 tx=0).stop 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- 172.21.15.5:0/952683348 shutdown_connections 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec85bd0 0x5592bec7e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec7a400 0x5592bec80010 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 --2- 172.21.15.5:0/952683348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7ad20 0x5592bec80550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- 172.21.15.5:0/952683348 >> 172.21.15.5:0/952683348 conn(0x5592bebef260 msgr2=0x5592bec90740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- 172.21.15.5:0/952683348 shutdown_connections 2023-09-23T04:47:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- 172.21.15.5:0/952683348 wait complete. 2023-09-23T04:47:46.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.150+0000 7f2479e50b80 1 -- start start 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.151+0000 7f2479e50b80 1 Processor -- start 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.151+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.151+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 0x5592bec87e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.151+0000 7f2479e50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bece7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2479e50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bec88650 con 0x5592bec85bd0 2023-09-23T04:47:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2479e50b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bec887c0 con 0x5592bec7a400 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2479e50b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bece82b0 con 0x5592bec7ad20 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2479e50b80 10 client.0 _ll_register_callbacks cb 0x7f2448097940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53282/0 (socket says 172.21.15.5:53282) 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 -- 172.21.15.5:0/628168562 learned_addr learned my addr 172.21.15.5:0/628168562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2464b9c700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bece7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2457fff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 0x5592bec87e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 -- 172.21.15.5:0/628168562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 msgr2=0x5592bec87e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 0x5592bec87e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 -- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 msgr2=0x5592bece7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bece7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 -- 172.21.15.5:0/628168562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592be9f8de0 con 0x5592bec7a400 2023-09-23T04:47:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2457fff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 0x5592bec87e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f2464b9c700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bece7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f245ffff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f244809f4b0 tx=0x7f244809b180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2286110b157a34e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.152+0000 7f243f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480a1420 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f244809b510 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2448081080 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244809bc10 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 10 client.25958 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5592bea1f250 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244809b510 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f244809f120 con 0x5592bec7a400 2023-09-23T04:47:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 20 client.25958 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bece9510 0x5592beceb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 4 client.25958 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.153+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5592bebc6b30 con 0x5592bece9510 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f2457fff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bece9510 0x5592beceb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f243f7fe700 10 client.25958 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f2457fff700 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bece9510 0x5592beceb900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7c2fe5be6b57fd server_cookie=4d307e9ba0cf8139 in_seq=0 out_seq=0 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f243f7fe700 1 -- 172.21.15.5:0/628168562 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5592bebc6b30 con 0x5592bece9510 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f243f7fe700 10 client.25958 handle_command_reply: tid=0 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f2479e50b80 1 client.25958 shutdown 2023-09-23T04:47:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 msgr2=0x5592bec87950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.154+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f244809f4b0 tx=0x7f244809b180 comp rx=0 tx=0).stop 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 shutdown_connections 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5592bece9510 0x5592beceb900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592bec85bd0 0x5592bece7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5592bec7a400 0x5592bec87950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 --2- 172.21.15.5:0/628168562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5592bec7ad20 0x5592bec87e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f243f7fe700 0 client.25958 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 >> 172.21.15.5:0/628168562 conn(0x5592bebef260 msgr2=0x5592bec834c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 shutdown_connections 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 1 -- 172.21.15.5:0/628168562 wait complete. 2023-09-23T04:47:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f2479e50b80 20 client.25958 trim_cache size 0 max 16384 2023-09-23T04:47:46.160 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:47:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:47:46.160 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 msgr2=0x7f244c0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 0x7f244c0a3cf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f246019a750 tx=0x7f2450009260 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 msgr2=0x7f2460199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.155+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2448004730 tx=0x7f2448004760 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 shutdown_connections 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f244c0a1830 0x7f244c0a3cf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246019dae0 0x7f246019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24601010d0 0x7f2460199140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 --2- 172.21.15.5:0/4137870566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24601061f0 0x7f2460199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 >> 172.21.15.5:0/4137870566 conn(0x7f24600fc990 msgr2=0x7f246010d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 shutdown_connections 2023-09-23T04:47:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.156+0000 7f24665ff700 1 -- 172.21.15.5:0/4137870566 wait complete. 2023-09-23T04:47:46.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:45 smithi118 ceph-mon[134297]: pgmap v47: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 3.3 MiB/s wr, 127 op/s 2023-09-23T04:47:46.173 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:47:46.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 -- 172.21.15.5:0/138278702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc101a60 msgr2=0x7f7edc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/138278702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc101a60 0x7f7edc105aa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009990 tx=0x7f7ed0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 -- 172.21.15.5:0/138278702 shutdown_connections 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/138278702 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc1061d0 0x7f7edc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/138278702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/138278702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc101a60 0x7f7edc105aa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 -- 172.21.15.5:0/138278702 >> 172.21.15.5:0/138278702 conn(0x7f7edc0fc930 msgr2=0x7f7edc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.417+0000 7f7ee29ff700 1 -- 172.21.15.5:0/138278702 shutdown_connections 2023-09-23T04:47:46.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.418+0000 7f7ee29ff700 1 -- 172.21.15.5:0/138278702 wait complete. 2023-09-23T04:47:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.418+0000 7f7ee29ff700 1 Processor -- start 2023-09-23T04:47:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 -- start start 2023-09-23T04:47:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc19dbd0 0x7f7edc19e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc19e650 con 0x7f7edc1061d0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc19e7c0 con 0x7f7edc1010b0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee29ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc19e980 con 0x7f7edc19dbd0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7edb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7edbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7edb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55162/0 (socket says 172.21.15.5:55162) 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7edb7fe700 1 -- 172.21.15.5:0/1430675818 learned_addr learned my addr 172.21.15.5:0/1430675818 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.419+0000 7f7ee0f9c700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc19dbd0 0x7f7edc19e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 -- 172.21.15.5:0/1430675818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc19dbd0 msgr2=0x7f7edc19e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc19dbd0 0x7f7edc19e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 -- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 msgr2=0x7f7edc199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 -- 172.21.15.5:0/1430675818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc009520 con 0x7f7edc1010b0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edb7fe700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7edbfff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc199230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cb70 con 0x7f7edc1010b0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed00095f0 con 0x7f7edc1010b0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ecc00ee90 con 0x7f7edc1010b0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.420+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc003cf0 con 0x7f7edc1010b0 2023-09-23T04:47:46.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.421+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc1a61d0 con 0x7f7edc1010b0 2023-09-23T04:47:46.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.421+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc067ca0 con 0x7f7edc1010b0 2023-09-23T04:47:46.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.424+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f7ecc00ccd0 con 0x7f7edc1010b0 2023-09-23T04:47:46.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.425+0000 7f7ed97fa700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 0x7f7ec40a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.425+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7ecc014070 con 0x7f7edc1010b0 2023-09-23T04:47:46.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.425+0000 7f7edb7fe700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 0x7f7ec40a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.426+0000 7f7edb7fe700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 0x7f7ec40a3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7edc19a840 tx=0x7f7ed0005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.428+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7ecc0968d0 con 0x7f7edc1010b0 2023-09-23T04:47:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.687+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7edc101b00 con 0x7f7edc1010b0 2023-09-23T04:47:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.688+0000 7f7ed97fa700 1 -- 172.21.15.5:0/1430675818 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f7ecc099ef0 con 0x7f7edc1010b0 2023-09-23T04:47:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:47:46.688 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:47:40.065282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 msgr2=0x7f7ec40a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 0x7f7ec40a3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7edc19a840 tx=0x7f7ed0005fd0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 msgr2=0x7f7edc199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc199230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00ed00 comp rx=0 tx=0).stop 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 shutdown_connections 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ec40a1830 0x7f7ec40a3cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7edc19dbd0 0x7f7edc19e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7edc1010b0 0x7f7edc199230 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 --2- 172.21.15.5:0/1430675818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7edc1061d0 0x7f7edc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 >> 172.21.15.5:0/1430675818 conn(0x7f7edc0fc930 msgr2=0x7f7edc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 shutdown_connections 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.691+0000 7f7ee29ff700 1 -- 172.21.15.5:0/1430675818 wait complete. 2023-09-23T04:47:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2023-09-23T04:47:46.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:47:46.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22393 6.26 MB/sec warmup 22 sec latency 3070.068 ms 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.930+0000 7f38917ff700 1 -- 172.21.15.5:0/2838694302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 msgr2=0x7f388c073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.930+0000 7f38917ff700 1 --2- 172.21.15.5:0/2838694302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c073630 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 -- 172.21.15.5:0/2838694302 shutdown_connections 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 --2- 172.21.15.5:0/2838694302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c10d6a0 0x7f388c10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 --2- 172.21.15.5:0/2838694302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c073630 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 --2- 172.21.15.5:0/2838694302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c073b70 0x7f388c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 -- 172.21.15.5:0/2838694302 >> 172.21.15.5:0/2838694302 conn(0x7f388c0fc990 msgr2=0x7f388c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 -- 172.21.15.5:0/2838694302 shutdown_connections 2023-09-23T04:47:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.931+0000 7f38917ff700 1 -- 172.21.15.5:0/2838694302 wait complete. 2023-09-23T04:47:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 Processor -- start 2023-09-23T04:47:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 -- start start 2023-09-23T04:47:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 0x7f388c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c19e150 con 0x7f388c073250 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c19e2c0 con 0x7f388c073b70 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.932+0000 7f38917ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c19e480 con 0x7f388c10d6a0 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53316/0 (socket says 172.21.15.5:53316) 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 -- 172.21.15.5:0/878533402 learned_addr learned my addr 172.21.15.5:0/878533402 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 0x7f388c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 -- 172.21.15.5:0/878533402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 msgr2=0x7f388c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 0x7f388c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 -- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 msgr2=0x7f388c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 -- 172.21.15.5:0/878533402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0095f0 con 0x7f388c073b70 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388b7fe700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 0x7f388c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388affd700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.933+0000 7f388a7fc700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f388000e9a0 tx=0x7f388000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.934+0000 7f3873fff700 1 -- 172.21.15.5:0/878533402 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880009790 con 0x7f388c073b70 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.934+0000 7f3873fff700 1 -- 172.21.15.5:0/878533402 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3880010430 con 0x7f388c073b70 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.934+0000 7f3873fff700 1 -- 172.21.15.5:0/878533402 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880018680 con 0x7f388c073b70 2023-09-23T04:47:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.934+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c19e760 con 0x7f388c073b70 2023-09-23T04:47:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.934+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c1a5fa0 con 0x7f388c073b70 2023-09-23T04:47:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.939+0000 7f3873fff700 1 -- 172.21.15.5:0/878533402 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f38800105a0 con 0x7f388c073b70 2023-09-23T04:47:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 Processor -- start 2023-09-23T04:47:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 -- start start 2023-09-23T04:47:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c62741fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6269d460 con 0x559c62734400 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6268ec20 con 0x559c6273fbd0 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.940+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6268c540 con 0x559c62734d20 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c62741fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44542/0 (socket says 172.21.15.5:44542) 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 -- 172.21.15.5:0/3223849480 learned_addr learned my addr 172.21.15.5:0/3223849480 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f3873fff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 0x7f38740a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 -- 172.21.15.5:0/3223849480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 msgr2=0x559c62741fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c62741fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 -- 172.21.15.5:0/3223849480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 msgr2=0x559c627347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388affd700 1 -- 172.21.15.5:0/3223849480 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c626ea8d0 con 0x559c62734d20 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f3873fff700 1 -- 172.21.15.5:0/878533402 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3880014070 con 0x7f388c073b70 2023-09-23T04:47:46.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.941+0000 7f388b7fe700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 0x7f38740a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.942+0000 7f388affd700 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f387c005ee0 tx=0x7f387c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce8c3f59790fd6c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.942+0000 7f38717fa700 1 -- 172.21.15.5:0/3223849480 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c01d070 con 0x559c62734d20 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.942+0000 7f38717fa700 1 -- 172.21.15.5:0/3223849480 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f387c01e040 con 0x559c62734d20 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.942+0000 7f38717fa700 1 -- 172.21.15.5:0/3223849480 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c004e00 con 0x559c62734d20 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- 172.21.15.5:0/3223849480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 msgr2=0x559c6273f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f388b7fe700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 0x7f38740a3cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3884009110 tx=0x7f3884004ad0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f387c005ee0 tx=0x7f387c005f10 comp rx=0 tx=0).stop 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- 172.21.15.5:0/3223849480 shutdown_connections 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c62741fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/3223849480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273f690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- 172.21.15.5:0/3223849480 >> 172.21.15.5:0/3223849480 conn(0x559c626a9260 msgr2=0x559c62730790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- 172.21.15.5:0/3223849480 shutdown_connections 2023-09-23T04:47:46.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- 172.21.15.5:0/3223849480 wait complete. 2023-09-23T04:47:46.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.943+0000 7f38a51c8b80 1 -- start start 2023-09-23T04:47:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 Processor -- start 2023-09-23T04:47:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c6273f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c6273cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c62687e30 con 0x559c62734400 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c62624710 con 0x559c6273fbd0 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c626243b0 con 0x559c62734d20 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.944+0000 7f38a51c8b80 10 client.0 _ll_register_callbacks cb 0x559c62481800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44544/0 (socket says 172.21.15.5:44544) 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 -- 172.21.15.5:0/4196194167 learned_addr learned my addr 172.21.15.5:0/4196194167 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388affd700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c6273f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388b7fe700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c6273cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 -- 172.21.15.5:0/4196194167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 msgr2=0x559c6273cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c6273cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 -- 172.21.15.5:0/4196194167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 msgr2=0x559c6273f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c6273f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 -- 172.21.15.5:0/4196194167 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c1107d0 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388affd700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c6273f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.945+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f388009a1a0 tx=0x7f388009a9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ffc3735af66b6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800a25a0 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c62728250 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f388009aba0 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38800967e0 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 10 client.26029 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c624b2de0 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c62728250 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3880096290 con 0x559c62734d20 2023-09-23T04:47:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 10 client.26029 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 10 client.26029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.946+0000 7f38a51c8b80 20 client.26029 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c6274f4d0 0x559c627518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f38a51c8b80 4 client.26029 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c62680e10 con 0x559c6274f4d0 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f388affd700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c6274f4d0 0x559c627518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f388affd700 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c6274f4d0 0x559c627518f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c6309ab6294f370 server_cookie=beefd9adedbbc757 in_seq=0 out_seq=0 2023-09-23T04:47:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.947+0000 7f386eff5700 10 client.26029 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f386eff5700 1 -- 172.21.15.5:0/4196194167 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c62680e10 con 0x559c6274f4d0 2023-09-23T04:47:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f386eff5700 10 client.26029 handle_command_reply: tid=0 2023-09-23T04:47:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 client.26029 shutdown 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 msgr2=0x559c6273c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f388009a1a0 tx=0x7f388009a9e0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 shutdown_connections 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c6273fbd0 0x559c6273cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c6274f4d0 0x559c627518f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c6273f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4196194167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c62734d20 0x559c6273c840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f386eff5700 0 client.26029 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.948+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 >> 172.21.15.5:0/4196194167 conn(0x559c626a9260 msgr2=0x559c62730320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.949+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 shutdown_connections 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.949+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4196194167 wait complete. 2023-09-23T04:47:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.949+0000 7f38a51c8b80 20 client.26029 trim_cache size 0 max 16384 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 Processor -- start 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 -- start start 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 0x559c6273f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c6273c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c62508920 con 0x559c62734d20 2023-09-23T04:47:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c627320b0 con 0x559c62734400 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.964+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6273bdd0 con 0x559c6273fbd0 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53336/0 (socket says 172.21.15.5:53336) 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 -- 172.21.15.5:0/1034421837 learned_addr learned my addr 172.21.15.5:0/1034421837 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388b7fe700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c6273c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388a7fc700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 0x559c6273f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 -- 172.21.15.5:0/1034421837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 msgr2=0x559c6273c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c6273c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 -- 172.21.15.5:0/1034421837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 msgr2=0x559c6273f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 0x559c6273f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 -- 172.21.15.5:0/1034421837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c6249bc60 con 0x559c62734400 2023-09-23T04:47:46.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388a7fc700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 0x559c6273f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:47:46.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.965+0000 7f388affd700 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f387c000c00 tx=0x7f387c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be8eb3521b170c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.966+0000 7f3871ffb700 1 -- 172.21.15.5:0/1034421837 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c01d040 con 0x559c62734400 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.966+0000 7f3871ffb700 1 -- 172.21.15.5:0/1034421837 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f387c034cd0 con 0x559c62734400 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.966+0000 7f3871ffb700 1 -- 172.21.15.5:0/1034421837 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c022a00 con 0x559c62734400 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- 172.21.15.5:0/1034421837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 msgr2=0x559c627372a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f387c000c00 tx=0x7f387c01a040 comp rx=0 tx=0).stop 2023-09-23T04:47:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- 172.21.15.5:0/1034421837 shutdown_connections 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c6273c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734400 0x559c627372a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/1034421837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734d20 0x559c6273f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- 172.21.15.5:0/1034421837 >> 172.21.15.5:0/1034421837 conn(0x559c626a9260 msgr2=0x559c62737b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- 172.21.15.5:0/1034421837 shutdown_connections 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- 172.21.15.5:0/1034421837 wait complete. 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 -- start start 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.967+0000 7f38a51c8b80 1 Processor -- start 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 0x559c62744be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c62745160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c627a24f0 con 0x559c62734400 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c627a2660 con 0x559c62734d20 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c627a2820 con 0x559c6273fbd0 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f38a51c8b80 10 client.0 _ll_register_callbacks cb 0x7f3880010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f388affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f388affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55234/0 (socket says 172.21.15.5:55234) 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f388affd700 1 -- 172.21.15.5:0/4205996817 learned_addr learned my addr 172.21.15.5:0/4205996817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f388b7fe700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c62745160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.968+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 0x559c62744be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 -- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 msgr2=0x559c62745160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c62745160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 -- 172.21.15.5:0/4205996817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 msgr2=0x559c62744be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 0x559c62744be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 -- 172.21.15.5:0/4205996817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c626ea8d0 con 0x559c62734400 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 0x559c62744be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f388affd700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2d9d12d091ea2a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f386eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c01d070 con 0x559c62734400 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f387c01e040 con 0x559c62734400 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f387c034e40 con 0x559c62734400 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f387c004680 con 0x559c62734400 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.969+0000 7f38a51c8b80 10 client.15996 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c624d9250 con 0x559c62734400 2023-09-23T04:47:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f387c01e040 con 0x559c62734400 2023-09-23T04:47:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f387c03c070 con 0x559c62734400 2023-09-23T04:47:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 10 client.15996 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:47:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 20 client.15996 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c627a3a80 0x559c627a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 4 client.15996 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c62680b30 con 0x559c627a3a80 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.970+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c627a3a80 0x559c627a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f388a7fc700 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c627a3a80 0x559c627a5ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ba108dde4178ec2 server_cookie=2e16ffe697999eb4 in_seq=0 out_seq=0 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f386eff5700 10 client.15996 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f386eff5700 1 -- 172.21.15.5:0/4205996817 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c62680b30 con 0x559c627a3a80 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f386eff5700 10 client.15996 handle_command_reply: tid=0 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 client.15996 shutdown 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 msgr2=0x559c627446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c00b410 comp rx=0 tx=0).stop 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 shutdown_connections 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559c6273fbd0 0x559c62745160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x559c627a3a80 0x559c627a5ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c62734400 0x559c627446a0 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c00b410 comp rx=0 tx=0).stop 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 --2- 172.21.15.5:0/4205996817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559c62734d20 0x559c62744be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f386eff5700 0 client.15996 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 >> 172.21.15.5:0/4205996817 conn(0x559c626a9260 msgr2=0x559c6273c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 shutdown_connections 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.971+0000 7f38a51c8b80 1 -- 172.21.15.5:0/4205996817 wait complete. 2023-09-23T04:47:46.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38a51c8b80 20 client.15996 trim_cache size 0 max 16384 2023-09-23T04:47:46.977 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 msgr2=0x7f38740a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 0x7f38740a3cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3884009110 tx=0x7f3884004ad0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 msgr2=0x7f388c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f388000e9a0 tx=0x7f388000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 shutdown_connections 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f38740a1830 0x7f38740a3cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f388c10d6a0 0x7f388c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c073250 0x7f388c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 --2- 172.21.15.5:0/878533402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f388c073b70 0x7f388c1996b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 >> 172.21.15.5:0/878533402 conn(0x7f388c0fc990 msgr2=0x7f388c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.972+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 shutdown_connections 2023-09-23T04:47:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:46.973+0000 7f38917ff700 1 -- 172.21.15.5:0/878533402 wait complete. 2023-09-23T04:47:46.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:46 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1430675818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:46.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:47:46.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:46 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1430675818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:46 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1430675818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.232+0000 7f8633dff700 1 -- 172.21.15.5:0/4057923803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c10d6f0 msgr2=0x7f862c10fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.232+0000 7f8633dff700 1 --2- 172.21.15.5:0/4057923803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c10d6f0 0x7f862c10fae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f862400b0a0 tx=0x7f862400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 -- 172.21.15.5:0/4057923803 shutdown_connections 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 --2- 172.21.15.5:0/4057923803 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c10d6f0 0x7f862c10fae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 --2- 172.21.15.5:0/4057923803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c0731d0 0x7f862c0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 --2- 172.21.15.5:0/4057923803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c073af0 0x7f862c10d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 -- 172.21.15.5:0/4057923803 >> 172.21.15.5:0/4057923803 conn(0x7f862c0fc920 msgr2=0x7f862c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 -- 172.21.15.5:0/4057923803 shutdown_connections 2023-09-23T04:47:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.233+0000 7f8633dff700 1 -- 172.21.15.5:0/4057923803 wait complete. 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.234+0000 7f8633dff700 1 Processor -- start 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.234+0000 7f8633dff700 1 -- start start 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.234+0000 7f8633dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 0x7f862c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.234+0000 7f8633dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 0x7f862c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.234+0000 7f8633dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 0x7f862c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8633dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c19e280 con 0x7f862c10d6f0 2023-09-23T04:47:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8633dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c19e3f0 con 0x7f862c073af0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8633dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c19e5b0 con 0x7f862c0731d0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 0x7f862c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 0x7f862c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 0x7f862c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44580/0 (socket says 172.21.15.5:44580) 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f8631b9b700 1 -- 172.21.15.5:0/637912167 learned_addr learned my addr 172.21.15.5:0/637912167 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863239c700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 0x7f862c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 -- 172.21.15.5:0/637912167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 msgr2=0x7f862c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 0x7f862c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 -- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 msgr2=0x7f862c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 0x7f862c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863139a700 1 -- 172.21.15.5:0/637912167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624009d00 con 0x7f862c073af0 2023-09-23T04:47:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.235+0000 7f863239c700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 0x7f862c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f863139a700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 0x7f862c1997e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f861c00d950 tx=0x7f861c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f8622ffd700 1 -- 172.21.15.5:0/637912167 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00f890 con 0x7f862c073af0 2023-09-23T04:47:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f8622ffd700 1 -- 172.21.15.5:0/637912167 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861c010460 con 0x7f862c073af0 2023-09-23T04:47:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c19e890 con 0x7f862c073af0 2023-09-23T04:47:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f8622ffd700 1 -- 172.21.15.5:0/637912167 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00e610 con 0x7f862c073af0 2023-09-23T04:47:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.236+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c1a60d0 con 0x7f862c073af0 2023-09-23T04:47:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.240+0000 7f8622ffd700 1 -- 172.21.15.5:0/637912167 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f861c0108d0 con 0x7f862c073af0 2023-09-23T04:47:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.241+0000 7f8647784b80 1 Processor -- start 2023-09-23T04:47:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 -- start start 2023-09-23T04:47:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a7867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a793fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a6ef460 con 0x561f0a786400 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a6e0c20 con 0x561f0a786d20 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8647784b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a6de540 con 0x561f0a791bd0 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f863239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a7867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53374/0 (socket says 172.21.15.5:53374) 2023-09-23T04:47:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.242+0000 7f8631b9b700 1 -- 172.21.15.5:0/2705344754 learned_addr learned my addr 172.21.15.5:0/2705344754 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f863139a700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a793fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8622ffd700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 0x7f86180a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 -- 172.21.15.5:0/2705344754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 msgr2=0x561f0a793fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a793fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 -- 172.21.15.5:0/2705344754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 msgr2=0x561f0a7867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a7867e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 -- 172.21.15.5:0/2705344754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f0a73c8d0 con 0x561f0a786d20 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f863139a700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a793fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f863239c700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a7867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f863239c700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 0x7f86180a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8622ffd700 1 -- 172.21.15.5:0/637912167 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f861c0cd550 con 0x7f862c073af0 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.243+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f862800ae40 tx=0x7f862800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d843e9b1443ad022 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.244+0000 7f8617fff700 1 -- 172.21.15.5:0/2705344754 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800a940 con 0x561f0a786d20 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.244+0000 7f8617fff700 1 -- 172.21.15.5:0/2705344754 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8628018900 con 0x561f0a786d20 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.244+0000 7f8617fff700 1 -- 172.21.15.5:0/2705344754 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280109e0 con 0x561f0a786d20 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- 172.21.15.5:0/2705344754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 msgr2=0x561f0a791690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f862800ae40 tx=0x7f862800f6a0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f863239c700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 0x7f86180a3d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f862400b830 tx=0x7f86240099e0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- 172.21.15.5:0/2705344754 shutdown_connections 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a793fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a7867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 --2- 172.21.15.5:0/2705344754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a791690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- 172.21.15.5:0/2705344754 >> 172.21.15.5:0/2705344754 conn(0x561f0a6fb260 msgr2=0x561f0a782790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- 172.21.15.5:0/2705344754 shutdown_connections 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- 172.21.15.5:0/2705344754 wait complete. 2023-09-23T04:47:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.245+0000 7f8647784b80 1 -- start start 2023-09-23T04:47:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 Processor -- start 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a78ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a791bd0 0x561f0a78f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a6d9e30 con 0x561f0a791bd0 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a676710 con 0x561f0a786d20 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a6763b0 con 0x561f0a786400 2023-09-23T04:47:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.246+0000 7f8647784b80 10 client.0 _ll_register_callbacks cb 0x561f0a4d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f863139a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a78ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44586/0 (socket says 172.21.15.5:44586) 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 -- 172.21.15.5:0/2112269783 learned_addr learned my addr 172.21.15.5:0/2112269783 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f863239c700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a791bd0 0x561f0a78f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 -- 172.21.15.5:0/2112269783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 msgr2=0x561f0a78ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a78ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 -- 172.21.15.5:0/2112269783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a791bd0 msgr2=0x561f0a78f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a791bd0 0x561f0a78f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 -- 172.21.15.5:0/2112269783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c1a60d0 con 0x561f0a786400 2023-09-23T04:47:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f863139a700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a78ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f8631b9b700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8628010620 tx=0x7f8628018ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b56f87668541c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f86157fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.247+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800a920 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8628005250 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f0a77a250 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8628005410 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f8647784b80 10 client.26047 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f862800aaa0 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.248+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f0a77a250 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8628010680 con 0x561f0a786400 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 10 client.26047 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:47:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 20 client.26047 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7a15f0 0x561f0a7a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 4 client.26047 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f0a6d2e10 con 0x561f0a7a15f0 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.249+0000 7f863139a700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7a15f0 0x561f0a7a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.250+0000 7f86157fa700 10 client.26047 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.250+0000 7f863139a700 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7a15f0 0x561f0a7a3a10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca426fb4766107a2 server_cookie=3fd3fb28eccd599f in_seq=0 out_seq=0 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f86157fa700 1 -- 172.21.15.5:0/2112269783 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f0a6d2e10 con 0x561f0a7a15f0 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f86157fa700 10 client.26047 handle_command_reply: tid=0 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 client.26047 shutdown 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 msgr2=0x561f0a78e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8628010620 tx=0x7f8628018ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 shutdown_connections 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7a15f0 0x561f0a7a3a10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a791bd0 0x561f0a78f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a786400 0x561f0a78e910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 --2- 172.21.15.5:0/2112269783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a78ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 >> 172.21.15.5:0/2112269783 conn(0x561f0a6fb260 msgr2=0x561f0a782320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f86157fa700 0 client.26047 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 shutdown_connections 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 1 -- 172.21.15.5:0/2112269783 wait complete. 2023-09-23T04:47:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.251+0000 7f8647784b80 20 client.26047 trim_cache size 0 max 16384 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 Processor -- start 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 -- start start 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 0x561f0a789370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786d20 0x561f0a78f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a78d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a55a920 con 0x561f0a786d20 2023-09-23T04:47:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a79e0a0 con 0x561f0a786400 2023-09-23T04:47:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.266+0000 7f8647784b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a78a4c0 con 0x561f0a791bd0 2023-09-23T04:47:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f863239c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a78d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 0x561f0a789370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f863239c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a78d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44598/0 (socket says 172.21.15.5:44598) 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f863239c700 1 -- 172.21.15.5:0/209750213 learned_addr learned my addr 172.21.15.5:0/209750213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f863139a700 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786d20 0x561f0a78f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 -- 172.21.15.5:0/209750213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 msgr2=0x561f0a78d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a78d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 -- 172.21.15.5:0/209750213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786d20 msgr2=0x561f0a78f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786d20 0x561f0a78f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 -- 172.21.15.5:0/209750213 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f0a504de0 con 0x561f0a786400 2023-09-23T04:47:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.267+0000 7f8631b9b700 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 0x561f0a789370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f862800ad90 tx=0x7f8628001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0e4bd242f016c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8620bff700 1 -- 172.21.15.5:0/209750213 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628010430 con 0x561f0a786400 2023-09-23T04:47:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8620bff700 1 -- 172.21.15.5:0/209750213 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8628005710 con 0x561f0a786400 2023-09-23T04:47:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8620bff700 1 -- 172.21.15.5:0/209750213 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628004950 con 0x561f0a786400 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 -- 172.21.15.5:0/209750213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 msgr2=0x561f0a789370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 0x561f0a789370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f862800ad90 tx=0x7f8628001e70 comp rx=0 tx=0).stop 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 -- 172.21.15.5:0/209750213 shutdown_connections 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a78d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786400 0x561f0a789370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 --2- 172.21.15.5:0/209750213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786d20 0x561f0a78f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.268+0000 7f8647784b80 1 -- 172.21.15.5:0/209750213 >> 172.21.15.5:0/209750213 conn(0x561f0a6fb260 msgr2=0x561f0a78ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.269+0000 7f8647784b80 1 -- 172.21.15.5:0/209750213 shutdown_connections 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.269+0000 7f8647784b80 1 -- 172.21.15.5:0/209750213 wait complete. 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.269+0000 7f8647784b80 1 -- start start 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.269+0000 7f8647784b80 1 Processor -- start 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a796320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a796860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a7f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a796f40 con 0x561f0a786400 2023-09-23T04:47:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a7f4980 con 0x561f0a786d20 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0a7f4ac0 con 0x561f0a791bd0 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8647784b80 10 client.0 _ll_register_callbacks cb 0x7f8624009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a796320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a796320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55280/0 (socket says 172.21.15.5:55280) 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f8631b9b700 1 -- 172.21.15.5:0/3689996730 learned_addr learned my addr 172.21.15.5:0/3689996730 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a796860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863239c700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a7f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 -- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 msgr2=0x561f0a7f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a7f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 -- 172.21.15.5:0/3689996730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 msgr2=0x561f0a796320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a796320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.270+0000 7f863139a700 1 -- 172.21.15.5:0/3689996730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862800aaa0 con 0x561f0a786d20 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f863139a700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a796860 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ca6e0 tx=0x7f861c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bad9d2df79ea966 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f86157fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c014070 con 0x561f0a786d20 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8628018900 con 0x561f0a786d20 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f861c09a530 con 0x561f0a786d20 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f861c099790 con 0x561f0a786d20 2023-09-23T04:47:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f8647784b80 10 client.25997 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.271+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561f0a4edc60 con 0x561f0a786d20 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861c0b6080 con 0x561f0a786d20 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f861c09a090 con 0x561f0a786d20 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 10 client.25997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 20 client.25997 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7f6290 0x561f0a7f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 4 client.25997 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f0a6d2b30 con 0x561f0a7f6290 2023-09-23T04:47:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.272+0000 7f8631b9b700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7f6290 0x561f0a7f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8631b9b700 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7f6290 0x561f0a7f86b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c2e99539deebd41 server_cookie=8f3bf96617f7a7f3 in_seq=0 out_seq=0 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f86157fa700 10 client.25997 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f86157fa700 1 -- 172.21.15.5:0/3689996730 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f0a6d2b30 con 0x561f0a7f6290 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f86157fa700 10 client.25997 handle_command_reply: tid=0 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 client.25997 shutdown 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 msgr2=0x561f0a796860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a796860 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ca6e0 tx=0x7f861c00bf90 comp rx=0 tx=0).stop 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 shutdown_connections 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561f0a791bd0 0x561f0a7f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x561f0a7f6290 0x561f0a7f86b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f0a786400 0x561f0a796320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 --2- 172.21.15.5:0/3689996730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561f0a786d20 0x561f0a796860 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 >> 172.21.15.5:0/3689996730 conn(0x561f0a6fb260 msgr2=0x561f0a78eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.273+0000 7f86157fa700 0 client.25997 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 shutdown_connections 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8647784b80 1 -- 172.21.15.5:0/3689996730 wait complete. 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8647784b80 20 client.25997 trim_cache size 0 max 16384 2023-09-23T04:47:47.278 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 msgr2=0x7f86180a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 0x7f86180a3d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f862400b830 tx=0x7f86240099e0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 msgr2=0x7f862c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 0x7f862c1997e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f861c00d950 tx=0x7f861c00dc60 comp rx=0 tx=0).stop 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 shutdown_connections 2023-09-23T04:47:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f86180a18b0 0x7f86180a3d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f862c10d6f0 0x7f862c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f862c0731d0 0x7f862c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 --2- 172.21.15.5:0/637912167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f862c073af0 0x7f862c1997e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.274+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 >> 172.21.15.5:0/637912167 conn(0x7f862c0fc920 msgr2=0x7f862c105150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.275+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 shutdown_connections 2023-09-23T04:47:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.275+0000 7f8633dff700 1 -- 172.21.15.5:0/637912167 wait complete. 2023-09-23T04:47:47.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:47:47.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 -- 172.21.15.5:0/2111601131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 msgr2=0x7f6ed40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/2111601131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed40735d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009990 tx=0x7f6ec4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 -- 172.21.15.5:0/2111601131 shutdown_connections 2023-09-23T04:47:47.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/2111601131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 0x7f6ed410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/2111601131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed40735d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/2111601131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 0x7f6ed410d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 -- 172.21.15.5:0/2111601131 >> 172.21.15.5:0/2111601131 conn(0x7f6ed40fc920 msgr2=0x7f6ed40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 -- 172.21.15.5:0/2111601131 shutdown_connections 2023-09-23T04:47:47.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.517+0000 7f6eda3ff700 1 -- 172.21.15.5:0/2111601131 wait complete. 2023-09-23T04:47:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.518+0000 7f6eda3ff700 1 Processor -- start 2023-09-23T04:47:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 -- start start 2023-09-23T04:47:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 0x7f6ed4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 0x7f6ed419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed419e1d0 con 0x7f6ed40731f0 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed419e340 con 0x7f6ed410d6d0 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6eda3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed419e500 con 0x7f6ed4073b10 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55304/0 (socket says 172.21.15.5:55304) 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6ed3fff700 1 -- 172.21.15.5:0/188492114 learned_addr learned my addr 172.21.15.5:0/188492114 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6ed899c700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 0x7f6ed419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.519+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 0x7f6ed4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 -- 172.21.15.5:0/188492114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 msgr2=0x7f6ed4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 0x7f6ed4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 -- 172.21.15.5:0/188492114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 msgr2=0x7f6ed419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 0x7f6ed419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 -- 172.21.15.5:0/188492114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40095f0 con 0x7f6ed40731f0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005910 tx=0x7f6ec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6ed17fa700 1 -- 172.21.15.5:0/188492114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec401d070 con 0x7f6ed40731f0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.520+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed419e780 con 0x7f6ed40731f0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.521+0000 7f6ed17fa700 1 -- 172.21.15.5:0/188492114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec4022470 con 0x7f6ed40731f0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.521+0000 7f6ed17fa700 1 -- 172.21.15.5:0/188492114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4017650 con 0x7f6ed40731f0 2023-09-23T04:47:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.521+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed41a6020 con 0x7f6ed40731f0 2023-09-23T04:47:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.524+0000 7f6ed17fa700 1 -- 172.21.15.5:0/188492114 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7f6ec400fad0 con 0x7f6ed40731f0 2023-09-23T04:47:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.525+0000 7f6eede1fb80 1 Processor -- start 2023-09-23T04:47:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 -- start start 2023-09-23T04:47:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed17fa700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 0x7f6ebc0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555daea5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dae01460 con 0x5555daea3bd0 2023-09-23T04:47:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dadf2c20 con 0x5555dae98d20 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dadf0540 con 0x5555dae98400 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed17fa700 1 -- 172.21.15.5:0/188492114 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f6ec40cee50 con 0x7f6ed40731f0 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53428/0 (socket says 172.21.15.5:53428) 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.526+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44628/0 (socket says 172.21.15.5:44628) 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 -- 172.21.15.5:0/786636213 learned_addr learned my addr 172.21.15.5:0/786636213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed899c700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 0x7f6ebc0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed899c700 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555daea5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 -- 172.21.15.5:0/786636213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 msgr2=0x5555dae987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 -- 172.21.15.5:0/786636213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 msgr2=0x5555daea5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555daea5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 -- 172.21.15.5:0/786636213 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dae4e8d0 con 0x5555dae98d20 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bd30 tx=0x7f6ec80106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee407af85d947bdb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6ed899c700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 0x7f6ebc0a3cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009ec0 tx=0x7f6ecc009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.527+0000 7f6eba7fc700 1 -- 172.21.15.5:0/786636213 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8010e10 con 0x5555dae98d20 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eba7fc700 1 -- 172.21.15.5:0/786636213 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec8015070 con 0x5555dae98d20 2023-09-23T04:47:47.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eba7fc700 1 -- 172.21.15.5:0/786636213 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8003e60 con 0x5555dae98d20 2023-09-23T04:47:47.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 -- 172.21.15.5:0/786636213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 msgr2=0x5555daea3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bd30 tx=0x7f6ec80106b0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 -- 172.21.15.5:0/786636213 shutdown_connections 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555daea5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/786636213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daea3690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.528+0000 7f6eede1fb80 1 -- 172.21.15.5:0/786636213 >> 172.21.15.5:0/786636213 conn(0x5555dae0d260 msgr2=0x5555dae94790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.529+0000 7f6eede1fb80 1 -- 172.21.15.5:0/786636213 shutdown_connections 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.529+0000 7f6eede1fb80 1 -- 172.21.15.5:0/786636213 wait complete. 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.529+0000 7f6eede1fb80 1 -- start start 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.529+0000 7f6eede1fb80 1 Processor -- start 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98d20 0x5555daea08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555daea3bd0 0x5555daea0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dadebe30 con 0x5555dae98400 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dad88710 con 0x5555daea3bd0 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dad883b0 con 0x5555dae98d20 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6eede1fb80 10 client.0 _ll_register_callbacks cb 0x5555dabe5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55320/0 (socket says 172.21.15.5:55320) 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6ed3fff700 1 -- 172.21.15.5:0/4095959598 learned_addr learned my addr 172.21.15.5:0/4095959598 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.530+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98d20 0x5555daea08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed899c700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555daea3bd0 0x5555daea0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 -- 172.21.15.5:0/4095959598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98d20 msgr2=0x5555daea08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98d20 0x5555daea08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 -- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555daea3bd0 msgr2=0x5555daea0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555daea3bd0 0x5555daea0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 -- 172.21.15.5:0/4095959598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed41a6020 con 0x5555dae98400 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4097cc0 tx=0x7f6ec409b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102c6e3b6940a1f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6eb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec401b2a0 con 0x5555dae98400 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.531+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f6ec409bec0 con 0x5555dae98400 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555dae8c250 con 0x5555dae98400 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ec401b0c0 con 0x5555dae98400 2023-09-23T04:47:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 10 client.16026 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f6ec800b960 con 0x5555dae98400 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555dae8c250 con 0x5555dae98400 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6ec409ba20 con 0x5555dae98400 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 20 client.16026 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daebbac0 0x5555daebdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 4 client.16026 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.532+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555dade4e10 con 0x5555daebbac0 2023-09-23T04:47:47.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.533+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daebbac0 0x5555daebdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.533+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daebbac0 0x5555daebdeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3409a963fcb2c99 server_cookie=cac4782326480398 in_seq=0 out_seq=0 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.533+0000 7f6eb37fe700 10 client.16026 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eb37fe700 1 -- 172.21.15.5:0/4095959598 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555dade4e10 con 0x5555daebbac0 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eb37fe700 10 client.16026 handle_command_reply: tid=0 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 client.16026 shutdown 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 msgr2=0x5555daea3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4097cc0 tx=0x7f6ec409b610 comp rx=0 tx=0).stop 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 shutdown_connections 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daebbac0 0x5555daebdeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555daea3bd0 0x5555daea0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daea3650 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eb37fe700 0 client.16026 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/4095959598 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98d20 0x5555daea08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 >> 172.21.15.5:0/4095959598 conn(0x5555dae0d260 msgr2=0x5555dae94320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 shutdown_connections 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.534+0000 7f6eede1fb80 1 -- 172.21.15.5:0/4095959598 wait complete. 2023-09-23T04:47:47.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.535+0000 7f6eede1fb80 20 client.16026 trim_cache size 0 max 16384 2023-09-23T04:47:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 Processor -- start 2023-09-23T04:47:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 -- start start 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555dae940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dac6c920 con 0x5555daea3bd0 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dae90d60 con 0x5555dae98d20 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dae99620 con 0x5555dae98400 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6ed899c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555dae940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53442/0 (socket says 172.21.15.5:53442) 2023-09-23T04:47:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.550+0000 7f6ed37fe700 1 -- 172.21.15.5:0/2104571546 learned_addr learned my addr 172.21.15.5:0/2104571546 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 -- 172.21.15.5:0/2104571546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 msgr2=0x5555dae9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 -- 172.21.15.5:0/2104571546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 msgr2=0x5555dae940b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555dae940b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 -- 172.21.15.5:0/2104571546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dae90480 con 0x5555dae98d20 2023-09-23T04:47:47.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005d90 tx=0x7f6ec8005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e868fbcee7b354 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ebaffd700 1 -- 172.21.15.5:0/2104571546 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8020070 con 0x5555dae98d20 2023-09-23T04:47:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ebaffd700 1 -- 172.21.15.5:0/2104571546 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec8021040 con 0x5555dae98d20 2023-09-23T04:47:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.551+0000 7f6ebaffd700 1 -- 172.21.15.5:0/2104571546 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8003da0 con 0x5555dae98d20 2023-09-23T04:47:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- 172.21.15.5:0/2104571546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 msgr2=0x5555daeb2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005d90 tx=0x7f6ec8005d00 comp rx=0 tx=0).stop 2023-09-23T04:47:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- 172.21.15.5:0/2104571546 shutdown_connections 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555daea3bd0 0x5555dae940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555dae98400 0x5555dae9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/2104571546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb2f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- 172.21.15.5:0/2104571546 >> 172.21.15.5:0/2104571546 conn(0x5555dae0d260 msgr2=0x5555dae939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- 172.21.15.5:0/2104571546 shutdown_connections 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- 172.21.15.5:0/2104571546 wait complete. 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.552+0000 7f6eede1fb80 1 -- start start 2023-09-23T04:47:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 Processor -- start 2023-09-23T04:47:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daeb1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555daea3bd0 0x5555daf0e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555daf0ee30 con 0x5555dae98400 2023-09-23T04:47:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555daf0efa0 con 0x5555dae98d20 2023-09-23T04:47:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555daf0f160 con 0x5555daea3bd0 2023-09-23T04:47:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6eede1fb80 10 client.0 _ll_register_callbacks cb 0x7f6ec4097b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6ed3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daeb1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6ed37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53456/0 (socket says 172.21.15.5:53456) 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6ed37fe700 1 -- 172.21.15.5:0/1640139267 learned_addr learned my addr 172.21.15.5:0/1640139267 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.553+0000 7f6ed899c700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555daea3bd0 0x5555daf0e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 -- 172.21.15.5:0/1640139267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555daea3bd0 msgr2=0x5555daf0e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555daea3bd0 0x5555daf0e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 -- 172.21.15.5:0/1640139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 msgr2=0x5555daeb1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daeb1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 -- 172.21.15.5:0/1640139267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555dae4e8d0 con 0x5555dae98d20 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daeb1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6ed37fe700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b7e0 tx=0x7f6ec8010de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c0cd7b94e3fa46 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6eb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8020030 con 0x5555dae98d20 2023-09-23T04:47:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f6ec8017070 con 0x5555dae98d20 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.554+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ec8015070 con 0x5555dae98d20 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ec8028e00 con 0x5555dae98d20 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 10 client.26015 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5555dac16de0 con 0x5555dae98d20 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec8015070 con 0x5555dae98d20 2023-09-23T04:47:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6ec80050e0 con 0x5555dae98d20 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 10 client.26015 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 10 client.26015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 20 client.26015 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daf10490 0x5555daf128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 4 client.26015 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555dade4b30 con 0x5555daf10490 2023-09-23T04:47:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.555+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daf10490 0x5555daf128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6ed3fff700 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daf10490 0x5555daf128b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9ce720eda0e77b6 server_cookie=df130ab3b9fea957 in_seq=0 out_seq=0 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eb37fe700 10 client.26015 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eb37fe700 1 -- 172.21.15.5:0/1640139267 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555dade4b30 con 0x5555daf10490 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eb37fe700 10 client.26015 handle_command_reply: tid=0 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 client.26015 shutdown 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 msgr2=0x5555daeb1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b7e0 tx=0x7f6ec8010de0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 shutdown_connections 2023-09-23T04:47:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5555daf10490 0x5555daf128b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5555daea3bd0 0x5555daf0e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555dae98400 0x5555daeb1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 --2- 172.21.15.5:0/1640139267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5555dae98d20 0x5555daeb1640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eb37fe700 0 client.26015 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:47:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.556+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 >> 172.21.15.5:0/1640139267 conn(0x5555dae0d260 msgr2=0x5555dae94230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 shutdown_connections 2023-09-23T04:47:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eede1fb80 1 -- 172.21.15.5:0/1640139267 wait complete. 2023-09-23T04:47:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eede1fb80 20 client.26015 trim_cache size 0 max 16384 2023-09-23T04:47:47.572 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 msgr2=0x7f6ebc0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 0x7f6ebc0a3cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009ec0 tx=0x7f6ecc009920 comp rx=0 tx=0).stop 2023-09-23T04:47:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 msgr2=0x7f6ed41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.557+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005910 tx=0x7f6ec4004730 comp rx=0 tx=0).stop 2023-09-23T04:47:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 shutdown_connections 2023-09-23T04:47:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6ebc0a1830 0x7f6ebc0a3cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ed410d6d0 0x7f6ed419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed40731f0 0x7f6ed41991f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 --2- 172.21.15.5:0/188492114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6ed4073b10 0x7f6ed4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 >> 172.21.15.5:0/188492114 conn(0x7f6ed40fc920 msgr2=0x7f6ed410f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 shutdown_connections 2023-09-23T04:47:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.558+0000 7f6eda3ff700 1 -- 172.21.15.5:0/188492114 wait complete. 2023-09-23T04:47:47.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 -- 172.21.15.5:0/3897606888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 msgr2=0x7fbbf4073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/3897606888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4073630 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009990 tx=0x7fbbe4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 -- 172.21.15.5:0/3897606888 shutdown_connections 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/3897606888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 0x7fbbf410fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/3897606888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4073630 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/3897606888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 0x7fbbf410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.815+0000 7fbbf9dff700 1 -- 172.21.15.5:0/3897606888 >> 172.21.15.5:0/3897606888 conn(0x7fbbf40fc990 msgr2=0x7fbbf40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.816+0000 7fbbf9dff700 1 -- 172.21.15.5:0/3897606888 shutdown_connections 2023-09-23T04:47:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.816+0000 7fbbf9dff700 1 -- 172.21.15.5:0/3897606888 wait complete. 2023-09-23T04:47:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 Processor -- start 2023-09-23T04:47:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 -- start start 2023-09-23T04:47:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 0x7fbbf4199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 0x7fbbf419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419e1f0 con 0x7fbbf4073250 2023-09-23T04:47:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419e360 con 0x7fbbf4073b70 2023-09-23T04:47:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.817+0000 7fbbf9dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf419e520 con 0x7fbbf410d6a0 2023-09-23T04:47:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55356/0 (socket says 172.21.15.5:55356) 2023-09-23T04:47:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 -- 172.21.15.5:0/2807052401 learned_addr learned my addr 172.21.15.5:0/2807052401 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 0x7fbbf4199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 0x7fbbf419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 -- 172.21.15.5:0/2807052401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 msgr2=0x7fbbf419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 0x7fbbf419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 -- 172.21.15.5:0/2807052401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 msgr2=0x7fbbf4199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 0x7fbbf4199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 -- 172.21.15.5:0/2807052401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40095f0 con 0x7fbbf4073250 2023-09-23T04:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.818+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4000c00 tx=0x7fbbe4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.819+0000 7fbbf0ff9700 1 -- 172.21.15.5:0/2807052401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401d070 con 0x7fbbf4073250 2023-09-23T04:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.819+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf419e7a0 con 0x7fbbf4073250 2023-09-23T04:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.819+0000 7fbbf0ff9700 1 -- 172.21.15.5:0/2807052401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe4022470 con 0x7fbbf4073250 2023-09-23T04:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.819+0000 7fbbf0ff9700 1 -- 172.21.15.5:0/2807052401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4017630 con 0x7fbbf4073250 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.819+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf419ea10 con 0x7fbbf4073250 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.823+0000 7fbbf0ff9700 1 -- 172.21.15.5:0/2807052401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7fbbe400fad0 con 0x7fbbf4073250 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.824+0000 7fbbf0ff9700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 0x7fbbdc0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.824+0000 7fbc0d814b80 1 Processor -- start 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.824+0000 7fbbf0ff9700 1 -- 172.21.15.5:0/2807052401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbbe40cee00 con 0x7fbbf4073250 2023-09-23T04:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.824+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 0x7fbbdc0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.824+0000 7fbc0d814b80 1 -- start start 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a6059817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981d20 0x55a60598c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6058ea460 con 0x55a605981d20 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6058dbc20 con 0x55a60598cbd0 2023-09-23T04:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6058d9540 con 0x55a605981400 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 0x7fbbdc0a3cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbf419a820 tx=0x7fbbe800b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981d20 0x55a60598c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a6059817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53494/0 (socket says 172.21.15.5:53494) 2023-09-23T04:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.825+0000 7fbbf3fff700 1 -- 172.21.15.5:0/3151666519 learned_addr learned my addr 172.21.15.5:0/3151666519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 -- 172.21.15.5:0/3151666519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 msgr2=0x55a6059817e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a6059817e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 -- 172.21.15.5:0/3151666519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981d20 msgr2=0x55a60598c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981d20 0x55a60598c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 -- 172.21.15.5:0/3151666519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6059378d0 con 0x55a60598cbd0 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009cb0 tx=0x7fbbec011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc51d0d2edbc159 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbd9ffb700 1 -- 172.21.15.5:0/3151666519 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec011d80 con 0x55a60598cbd0 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbd9ffb700 1 -- 172.21.15.5:0/3151666519 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec0044f0 con 0x55a60598cbd0 2023-09-23T04:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.826+0000 7fbbd9ffb700 1 -- 172.21.15.5:0/3151666519 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec019420 con 0x55a60598cbd0 2023-09-23T04:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 -- 172.21.15.5:0/3151666519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 msgr2=0x55a60598efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009cb0 tx=0x7fbbec011440 comp rx=0 tx=0).stop 2023-09-23T04:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 -- 172.21.15.5:0/3151666519 shutdown_connections 2023-09-23T04:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a60598cbd0 0x55a60598efc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a6059817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/3151666519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981d20 0x55a60598c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 -- 172.21.15.5:0/3151666519 >> 172.21.15.5:0/3151666519 conn(0x55a6058f6260 msgr2=0x55a60597d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 -- 172.21.15.5:0/3151666519 shutdown_connections 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.827+0000 7fbc0d814b80 1 -- 172.21.15.5:0/3151666519 wait complete. 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.828+0000 7fbc0d814b80 1 -- start start 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.828+0000 7fbc0d814b80 1 Processor -- start 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a60598c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a60598a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6058d4e30 con 0x55a60598cbd0 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a605871710 con 0x55a605981d20 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6058713b0 con 0x55a605981400 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbc0d814b80 10 client.0 _ll_register_callbacks cb 0x55a6056ce800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53496/0 (socket says 172.21.15.5:53496) 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2091125569 learned_addr learned my addr 172.21.15.5:0/2091125569 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a60598c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a60598a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 msgr2=0x55a60598c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a60598c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2091125569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 msgr2=0x55a60598a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a60598a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2091125569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec009960 con 0x55a605981d20 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.829+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a60598c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8004590 tx=0x7fbbe8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd36d66cc1979861 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800e040 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a605975250 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbbe80048a0 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe8015cf0 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbc0d814b80 10 client.26027 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a6056ffde0 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.830+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a605975250 con 0x55a605981d20 2023-09-23T04:47:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbbe8017bf0 con 0x55a605981d20 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 10 client.26027 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 20 client.26027 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a60599c6f0 0x55a60599eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 4 client.26027 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6058cde10 con 0x55a60599c6f0 2023-09-23T04:47:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a60599c6f0 0x55a60599eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a60599c6f0 0x55a60599eb10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ee9a79d80527a93 server_cookie=9b0684e925e9dc8 in_seq=0 out_seq=0 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.831+0000 7fbbd2ffd700 10 client.26027 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.832+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2091125569 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a6058cde10 con 0x55a60599c6f0 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.832+0000 7fbbd2ffd700 10 client.26027 handle_command_reply: tid=0 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.832+0000 7fbc0d814b80 1 client.26027 shutdown 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.832+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 msgr2=0x55a605989970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.832+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8004590 tx=0x7fbbe8004ad0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 shutdown_connections 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a60598a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a60598c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a60599c6f0 0x55a60599eb10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2091125569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 >> 172.21.15.5:0/2091125569 conn(0x55a6058f6260 msgr2=0x55a60597d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbbd2ffd700 0 client.26027 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 shutdown_connections 2023-09-23T04:47:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2091125569 wait complete. 2023-09-23T04:47:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.833+0000 7fbc0d814b80 20 client.26027 trim_cache size 0 max 16384 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.847+0000 7fbc0d814b80 1 Processor -- start 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.847+0000 7fbc0d814b80 1 -- start start 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 0x55a60598c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 0x55a605989690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a605755920 con 0x55a605981400 2023-09-23T04:47:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60597c580 con 0x55a605981d20 2023-09-23T04:47:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a605985800 con 0x55a60598cbd0 2023-09-23T04:47:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 0x55a60598c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 0x55a605989690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 0x55a60598c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55388/0 (socket says 172.21.15.5:55388) 2023-09-23T04:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf37fe700 1 -- 172.21.15.5:0/1889310451 learned_addr learned my addr 172.21.15.5:0/1889310451 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/1889310451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 msgr2=0x55a605989690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 0x55a605989690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/1889310451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 msgr2=0x55a60598c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 0x55a60598c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/1889310451 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6056e8c60 con 0x55a605981d20 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.848+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 0x55a605989690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.849+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000f80 tx=0x7fbbe800fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99598bc5266a5d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.849+0000 7fbbda7fc700 1 -- 172.21.15.5:0/1889310451 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800e040 con 0x55a605981d20 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.849+0000 7fbbda7fc700 1 -- 172.21.15.5:0/1889310451 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbe800f040 con 0x55a605981d20 2023-09-23T04:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.849+0000 7fbbda7fc700 1 -- 172.21.15.5:0/1889310451 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8005760 con 0x55a605981d20 2023-09-23T04:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- 172.21.15.5:0/1889310451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 msgr2=0x55a605989150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8000f80 tx=0x7fbbe800fc60 comp rx=0 tx=0).stop 2023-09-23T04:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- 172.21.15.5:0/1889310451 shutdown_connections 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a60598cbd0 0x55a605989690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a605981400 0x55a60598c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/1889310451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605989150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- 172.21.15.5:0/1889310451 >> 172.21.15.5:0/1889310451 conn(0x55a6058f6260 msgr2=0x55a605984c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- 172.21.15.5:0/1889310451 shutdown_connections 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- 172.21.15.5:0/1889310451 wait complete. 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.850+0000 7fbc0d814b80 1 -- start start 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 Processor -- start 2023-09-23T04:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a605991700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a6059ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a605992530 con 0x55a60598cbd0 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6059efa80 con 0x55a605981d20 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6059efbf0 con 0x55a605981400 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbc0d814b80 10 client.0 _ll_register_callbacks cb 0x55a6056ce800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbbf2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbbf2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53514/0 (socket says 172.21.15.5:53514) 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2788922446 learned_addr learned my addr 172.21.15.5:0/2788922446 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a605991700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.851+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a6059ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 msgr2=0x55a605991700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a605991700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2788922446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 msgr2=0x55a6059ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a6059ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 -- 172.21.15.5:0/2788922446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6059378d0 con 0x55a605981d20 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a605991700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf3fff700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a6059ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbf2ffd700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80060b0 tx=0x7fbbe8016d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9574150d9a56633f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbd2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800e050 con 0x55a605981d20 2023-09-23T04:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbec0044f0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbbe8004ce0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.852+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe8004ea0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 10 client.26033 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a6056ffde0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec0044f0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbbe80179d0 con 0x55a605981d20 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 10 client.26033 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 10 client.26033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 20 client.26033 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a6059f0fc0 0x55a6059f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 4 client.26033 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6058cdb30 con 0x55a6059f0fc0 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.853+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a6059f0fc0 0x55a6059f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbbf37fe700 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a6059f0fc0 0x55a6059f33e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cb3550210583fe9 server_cookie=3e6debf24b840eef in_seq=0 out_seq=0 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbbd2ffd700 10 client.26033 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbbd2ffd700 1 -- 172.21.15.5:0/2788922446 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6058cdb30 con 0x55a6059f0fc0 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbbd2ffd700 10 client.26033 handle_command_reply: tid=0 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 client.26033 shutdown 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 msgr2=0x55a605991c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80060b0 tx=0x7fbbe8016d30 comp rx=0 tx=0).stop 2023-09-23T04:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 shutdown_connections 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a60598cbd0 0x55a6059ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a605981400 0x55a605991700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55a6059f0fc0 0x55a6059f33e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 --2- 172.21.15.5:0/2788922446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a605981d20 0x55a605991c40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbbd2ffd700 0 client.26033 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 >> 172.21.15.5:0/2788922446 conn(0x55a6058f6260 msgr2=0x55a605983290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 shutdown_connections 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.854+0000 7fbc0d814b80 1 -- 172.21.15.5:0/2788922446 wait complete. 2023-09-23T04:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbc0d814b80 20 client.26033 trim_cache size 0 max 16384 2023-09-23T04:47:47.863 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 msgr2=0x7fbbdc0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 0x7fbbdc0a3cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbbf419a820 tx=0x7fbbe800b040 comp rx=0 tx=0).stop 2023-09-23T04:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 msgr2=0x7fbbf4199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4000c00 tx=0x7fbbe4005720 comp rx=0 tx=0).stop 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 shutdown_connections 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbbdc0a1830 0x7fbbdc0a3cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbf410d6a0 0x7fbbf419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbf4073250 0x7fbbf4199210 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 --2- 172.21.15.5:0/2807052401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbf4073b70 0x7fbbf4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.855+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 >> 172.21.15.5:0/2807052401 conn(0x7fbbf40fc990 msgr2=0x7fbbf410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.856+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 shutdown_connections 2023-09-23T04:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:47.856+0000 7fbbf9dff700 1 -- 172.21.15.5:0/2807052401 wait complete. 2023-09-23T04:47:47.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:47 smithi005 ceph-mon[97638]: pgmap v48: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 3.0 MiB/s wr, 80 op/s 2023-09-23T04:47:47.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:47:47.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 24950 6.67 MB/sec warmup 23 sec latency 12.389 ms 2023-09-23T04:47:47.993 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:47 smithi069 ceph-mon[133931]: pgmap v48: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 3.0 MiB/s wr, 80 op/s 2023-09-23T04:47:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 -- 172.21.15.5:0/2010942557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74101a90 msgr2=0x7fab74105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 --2- 172.21.15.5:0/2010942557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74101a90 0x7fab74105ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab68009990 tx=0x7fab68009ca0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 -- 172.21.15.5:0/2010942557 shutdown_connections 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 --2- 172.21.15.5:0/2010942557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab7410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 --2- 172.21.15.5:0/2010942557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab741014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 --2- 172.21.15.5:0/2010942557 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74101a90 0x7fab74105ad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 -- 172.21.15.5:0/2010942557 >> 172.21.15.5:0/2010942557 conn(0x7fab740fc980 msgr2=0x7fab740feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.099+0000 7fab78dff700 1 -- 172.21.15.5:0/2010942557 shutdown_connections 2023-09-23T04:47:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.100+0000 7fab78dff700 1 -- 172.21.15.5:0/2010942557 wait complete. 2023-09-23T04:47:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.101+0000 7fab78dff700 1 Processor -- start 2023-09-23T04:47:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.101+0000 7fab78dff700 1 -- start start 2023-09-23T04:47:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.101+0000 7fab78dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab74073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab78dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab78dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 0x7fab741a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab78dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741a5850 con 0x7fab741010e0 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab78dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741a59c0 con 0x7fab74106200 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab78dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741a5b80 con 0x7fab74073ba0 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab7259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab74073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab72d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 0x7fab741a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53526/0 (socket says 172.21.15.5:53526) 2023-09-23T04:47:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 -- 172.21.15.5:0/780844683 learned_addr learned my addr 172.21.15.5:0/780844683 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 -- 172.21.15.5:0/780844683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 msgr2=0x7fab741a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 0x7fab741a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 -- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 msgr2=0x7fab74073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab74073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 -- 172.21.15.5:0/780844683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680095f0 con 0x7fab74106200 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab72d9d700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 0x7fab741a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab7259c700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab74073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.102+0000 7fab71d9b700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab68000c00 tx=0x7fab68005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.103+0000 7fab637fe700 1 -- 172.21.15.5:0/780844683 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6801d070 con 0x7fab74106200 2023-09-23T04:47:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.103+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab741a5e00 con 0x7fab74106200 2023-09-23T04:47:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.103+0000 7fab637fe700 1 -- 172.21.15.5:0/780844683 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab68021410 con 0x7fab74106200 2023-09-23T04:47:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.103+0000 7fab637fe700 1 -- 172.21.15.5:0/780844683 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68017650 con 0x7fab74106200 2023-09-23T04:47:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.103+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab741a6220 con 0x7fab74106200 2023-09-23T04:47:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.106+0000 7fab637fe700 1 -- 172.21.15.5:0/780844683 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) v1 ==== 136424+0+0 (secure 0 0 0) 0x7fab68021580 con 0x7fab74106200 2023-09-23T04:47:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.108+0000 7fab637fe700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 0x7fab5c0a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.108+0000 7fab637fe700 1 -- 172.21.15.5:0/780844683 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fab680cec10 con 0x7fab74106200 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.108+0000 7fab7259c700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 0x7fab5c0a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.108+0000 7fab8c725b80 1 Processor -- start 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.108+0000 7fab8c725b80 1 -- start start 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab7259c700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 0x7fab5c0a3cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab64011490 tx=0x7fab64009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cbab460 con 0x55f53cc42400 2023-09-23T04:47:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cb9cc20 con 0x55f53cc42d20 2023-09-23T04:47:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab8c725b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cb9a540 con 0x55f53cc4dbd0 2023-09-23T04:47:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab7259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53528/0 (socket says 172.21.15.5:53528) 2023-09-23T04:47:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab71d9b700 1 -- 172.21.15.5:0/1260438034 learned_addr learned my addr 172.21.15.5:0/1260438034 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.109+0000 7fab72d9d700 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 -- 172.21.15.5:0/1260438034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 msgr2=0x55f53cc4ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 -- 172.21.15.5:0/1260438034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 msgr2=0x55f53cc427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 -- 172.21.15.5:0/1260438034 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f53cbf88d0 con 0x55f53cc42d20 2023-09-23T04:47:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab71d9b700 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab68097a80 tx=0x7fab6809b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89501a722eb25358 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab60ff9700 1 -- 172.21.15.5:0/1260438034 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6809aee0 con 0x55f53cc42d20 2023-09-23T04:47:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab60ff9700 1 -- 172.21.15.5:0/1260438034 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab6809bc80 con 0x55f53cc42d20 2023-09-23T04:47:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.110+0000 7fab60ff9700 1 -- 172.21.15.5:0/1260438034 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6809aa70 con 0x55f53cc42d20 2023-09-23T04:47:48.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- 172.21.15.5:0/1260438034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 msgr2=0x55f53cc4d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab68097a80 tx=0x7fab6809b350 comp rx=0 tx=0).stop 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- 172.21.15.5:0/1260438034 shutdown_connections 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 --2- 172.21.15.5:0/1260438034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4d690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- 172.21.15.5:0/1260438034 >> 172.21.15.5:0/1260438034 conn(0x55f53cbb7260 msgr2=0x55f53cc3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- 172.21.15.5:0/1260438034 shutdown_connections 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- 172.21.15.5:0/1260438034 wait complete. 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.111+0000 7fab8c725b80 1 -- start start 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 Processor -- start 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cb95e30 con 0x55f53cc4dbd0 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cb32710 con 0x55f53cc42d20 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cb323b0 con 0x55f53cc42400 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.112+0000 7fab8c725b80 10 client.0 _ll_register_callbacks cb 0x55f53c98f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44740/0 (socket says 172.21.15.5:44740) 2023-09-23T04:47:48.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 -- 172.21.15.5:0/2692541448 learned_addr learned my addr 172.21.15.5:0/2692541448 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab72d9d700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 -- 172.21.15.5:0/2692541448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 msgr2=0x55f53cc4e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 -- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 msgr2=0x55f53cc4ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 -- 172.21.15.5:0/2692541448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab741a6220 con 0x55f53cc42400 2023-09-23T04:47:48.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab72d9d700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab7259c700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab640039d0 tx=0x7fab640047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2341a920cab17c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.113+0000 7fab59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6401fa90 con 0x55f53cc42400 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f53cc36250 con 0x55f53cc42400 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fab6401fbf0 con 0x55f53cc42400 2023-09-23T04:47:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab64003e50 con 0x55f53cc42400 2023-09-23T04:47:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab8c725b80 10 client.26101 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f53c9c0de0 con 0x55f53cc42400 2023-09-23T04:47:48.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.114+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f53cc36250 con 0x55f53cc42400 2023-09-23T04:47:48.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fab6400e3b0 con 0x55f53cc42400 2023-09-23T04:47:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 10 client.26101 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 20 client.26101 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53cc5d370 0x55f53cc5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 4 client.26101 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f53cb8ee10 con 0x55f53cc5d370 2023-09-23T04:47:48.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53cc5d370 0x55f53cc5f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53cc5d370 0x55f53cc5f790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ee9b075c4f035c server_cookie=b990fb7654991f4a in_seq=0 out_seq=0 2023-09-23T04:47:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.115+0000 7fab59ffb700 10 client.26101 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab59ffb700 1 -- 172.21.15.5:0/2692541448 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f53cb8ee10 con 0x55f53cc5d370 2023-09-23T04:47:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab59ffb700 10 client.26101 handle_command_reply: tid=0 2023-09-23T04:47:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 client.26101 shutdown 2023-09-23T04:47:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 msgr2=0x55f53cc4dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab640039d0 tx=0x7fab640047c0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 shutdown_connections 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53cc5d370 0x55f53cc5f790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc42400 0x55f53cc4dfb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2692541448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab59ffb700 0 client.26101 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 >> 172.21.15.5:0/2692541448 conn(0x55f53cbb7260 msgr2=0x55f53cc3e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.116+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 shutdown_connections 2023-09-23T04:47:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.117+0000 7fab8c725b80 1 -- 172.21.15.5:0/2692541448 wait complete. 2023-09-23T04:47:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.117+0000 7fab8c725b80 20 client.26101 trim_cache size 0 max 16384 2023-09-23T04:47:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.131+0000 7fab8c725b80 1 Processor -- start 2023-09-23T04:47:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.131+0000 7fab8c725b80 1 -- start start 2023-09-23T04:47:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc51fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53ca16920 con 0x55f53cc42400 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cc52550 con 0x55f53cc42d20 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab8c725b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53cc48a50 con 0x55f53cc4dbd0 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53560/0 (socket says 172.21.15.5:53560) 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 -- 172.21.15.5:0/2207318470 learned_addr learned my addr 172.21.15.5:0/2207318470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab72d9d700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab7259c700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc51fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 -- 172.21.15.5:0/2207318470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 msgr2=0x55f53cc4c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 -- 172.21.15.5:0/2207318470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 msgr2=0x55f53cc51fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc51fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab71d9b700 1 -- 172.21.15.5:0/2207318470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f53c9a9c60 con 0x55f53cc42d20 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.132+0000 7fab7259c700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc51fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.133+0000 7fab71d9b700 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab680998d0 tx=0x7fab6809bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4108e6f8dac536a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.133+0000 7fab617fa700 1 -- 172.21.15.5:0/2207318470 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680b0040 con 0x55f53cc42d20 2023-09-23T04:47:48.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.133+0000 7fab617fa700 1 -- 172.21.15.5:0/2207318470 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab6809aee0 con 0x55f53cc42d20 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.133+0000 7fab617fa700 1 -- 172.21.15.5:0/2207318470 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6809aa70 con 0x55f53cc42d20 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- 172.21.15.5:0/2207318470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 msgr2=0x55f53cc4c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab680998d0 tx=0x7fab6809bdf0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- 172.21.15.5:0/2207318470 shutdown_connections 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cc4c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42400 0x55f53cc51fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 --2- 172.21.15.5:0/2207318470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42d20 0x55f53cc4c3a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- 172.21.15.5:0/2207318470 >> 172.21.15.5:0/2207318470 conn(0x55f53cbb7260 msgr2=0x55f53cc47f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- 172.21.15.5:0/2207318470 shutdown_connections 2023-09-23T04:47:48.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- 172.21.15.5:0/2207318470 wait complete. 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 -- start start 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.134+0000 7fab8c725b80 1 Processor -- start 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 0x55f53cca50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cca5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53ccb0340 con 0x55f53cc42d20 2023-09-23T04:47:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53ccb04b0 con 0x55f53cc42400 2023-09-23T04:47:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f53ccb0670 con 0x55f53cc4dbd0 2023-09-23T04:47:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab8c725b80 10 client.0 _ll_register_callbacks cb 0x7fab640045e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab71d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55434/0 (socket says 172.21.15.5:55434) 2023-09-23T04:47:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab71d9b700 1 -- 172.21.15.5:0/429898611 learned_addr learned my addr 172.21.15.5:0/429898611 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab72d9d700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cca5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.135+0000 7fab7259c700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 0x55f53cca50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 -- 172.21.15.5:0/429898611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 msgr2=0x55f53cca50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 0x55f53cca50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 -- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 msgr2=0x55f53cca5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 -- 172.21.15.5:0/429898611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f53cbf88d0 con 0x55f53cc4dbd0 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab71d9b700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab7259c700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 0x55f53cca50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab72d9d700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cca5bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55f53cc4d0a0 tx=0x7fab6c00e4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cebb2d238a530bb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01c070 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab6809bc80 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.136+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fab6c00ec80 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab6c00ee40 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 10 client.26107 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f53c9e7250 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab6809bc80 con 0x55f53cc4dbd0 2023-09-23T04:47:48.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fab6c0058d0 con 0x55f53cc4dbd0 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 20 client.26107 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53ccb1e90 0x55f53ccb42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 4 client.26107 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f53cb8eb30 con 0x55f53ccb1e90 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.137+0000 7fab7259c700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53ccb1e90 0x55f53ccb42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.138+0000 7fab7259c700 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53ccb1e90 0x55f53ccb42b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826096ac1d07e2ff server_cookie=35a3fd4eec076028 in_seq=0 out_seq=0 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.138+0000 7fab59ffb700 10 client.26107 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.138+0000 7fab59ffb700 1 -- 172.21.15.5:0/429898611 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f53cb8eb30 con 0x55f53ccb1e90 2023-09-23T04:47:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.138+0000 7fab59ffb700 10 client.26107 handle_command_reply: tid=0 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 client.26107 shutdown 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 msgr2=0x55f53cca5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cca5bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55f53cc4d0a0 tx=0x7fab6c00e4f0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 shutdown_connections 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f53ccb1e90 0x55f53ccb42b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f53cc4dbd0 0x55f53cca5bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab59ffb700 0 client.26107 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f53cc42400 0x55f53cca50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 --2- 172.21.15.5:0/429898611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f53cc42d20 0x55f53cca5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 >> 172.21.15.5:0/429898611 conn(0x55f53cbb7260 msgr2=0x55f53cc3d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 shutdown_connections 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 1 -- 172.21.15.5:0/429898611 wait complete. 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.139+0000 7fab8c725b80 20 client.26107 trim_cache size 0 max 16384 2023-09-23T04:47:48.160 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 msgr2=0x7fab5c0a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 0x7fab5c0a3cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab64011490 tx=0x7fab64009260 comp rx=0 tx=0).stop 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 msgr2=0x7fab74073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab68000c00 tx=0x7fab68005740 comp rx=0 tx=0).stop 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 shutdown_connections 2023-09-23T04:47:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab5c0a1830 0x7fab5c0a3cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fab74073ba0 0x7fab741a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab741010e0 0x7fab74073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 --2- 172.21.15.5:0/780844683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab74106200 0x7fab74073660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 >> 172.21.15.5:0/780844683 conn(0x7fab740fc980 msgr2=0x7fab740fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 shutdown_connections 2023-09-23T04:47:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:48.140+0000 7fab78dff700 1 -- 172.21.15.5:0/780844683 wait complete. 2023-09-23T04:47:48.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:47 smithi118 ceph-mon[134297]: pgmap v48: 129 pgs: 129 active+clean; 29 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 3.0 MiB/s wr, 80 op/s 2023-09-23T04:47:48.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25748 6.48 MB/sec warmup 24 sec latency 199.273 ms 2023-09-23T04:47:48.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:48 smithi069 ceph-mon[133931]: pgmap v49: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2023-09-23T04:47:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:48 smithi005 ceph-mon[97638]: pgmap v49: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2023-09-23T04:47:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:48 smithi118 ceph-mon[134297]: pgmap v49: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2023-09-23T04:47:49.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:47:49.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:47:49.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 -- 172.21.15.5:0/2880282281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 msgr2=0x7f42cc10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 --2- 172.21.15.5:0/2880282281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc10dd70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009f90 tx=0x7f42c400b330 comp rx=0 tx=0).stop 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 -- 172.21.15.5:0/2880282281 shutdown_connections 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 --2- 172.21.15.5:0/2880282281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc10dd70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 --2- 172.21.15.5:0/2880282281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc1010d0 0x7f42cc1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 --2- 172.21.15.5:0/2880282281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc101a80 0x7f42cc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 -- 172.21.15.5:0/2880282281 >> 172.21.15.5:0/2880282281 conn(0x7f42cc0fc990 msgr2=0x7f42cc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 -- 172.21.15.5:0/2880282281 shutdown_connections 2023-09-23T04:47:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.379+0000 7f42d1dff700 1 -- 172.21.15.5:0/2880282281 wait complete. 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.380+0000 7f42d1dff700 1 Processor -- start 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 -- start start 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc19dc80 0x7f42cc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19e700 con 0x7f42cc1061f0 2023-09-23T04:47:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19e870 con 0x7f42cc1010d0 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42d1dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc19ea30 con 0x7f42cc19dc80 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53586/0 (socket says 172.21.15.5:53586) 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3465116049 learned_addr learned my addr 172.21.15.5:0/3465116049 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42caffd700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.381+0000 7f42cbfff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc19dc80 0x7f42cc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3465116049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc19dc80 msgr2=0x7f42cc19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc19dc80 0x7f42cc19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 msgr2=0x7f42cc199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3465116049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00095f0 con 0x7f42cc1010d0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00e9a0 tx=0x7f42bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42c8ff9700 1 -- 172.21.15.5:0/3465116049 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc009790 con 0x7f42cc1010d0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c4009c40 con 0x7f42cc1010d0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42c8ff9700 1 -- 172.21.15.5:0/3465116049 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42bc00cbd0 con 0x7f42cc1010d0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42c8ff9700 1 -- 172.21.15.5:0/3465116049 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc018680 con 0x7f42cc1010d0 2023-09-23T04:47:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.382+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1a6290 con 0x7f42cc1010d0 2023-09-23T04:47:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.387+0000 7f42c8ff9700 1 -- 172.21.15.5:0/3465116049 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f42bc00cd40 con 0x7f42cc1010d0 2023-09-23T04:47:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.388+0000 7f42e571bb80 1 Processor -- start 2023-09-23T04:47:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.388+0000 7f42e571bb80 1 -- start start 2023-09-23T04:47:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.388+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.388+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d6597b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42e571bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d658d64e0 con 0x562d65978c50 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42e571bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d658c7ca0 con 0x562d6596dda0 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42e571bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d658c55c0 con 0x562d6596d480 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44776/0 (socket says 172.21.15.5:44776) 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 -- 172.21.15.5:0/16887412 learned_addr learned my addr 172.21.15.5:0/16887412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42caffd700 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d6597b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42c8ff9700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 0x7f42b40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 -- 172.21.15.5:0/16887412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 msgr2=0x562d65978710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42c8ff9700 1 -- 172.21.15.5:0/3465116049 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f42bc014070 con 0x7f42cc1010d0 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 -- 172.21.15.5:0/16887412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 msgr2=0x562d6597b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d6597b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 -- 172.21.15.5:0/16887412 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d65923950 con 0x562d6596d480 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.389+0000 7f42cbfff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 0x7f42b40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.390+0000 7f42cbfff700 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009420 tx=0x7f42c4009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4da3de274c391ac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.390+0000 7f42b1ffb700 1 -- 172.21.15.5:0/16887412 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400e050 con 0x562d6596d480 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.390+0000 7f42cbfff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 0x7f42b40a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42c4029000 tx=0x7f42c4026000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.390+0000 7f42b1ffb700 1 -- 172.21.15.5:0/16887412 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c400be50 con 0x562d6596d480 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.390+0000 7f42b1ffb700 1 -- 172.21.15.5:0/16887412 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c401d920 con 0x562d6596d480 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 -- 172.21.15.5:0/16887412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 msgr2=0x562d6596d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009420 tx=0x7f42c4009450 comp rx=0 tx=0).stop 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 -- 172.21.15.5:0/16887412 shutdown_connections 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d6597b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596d480 0x562d6596d860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 --2- 172.21.15.5:0/16887412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 -- 172.21.15.5:0/16887412 >> 172.21.15.5:0/16887412 conn(0x562d658e22e0 msgr2=0x562d65969810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 -- 172.21.15.5:0/16887412 shutdown_connections 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.391+0000 7f42e571bb80 1 -- 172.21.15.5:0/16887412 wait complete. 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.392+0000 7f42e571bb80 1 -- start start 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.392+0000 7f42e571bb80 1 Processor -- start 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 0x562d659758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d65976030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d658c0eb0 con 0x562d65978c50 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6585d790 con 0x562d6596d480 2023-09-23T04:47:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6585d430 con 0x562d6596dda0 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42e571bb80 10 client.0 _ll_register_callbacks cb 0x562d656ba880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53610/0 (socket says 172.21.15.5:53610) 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3976427482 learned_addr learned my addr 172.21.15.5:0/3976427482 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cbfff700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d65976030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42caffd700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 0x562d659758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3976427482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 msgr2=0x562d659758f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 0x562d659758f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 msgr2=0x562d65976030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d65976030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42cb7fe700 1 -- 172.21.15.5:0/3976427482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc10e5e0 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.393+0000 7f42caffd700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 0x562d659758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42bc096ee0 tx=0x7f42bc09a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79102c89c5794251 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc09a470 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f42bc09e5c0 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d659612d0 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42bc0a2800 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 10 client.26063 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562d656ebe60 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d659612d0 con 0x562d6596d480 2023-09-23T04:47:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f42bc09e0f0 con 0x562d6596d480 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 10 client.26063 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.394+0000 7f42e571bb80 20 client.26063 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d65988880 0x562d6598aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42e571bb80 4 client.26063 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d658b9e90 con 0x562d65988880 2023-09-23T04:47:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42caffd700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d65988880 0x562d6598aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42aaffd700 10 client.26063 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.395+0000 7f42caffd700 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d65988880 0x562d6598aca0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84aff3026476658 server_cookie=e2eb6fa892353b1b in_seq=0 out_seq=0 2023-09-23T04:47:49.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42aaffd700 1 -- 172.21.15.5:0/3976427482 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562d658b9e90 con 0x562d65988880 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42aaffd700 10 client.26063 handle_command_reply: tid=0 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 client.26063 shutdown 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 msgr2=0x562d659786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42bc096ee0 tx=0x7f42bc09a030 comp rx=0 tx=0).stop 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 shutdown_connections 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d65988880 0x562d6598aca0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d65978c50 0x562d65976030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596d480 0x562d659786e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 --2- 172.21.15.5:0/3976427482 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d6596dda0 0x562d659758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42aaffd700 0 client.26063 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 >> 172.21.15.5:0/3976427482 conn(0x562d658e22e0 msgr2=0x562d659693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 shutdown_connections 2023-09-23T04:47:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.396+0000 7f42e571bb80 1 -- 172.21.15.5:0/3976427482 wait complete. 2023-09-23T04:47:49.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.397+0000 7f42e571bb80 20 client.26063 trim_cache size 0 max 16384 2023-09-23T04:47:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.414+0000 7f42e571bb80 1 Processor -- start 2023-09-23T04:47:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.414+0000 7f42e571bb80 1 -- start start 2023-09-23T04:47:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d65972ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d657419a0 con 0x562d6596d480 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6596a660 con 0x562d6596dda0 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42e571bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65985d90 con 0x562d65978c50 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55462/0 (socket says 172.21.15.5:55462) 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 -- 172.21.15.5:0/2346835317 learned_addr learned my addr 172.21.15.5:0/2346835317 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42caffd700 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cbfff700 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d65972ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 -- 172.21.15.5:0/2346835317 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 msgr2=0x562d65972ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d65972ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 -- 172.21.15.5:0/2346835317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 msgr2=0x562d65978210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.415+0000 7f42cb7fe700 1 -- 172.21.15.5:0/2346835317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6598f8d0 con 0x562d6596d480 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42bc096970 tx=0x7f42bc099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7b2236b21b1e5a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42b27fc700 1 -- 172.21.15.5:0/2346835317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc09a470 con 0x562d6596d480 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42b27fc700 1 -- 172.21.15.5:0/2346835317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42bc0a2800 con 0x562d6596d480 2023-09-23T04:47:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42b27fc700 1 -- 172.21.15.5:0/2346835317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc099bc0 con 0x562d6596d480 2023-09-23T04:47:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 -- 172.21.15.5:0/2346835317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 msgr2=0x562d659758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42bc096970 tx=0x7f42bc099dd0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 -- 172.21.15.5:0/2346835317 shutdown_connections 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d65972ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d659758a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2346835317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d65978210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.416+0000 7f42e571bb80 1 -- 172.21.15.5:0/2346835317 >> 172.21.15.5:0/2346835317 conn(0x562d658e22e0 msgr2=0x562d65975190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.417+0000 7f42e571bb80 1 -- 172.21.15.5:0/2346835317 shutdown_connections 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.417+0000 7f42e571bb80 1 -- 172.21.15.5:0/2346835317 wait complete. 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.417+0000 7f42e571bb80 1 -- start start 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.417+0000 7f42e571bb80 1 Processor -- start 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d6597ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d6597e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d659dc6c0 con 0x562d6596d480 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d659dc800 con 0x562d6596dda0 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d659dc940 con 0x562d65978c50 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42e571bb80 10 client.0 _ll_register_callbacks cb 0x7f42bc0d3220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d6597ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44816/0 (socket says 172.21.15.5:44816) 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 -- 172.21.15.5:0/2516271106 learned_addr learned my addr 172.21.15.5:0/2516271106 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42caffd700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d6597e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 -- 172.21.15.5:0/2516271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 msgr2=0x562d6597e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d6597e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 -- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 msgr2=0x562d6597ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d6597ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.418+0000 7f42cbfff700 1 -- 172.21.15.5:0/2516271106 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d656ebe60 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42cbfff700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b6c0 tx=0x7f42c4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92213f7ec11cb926 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4007dc0 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42c400be50 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f42c400b6f0 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42c400e020 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42e571bb80 10 client.26125 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.419+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562d656d4ce0 con 0x562d65978c50 2023-09-23T04:47:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c400be50 con 0x562d65978c50 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f42c4029440 con 0x562d65978c50 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 10 client.26125 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 20 client.26125 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d659dde50 0x562d659e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 4 client.26125 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562d658b9bb0 con 0x562d659dde50 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.420+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d659dde50 0x562d659e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.421+0000 7f42cb7fe700 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d659dde50 0x562d659e0270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3199f72620ec2a server_cookie=4972c545f6212a2c in_seq=0 out_seq=0 2023-09-23T04:47:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.421+0000 7f42aaffd700 10 client.26125 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42aaffd700 1 -- 172.21.15.5:0/2516271106 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562d658b9bb0 con 0x562d659dde50 2023-09-23T04:47:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42aaffd700 10 client.26125 handle_command_reply: tid=0 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 client.26125 shutdown 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 msgr2=0x562d6597e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b6c0 tx=0x7f42c4009920 comp rx=0 tx=0).stop 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 shutdown_connections 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562d659dde50 0x562d659e0270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562d65978c50 0x562d6597e840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42aaffd700 0 client.26125 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6596d480 0x562d6597ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 --2- 172.21.15.5:0/2516271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6596dda0 0x562d6597e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.422+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 >> 172.21.15.5:0/2516271106 conn(0x562d658e22e0 msgr2=0x562d6596fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.423+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 shutdown_connections 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.423+0000 7f42e571bb80 1 -- 172.21.15.5:0/2516271106 wait complete. 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.423+0000 7f42e571bb80 20 client.26125 trim_cache size 0 max 16384 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c7304141-2dc2-46c9-90c4-acac729c3188", 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:47:49.428 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 msgr2=0x7f42b40a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 0x7f42b40a3c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42c4029000 tx=0x7f42c4026000 comp rx=0 tx=0).stop 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 msgr2=0x7f42cc1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00e9a0 tx=0x7f42bc00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 shutdown_connections 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f42b40a1790 0x7f42b40a3c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f42cc19dc80 0x7f42cc19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42cc1010d0 0x7f42cc1992e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 --2- 172.21.15.5:0/3465116049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42cc1061f0 0x7f42cc199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 >> 172.21.15.5:0/3465116049 conn(0x7f42cc0fc990 msgr2=0x7f42cc0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 shutdown_connections 2023-09-23T04:47:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.424+0000 7f42d1dff700 1 -- 172.21.15.5:0/3465116049 wait complete. 2023-09-23T04:47:49.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:47:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/3251266433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 msgr2=0x7ff3f810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/3251266433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f810fb10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b0a0 tx=0x7ff3f000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/3251266433 shutdown_connections 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/3251266433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f810fb10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/3251266433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073250 0x7ff3f8073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/3251266433 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073b70 0x7ff3f810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/3251266433 >> 172.21.15.5:0/3251266433 conn(0x7ff3f80fc990 msgr2=0x7ff3f80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/3251266433 shutdown_connections 2023-09-23T04:47:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.664+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/3251266433 wait complete. 2023-09-23T04:47:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.666+0000 7ff3fd7ff700 1 Processor -- start 2023-09-23T04:47:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.666+0000 7ff3fd7ff700 1 -- start start 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.666+0000 7ff3fd7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 0x7ff3f8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3fd7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 0x7ff3f81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3fd7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3fd7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f819e250 con 0x7ff3f8073b70 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3fd7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f819e3c0 con 0x7ff3f810d6a0 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3fd7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f819e580 con 0x7ff3f8073250 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 0x7ff3f8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 0x7ff3f81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 0x7ff3f8199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:44836/0 (socket says 172.21.15.5:44836) 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/2871621304 learned_addr learned my addr 172.21.15.5:0/2871621304 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 -- 172.21.15.5:0/2871621304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 msgr2=0x7ff3f8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 0x7ff3f8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 -- 172.21.15.5:0/2871621304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 msgr2=0x7ff3f819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f67fc700 1 -- 172.21.15.5:0/2871621304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0009d00 con 0x7ff3f8073b70 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.667+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 0x7ff3f81997b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00e9f0 tx=0x7ff3ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3dffff700 1 -- 172.21.15.5:0/2871621304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec00c9c0 con 0x7ff3f8073b70 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f819e860 con 0x7ff3f8073b70 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3dffff700 1 -- 172.21.15.5:0/2871621304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ec010430 con 0x7ff3f8073b70 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3dffff700 1 -- 172.21.15.5:0/2871621304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec018680 con 0x7ff3f8073b70 2023-09-23T04:47:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.668+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f81a60a0 con 0x7ff3f8073b70 2023-09-23T04:47:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.671+0000 7ff3dffff700 1 -- 172.21.15.5:0/2871621304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff3ec010860 con 0x7ff3f8073b70 2023-09-23T04:47:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.672+0000 7ff411217b80 1 Processor -- start 2023-09-23T04:47:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 -- start start 2023-09-23T04:47:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff3dffff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 0x7ff3e00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f22277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f2232690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2232bd0 0x5562f2234fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2190460 con 0x5562f2232bd0 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2181c20 con 0x5562f2227400 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff411217b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f217f540 con 0x5562f2227d20 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff3dffff700 1 -- 172.21.15.5:0/2871621304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff3ec014070 con 0x7ff3f8073b70 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 0x7ff3e00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.673+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f22277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f22277e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53654/0 (socket says 172.21.15.5:53654) 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/1040153877 learned_addr learned my addr 172.21.15.5:0/1040153877 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f2232690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2232bd0 0x5562f2234fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 -- 172.21.15.5:0/1040153877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 msgr2=0x5562f22277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f22277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 -- 172.21.15.5:0/1040153877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2232bd0 msgr2=0x5562f2234fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2232bd0 0x5562f2234fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 -- 172.21.15.5:0/1040153877 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f21dd8d0 con 0x5562f2227d20 2023-09-23T04:47:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 0x7ff3e00a3bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b780 tx=0x7ff3f00094d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.674+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f2232690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0cab20 tx=0x7ff3ec099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c465cde0ddcc06a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff3dd7fa700 1 -- 172.21.15.5:0/1040153877 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec09aa40 con 0x5562f2227d20 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff3dd7fa700 1 -- 172.21.15.5:0/1040153877 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ec09a340 con 0x5562f2227d20 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff3dd7fa700 1 -- 172.21.15.5:0/1040153877 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0cada0 con 0x5562f2227d20 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 -- 172.21.15.5:0/1040153877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 msgr2=0x5562f2232690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f2232690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0cab20 tx=0x7ff3ec099e80 comp rx=0 tx=0).stop 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 -- 172.21.15.5:0/1040153877 shutdown_connections 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2232bd0 0x5562f2234fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f22277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 --2- 172.21.15.5:0/1040153877 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f2232690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 -- 172.21.15.5:0/1040153877 >> 172.21.15.5:0/1040153877 conn(0x5562f219c260 msgr2=0x5562f2223790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.675+0000 7ff411217b80 1 -- 172.21.15.5:0/1040153877 shutdown_connections 2023-09-23T04:47:49.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.676+0000 7ff411217b80 1 -- 172.21.15.5:0/1040153877 wait complete. 2023-09-23T04:47:49.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.676+0000 7ff411217b80 1 -- start start 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.676+0000 7ff411217b80 1 Processor -- start 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f222f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 0x5562f222ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f217ae30 con 0x5562f2227400 2023-09-23T04:47:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2117710 con 0x5562f2232bd0 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f21173b0 con 0x5562f2227d20 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff411217b80 10 client.0 _ll_register_callbacks cb 0x5562f1f74800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f222f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55492/0 (socket says 172.21.15.5:55492) 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/3355550712 learned_addr learned my addr 172.21.15.5:0/3355550712 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 0x5562f222ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 -- 172.21.15.5:0/3355550712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 msgr2=0x5562f222ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 0x5562f222ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 -- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 msgr2=0x5562f2232660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f67fc700 1 -- 172.21.15.5:0/3355550712 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f81a60a0 con 0x5562f2227d20 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.677+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 0x5562f222ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f222f870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0a2890 tx=0x7ff3ec09aac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f51365b5c7e96f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec09a5a0 con 0x5562f2227d20 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562f221b250 con 0x5562f2227d20 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff3ec09a700 con 0x5562f2227d20 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3ec099830 con 0x5562f2227d20 2023-09-23T04:47:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff411217b80 10 client.26137 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5562f1fa5de0 con 0x5562f2227d20 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ec0b6070 con 0x5562f2227d20 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.678+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff3ec0a2240 con 0x5562f2227d20 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 10 client.26137 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 20 client.26137 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2242270 0x5562f2244690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 4 client.26137 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562f2173e10 con 0x5562f2242270 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2242270 0x5562f2244690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2242270 0x5562f2244690 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376765e36d079a64 server_cookie=5ccd13404422f7ef in_seq=0 out_seq=0 2023-09-23T04:47:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.679+0000 7ff3daff5700 10 client.26137 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff3daff5700 1 -- 172.21.15.5:0/3355550712 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562f2173e10 con 0x5562f2242270 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff3daff5700 10 client.26137 handle_command_reply: tid=0 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 client.26137 shutdown 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 msgr2=0x5562f222f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f222f870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0a2890 tx=0x7ff3ec09aac0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 shutdown_connections 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2242270 0x5562f2244690 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2232bd0 0x5562f222ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f2232660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff3daff5700 0 client.26137 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 --2- 172.21.15.5:0/3355550712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2227d20 0x5562f222f870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 >> 172.21.15.5:0/3355550712 conn(0x5562f219c260 msgr2=0x5562f2223320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 shutdown_connections 2023-09-23T04:47:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.680+0000 7ff411217b80 1 -- 172.21.15.5:0/3355550712 wait complete. 2023-09-23T04:47:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.681+0000 7ff411217b80 20 client.26137 trim_cache size 0 max 16384 2023-09-23T04:47:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 Processor -- start 2023-09-23T04:47:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 -- start start 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f222f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f223e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f1ffb920 con 0x5562f2227400 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2244d10 con 0x5562f2227d20 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff411217b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f221f570 con 0x5562f2232bd0 2023-09-23T04:47:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff3f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f222f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53662/0 (socket says 172.21.15.5:53662) 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff3f67fc700 1 -- 172.21.15.5:0/3421576889 learned_addr learned my addr 172.21.15.5:0/3421576889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.698+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f223e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/3421576889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 msgr2=0x5562f223e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f223e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/3421576889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 msgr2=0x5562f2239660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 -- 172.21.15.5:0/3421576889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f1f8ec60 con 0x5562f2227400 2023-09-23T04:47:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f223e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:47:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f222f830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5562f2244c90 tx=0x7ff3e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8308c5555d85db85 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3ddffb700 1 -- 172.21.15.5:0/3421576889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8015050 con 0x5562f2227400 2023-09-23T04:47:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3ddffb700 1 -- 172.21.15.5:0/3421576889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3e8020030 con 0x5562f2227400 2023-09-23T04:47:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.699+0000 7ff3ddffb700 1 -- 172.21.15.5:0/3421576889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8005380 con 0x5562f2227400 2023-09-23T04:47:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 -- 172.21.15.5:0/3421576889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 msgr2=0x5562f222f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f222f830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5562f2244c90 tx=0x7ff3e8009260 comp rx=0 tx=0).stop 2023-09-23T04:47:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 -- 172.21.15.5:0/3421576889 shutdown_connections 2023-09-23T04:47:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f223e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227400 0x5562f222f830 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 --2- 172.21.15.5:0/3421576889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227d20 0x5562f2239660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 -- 172.21.15.5:0/3421576889 >> 172.21.15.5:0/3421576889 conn(0x5562f219c260 msgr2=0x5562f2248ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 -- 172.21.15.5:0/3421576889 shutdown_connections 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.700+0000 7ff411217b80 1 -- 172.21.15.5:0/3421576889 wait complete. 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 -- start start 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 Processor -- start 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f2237300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f22950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f22955e0 con 0x5562f2227d20 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2295720 con 0x5562f2227400 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f2295860 con 0x5562f2232bd0 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff411217b80 10 client.0 _ll_register_callbacks cb 0x7ff3e8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.701+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55512/0 (socket says 172.21.15.5:55512) 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 -- 172.21.15.5:0/616570591 learned_addr learned my addr 172.21.15.5:0/616570591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f2237300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f77fe700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f22950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 -- 172.21.15.5:0/616570591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 msgr2=0x5562f22950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f22950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 -- 172.21.15.5:0/616570591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 msgr2=0x5562f2237300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f2237300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 -- 172.21.15.5:0/616570591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562f1fa5de0 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3f67fc700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec096db0 tx=0x7ff3ec009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a0b07d08c6619f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3daff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0a2440 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3ec09a340 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff3ec094040 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.702+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3ec094200 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 10 client.16104 fetch_fsmap learned FSMap version 29 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7ff3f81107d0 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ec063070 con 0x5562f2227d20 2023-09-23T04:47:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff3ec09a5a0 con 0x5562f2227d20 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 20 client.16104 populate_metadata read hostname 'smithi005' 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2296c80 0x5562f22990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 4 client.16104 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562f2173b30 con 0x5562f2296c80 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.703+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2296c80 0x5562f22990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff3daff5700 10 client.16104 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff3f6ffd700 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2296c80 0x5562f22990a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ab79ac3ad68045 server_cookie=ad6e7a4e2a8803a2 in_seq=0 out_seq=0 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff3daff5700 1 -- 172.21.15.5:0/616570591 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5562f2173b30 con 0x5562f2296c80 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff3daff5700 10 client.16104 handle_command_reply: tid=0 2023-09-23T04:47:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 client.16104 shutdown 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 msgr2=0x5562f2237840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec096db0 tx=0x7ff3ec009ef0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 shutdown_connections 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5562f2296c80 0x5562f22990a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562f2232bd0 0x5562f22950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562f2227400 0x5562f2237300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 --2- 172.21.15.5:0/616570591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562f2227d20 0x5562f2237840 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 >> 172.21.15.5:0/616570591 conn(0x5562f219c260 msgr2=0x5562f223e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff3daff5700 0 client.16104 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 shutdown_connections 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.704+0000 7ff411217b80 1 -- 172.21.15.5:0/616570591 wait complete. 2023-09-23T04:47:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff411217b80 20 client.16104 trim_cache size 0 max 16384 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "c7304141-2dc2-46c9-90c4-acac729c3188": { 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c7304141-2dc2-46c9-90c4-acac729c3188", 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:47:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 msgr2=0x7ff3e00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 0x7ff3e00a3bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000b780 tx=0x7ff3f00094d0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 msgr2=0x7ff3f81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 0x7ff3f81997b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00e9f0 tx=0x7ff3ec00ed00 comp rx=0 tx=0).stop 2023-09-23T04:47:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 shutdown_connections 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff3e00a1710 0x7ff3e00a3bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f810d6a0 0x7ff3f819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff3f8073250 0x7ff3f8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 --2- 172.21.15.5:0/2871621304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f8073b70 0x7ff3f81997b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.705+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 >> 172.21.15.5:0/2871621304 conn(0x7ff3f80fc990 msgr2=0x7ff3f8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.706+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 shutdown_connections 2023-09-23T04:47:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:47:49.706+0000 7ff3fd7ff700 1 -- 172.21.15.5:0/2871621304 wait complete. 2023-09-23T04:47:49.723 INFO:tasks.cephfs.filesystem:scrub status for tag:c7304141-2dc2-46c9-90c4-acac729c3188 - {'path': '/', 'tag': 'c7304141-2dc2-46c9-90c4-acac729c3188', 'options': 'recursive,force'} 2023-09-23T04:47:49.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:49 smithi005 ceph-mon[97638]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-23T04:47:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:49 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:49 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:49 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:49 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:49.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25854 6.23 MB/sec warmup 25 sec latency 736.717 ms 2023-09-23T04:47:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:49 smithi069 ceph-mon[133931]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-23T04:47:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:49 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:49 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:49 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:49 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:49 smithi118 ceph-mon[134297]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-09-23T04:47:50.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:49 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:50.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:49 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:49 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:50.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:49 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.920 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25854 5.99 MB/sec warmup 26 sec latency 1377.920 ms 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:50.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:47:50.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:50.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:50 smithi069 ceph-mon[133931]: pgmap v50: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 278 KiB/s wr, 63 op/s 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:47:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:50 smithi005 ceph-mon[97638]: pgmap v50: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 278 KiB/s wr, 63 op/s 2023-09-23T04:47:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:47:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:47:51.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:50 smithi118 ceph-mon[134297]: pgmap v50: 129 pgs: 129 active+clean; 31 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 278 KiB/s wr, 63 op/s 2023-09-23T04:47:51.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25867 5.77 MB/sec warmup 27 sec latency 2353.946 ms 2023-09-23T04:47:52.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 28506 6.16 MB/sec warmup 28 sec latency 12.010 ms 2023-09-23T04:47:53.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29290 6.02 MB/sec warmup 29 sec latency 270.317 ms 2023-09-23T04:47:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:53 smithi069 ceph-mon[133931]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 468 KiB/s wr, 102 op/s 2023-09-23T04:47:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:53 smithi005 ceph-mon[97638]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 468 KiB/s wr, 102 op/s 2023-09-23T04:47:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:53 smithi118 ceph-mon[134297]: pgmap v51: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 468 KiB/s wr, 102 op/s 2023-09-23T04:47:54.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29290 5.82 MB/sec warmup 30 sec latency 1270.446 ms 2023-09-23T04:47:55.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29290 5.63 MB/sec warmup 31 sec latency 2270.598 ms 2023-09-23T04:47:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:55 smithi069 ceph-mon[133931]: pgmap v52: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 382 KiB/s wr, 84 op/s 2023-09-23T04:47:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:55 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:47:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:55 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:47:55.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:55 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:47:56.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:55 smithi005 ceph-mon[97638]: pgmap v52: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 382 KiB/s wr, 84 op/s 2023-09-23T04:47:56.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:55 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:47:56.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:55 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:47:56.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:55 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:47:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:55 smithi118 ceph-mon[134297]: pgmap v52: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 382 KiB/s wr, 84 op/s 2023-09-23T04:47:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:55 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:47:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:55 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:47:56.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:55 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:47:56.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29330 5.46 MB/sec warmup 32 sec latency 3230.685 ms 2023-09-23T04:47:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:56 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:47:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:56 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:47:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:56 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:47:57.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 31780 5.71 MB/sec warmup 33 sec latency 33.052 ms 2023-09-23T04:47:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:57 smithi069 ceph-mon[133931]: pgmap v53: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 382 KiB/s wr, 82 op/s 2023-09-23T04:47:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:57 smithi005 ceph-mon[97638]: pgmap v53: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 382 KiB/s wr, 82 op/s 2023-09-23T04:47:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:57 smithi118 ceph-mon[134297]: pgmap v53: 129 pgs: 129 active+clean; 33 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 382 KiB/s wr, 82 op/s 2023-09-23T04:47:58.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32891 5.70 MB/sec warmup 34 sec latency 92.207 ms 2023-09-23T04:47:58.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:47:58 smithi069 ceph-mon[133931]: pgmap v54: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 560 KiB/s wr, 119 op/s 2023-09-23T04:47:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:47:58 smithi005 ceph-mon[97638]: pgmap v54: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 560 KiB/s wr, 119 op/s 2023-09-23T04:47:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:47:58 smithi118 ceph-mon[134297]: pgmap v54: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 560 KiB/s wr, 119 op/s 2023-09-23T04:47:59.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 35506 6.02 MB/sec warmup 35 sec latency 27.744 ms 2023-09-23T04:48:00.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 36517 5.92 MB/sec warmup 36 sec latency 49.329 ms 2023-09-23T04:48:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:01 smithi005 ceph-mon[97638]: pgmap v55: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 369 KiB/s wr, 78 op/s 2023-09-23T04:48:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:01 smithi118 ceph-mon[134297]: pgmap v55: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 369 KiB/s wr, 78 op/s 2023-09-23T04:48:01.921 INFO:tasks.workunit.client.0.smithi005.stdout: 1 37910 5.92 MB/sec warmup 37 sec latency 51.183 ms 2023-09-23T04:48:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:01 smithi069 ceph-mon[133931]: pgmap v55: 129 pgs: 129 active+clean; 34 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 369 KiB/s wr, 78 op/s 2023-09-23T04:48:02.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39466 6.09 MB/sec warmup 38 sec latency 23.841 ms 2023-09-23T04:48:03.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 41611 6.14 MB/sec warmup 39 sec latency 13.678 ms 2023-09-23T04:48:03.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:03 smithi069 ceph-mon[133931]: pgmap v56: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 915 KiB/s wr, 193 op/s 2023-09-23T04:48:04.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:03 smithi005 ceph-mon[97638]: pgmap v56: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 915 KiB/s wr, 193 op/s 2023-09-23T04:48:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:03 smithi118 ceph-mon[134297]: pgmap v56: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 915 KiB/s wr, 193 op/s 2023-09-23T04:48:04.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 43125 6.26 MB/sec warmup 40 sec latency 10.341 ms 2023-09-23T04:48:05.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 45475 6.38 MB/sec warmup 41 sec latency 16.345 ms 2023-09-23T04:48:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:05 smithi069 ceph-mon[133931]: pgmap v57: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 725 KiB/s wr, 153 op/s 2023-09-23T04:48:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:05 smithi005 ceph-mon[97638]: pgmap v57: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 725 KiB/s wr, 153 op/s 2023-09-23T04:48:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:05 smithi118 ceph-mon[134297]: pgmap v57: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 725 KiB/s wr, 153 op/s 2023-09-23T04:48:06.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 46747 6.42 MB/sec warmup 42 sec latency 15.555 ms 2023-09-23T04:48:07.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 49041 6.54 MB/sec warmup 43 sec latency 14.429 ms 2023-09-23T04:48:07.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:07 smithi069 ceph-mon[133931]: pgmap v58: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 725 KiB/s wr, 152 op/s 2023-09-23T04:48:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:07 smithi005 ceph-mon[97638]: pgmap v58: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 725 KiB/s wr, 152 op/s 2023-09-23T04:48:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:07 smithi118 ceph-mon[134297]: pgmap v58: 129 pgs: 129 active+clean; 39 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 725 KiB/s wr, 152 op/s 2023-09-23T04:48:08.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50308 6.56 MB/sec warmup 44 sec latency 66.864 ms 2023-09-23T04:48:09.923 INFO:tasks.workunit.client.0.smithi005.stdout: 1 52671 6.69 MB/sec warmup 45 sec latency 9.950 ms 2023-09-23T04:48:09.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:09 smithi069 ceph-mon[133931]: pgmap v59: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 248 op/s 2023-09-23T04:48:10.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:09 smithi005 ceph-mon[97638]: pgmap v59: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 248 op/s 2023-09-23T04:48:10.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:09 smithi118 ceph-mon[134297]: pgmap v59: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 248 op/s 2023-09-23T04:48:10.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 54017 6.70 MB/sec warmup 46 sec latency 13.302 ms 2023-09-23T04:48:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:11 smithi005 ceph-mon[97638]: pgmap v60: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2023-09-23T04:48:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:11 smithi118 ceph-mon[134297]: pgmap v60: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2023-09-23T04:48:11.922 INFO:tasks.workunit.client.0.smithi005.stdout: 1 56445 6.83 MB/sec warmup 47 sec latency 17.815 ms 2023-09-23T04:48:11.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:11 smithi069 ceph-mon[133931]: pgmap v60: 129 pgs: 129 active+clean; 44 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2023-09-23T04:48:12.923 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57759 6.83 MB/sec warmup 48 sec latency 16.805 ms 2023-09-23T04:48:13.924 INFO:tasks.workunit.client.0.smithi005.stdout: 1 60291 7.03 MB/sec warmup 49 sec latency 20.788 ms 2023-09-23T04:48:13.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:13 smithi069 ceph-mon[133931]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2023-09-23T04:48:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:13 smithi005 ceph-mon[97638]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2023-09-23T04:48:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:13 smithi118 ceph-mon[134297]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 303 op/s 2023-09-23T04:48:14.924 INFO:tasks.workunit.client.0.smithi005.stdout: 1 61516 6.96 MB/sec warmup 50 sec latency 12.224 ms 2023-09-23T04:48:15.924 INFO:tasks.workunit.client.0.smithi005.stdout: 1 64085 7.15 MB/sec warmup 51 sec latency 17.098 ms 2023-09-23T04:48:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:15 smithi069 ceph-mon[133931]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:15 smithi005 ceph-mon[97638]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:15 smithi118 ceph-mon[134297]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:16.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 65321 7.12 MB/sec warmup 52 sec latency 11.702 ms 2023-09-23T04:48:17.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67642 7.24 MB/sec warmup 53 sec latency 24.751 ms 2023-09-23T04:48:17.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:17 smithi069 ceph-mon[133931]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:17 smithi005 ceph-mon[97638]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:17 smithi118 ceph-mon[134297]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-09-23T04:48:18.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 69106 7.21 MB/sec warmup 54 sec latency 11.810 ms 2023-09-23T04:48:19.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:48:19.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71173 7.33 MB/sec warmup 55 sec latency 61.688 ms 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 -- 172.21.15.5:0/1591264356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 msgr2=0x7f2edc1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/1591264356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc1036c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009990 tx=0x7f2ed8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 -- 172.21.15.5:0/1591264356 shutdown_connections 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/1591264356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc108400 0x7f2edc10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/1591264356 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc1036c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/1591264356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc103c90 0x7f2edc107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 -- 172.21.15.5:0/1591264356 >> 172.21.15.5:0/1591264356 conn(0x7f2edc0feb40 msgr2=0x7f2edc100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 -- 172.21.15.5:0/1591264356 shutdown_connections 2023-09-23T04:48:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.979+0000 7f2ee43ff700 1 -- 172.21.15.5:0/1591264356 wait complete. 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.980+0000 7f2ee43ff700 1 Processor -- start 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.980+0000 7f2ee43ff700 1 -- start start 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.980+0000 7f2ee43ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.980+0000 7f2ee43ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc108400 0x7f2edc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee43ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc19dba0 0x7f2edc19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee43ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19e620 con 0x7f2edc19dba0 2023-09-23T04:48:19.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee43ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19e790 con 0x7f2edc108400 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee43ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc19e950 con 0x7f2edc1032e0 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee299c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc19dba0 0x7f2edc19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42108/0 (socket says 172.21.15.5:42108) 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 -- 172.21.15.5:0/2033808750 learned_addr learned my addr 172.21.15.5:0/2033808750 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee199a700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc108400 0x7f2edc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 -- 172.21.15.5:0/2033808750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc108400 msgr2=0x7f2edc199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc108400 0x7f2edc199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 -- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc19dba0 msgr2=0x7f2edc19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:19.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc19dba0 0x7f2edc19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.981+0000 7f2ee219b700 1 -- 172.21.15.5:0/2033808750 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed80095f0 con 0x7f2edc1032e0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005910 tx=0x7f2ed8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ed37fe700 1 -- 172.21.15.5:0/2033808750 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801d070 con 0x7f2edc1032e0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc1a5e50 con 0x7f2edc1032e0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ed37fe700 1 -- 172.21.15.5:0/2033808750 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed8022470 con 0x7f2edc1032e0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ed37fe700 1 -- 172.21.15.5:0/2033808750 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8017650 con 0x7f2edc1032e0 2023-09-23T04:48:19.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.982+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc1a6220 con 0x7f2edc1032e0 2023-09-23T04:48:19.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.986+0000 7f2ed37fe700 1 -- 172.21.15.5:0/2033808750 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2ed80225e0 con 0x7f2edc1032e0 2023-09-23T04:48:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.987+0000 7f2ef7ddfb80 1 Processor -- start 2023-09-23T04:48:19.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 -- start start 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x5601032437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x560103250fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601031ac460 con 0x560103243400 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010319dc20 con 0x560103243d20 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010319b540 con 0x56010324ebd0 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ed37fe700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 0x7f2ec80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:54954/0 (socket says 172.21.15.5:54954) 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ee219b700 1 -- 172.21.15.5:0/1254454705 learned_addr learned my addr 172.21.15.5:0/1254454705 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ee299c700 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x5601032437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ee199a700 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x560103250fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.988+0000 7f2ed37fe700 1 -- 172.21.15.5:0/2033808750 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2ed80cea60 con 0x7f2edc1032e0 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 0x7f2ec80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 -- 172.21.15.5:0/1254454705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 msgr2=0x560103250fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x560103250fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 -- 172.21.15.5:0/1254454705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 msgr2=0x5601032437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x5601032437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 -- 172.21.15.5:0/1254454705 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601031f98d0 con 0x560103243d20 2023-09-23T04:48:19.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809b5b0 tx=0x7f2ed8022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43d29ec0624b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:19.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:19 smithi069 ceph-mon[133931]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 294 op/s 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ed0bfe700 1 -- 172.21.15.5:0/1254454705 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80860d0 con 0x560103243d20 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ed0bfe700 1 -- 172.21.15.5:0/1254454705 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed809b160 con 0x560103243d20 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.989+0000 7f2ed0bfe700 1 -- 172.21.15.5:0/1254454705 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80a3920 con 0x560103243d20 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1254454705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 msgr2=0x56010324e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809b5b0 tx=0x7f2ed8022c90 comp rx=0 tx=0).stop 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 0x7f2ec80a3bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009ec0 tx=0x7f2ed400b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1254454705 shutdown_connections 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x560103250fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x5601032437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1254454705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243d20 0x56010324e690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1254454705 >> 172.21.15.5:0/1254454705 conn(0x5601031b8260 msgr2=0x56010323f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1254454705 shutdown_connections 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.990+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1254454705 wait complete. 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.991+0000 7f2ef7ddfb80 1 -- start start 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.991+0000 7f2ef7ddfb80 1 Processor -- start 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010324efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x56010324fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103196e30 con 0x560103243400 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103133710 con 0x56010324ebd0 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601031333b0 con 0x560103243d20 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ef7ddfb80 10 client.0 _ll_register_callbacks cb 0x560102f90800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010324efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42132/0 (socket says 172.21.15.5:42132) 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 -- 172.21.15.5:0/2232072418 learned_addr learned my addr 172.21.15.5:0/2232072418 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x56010324fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 -- 172.21.15.5:0/2232072418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 msgr2=0x56010324fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x56010324fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 -- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 msgr2=0x56010324efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010324efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee199a700 1 -- 172.21.15.5:0/2232072418 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc1a6220 con 0x560103243d20 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.992+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010324efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ee199a700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc012840 tx=0x7f2ecc012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9864ddca2aa923e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc017070 con 0x560103243d20 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560103237250 con 0x560103243d20 2023-09-23T04:48:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2ecc004830 con 0x560103243d20 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc004a30 con 0x560103243d20 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ef7ddfb80 10 client.26155 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.993+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560102fc1de0 con 0x560103243d20 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560103237250 con 0x560103243d20 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2ecc0105f0 con 0x560103243d20 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 10 client.26155 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 20 client.26155 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56010325e2a0 0x5601032606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 4 client.26155 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010318fe10 con 0x56010325e2a0 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.994+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56010325e2a0 0x5601032606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.995+0000 7f2ee219b700 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56010325e2a0 0x5601032606c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc1706dec737b01 server_cookie=307e816440c88998 in_seq=0 out_seq=0 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.995+0000 7f2ec5ffb700 10 client.26155 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.995+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/2232072418 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56010318fe10 con 0x56010325e2a0 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.995+0000 7f2ec5ffb700 10 client.26155 handle_command_reply: tid=0 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.995+0000 7f2ef7ddfb80 1 client.26155 shutdown 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 msgr2=0x56010324f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc012840 tx=0x7f2ecc012b50 comp rx=0 tx=0).stop 2023-09-23T04:48:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 shutdown_connections 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56010325e2a0 0x5601032606c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x56010324fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010324efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2232072418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x56010324f4f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ec5ffb700 0 client.26155 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 >> 172.21.15.5:0/2232072418 conn(0x5601031b8260 msgr2=0x56010323f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 shutdown_connections 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2232072418 wait complete. 2023-09-23T04:48:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:19.996+0000 7f2ef7ddfb80 20 client.26155 trim_cache size 0 max 16384 2023-09-23T04:48:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.017+0000 7f2ef7ddfb80 1 Processor -- start 2023-09-23T04:48:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.017+0000 7f2ef7ddfb80 1 -- start start 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010325a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x560103248360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103017920 con 0x560103243400 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010324c250 con 0x56010324ebd0 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103249b60 con 0x560103243d20 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee199a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee219b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010325a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee299c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x560103248360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee199a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42144/0 (socket says 172.21.15.5:42144) 2023-09-23T04:48:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee199a700 1 -- 172.21.15.5:0/2264148367 learned_addr learned my addr 172.21.15.5:0/2264148367 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee299c700 1 -- 172.21.15.5:0/2264148367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 msgr2=0x560103259720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee299c700 1 -- 172.21.15.5:0/2264148367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 msgr2=0x56010325a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.018+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010325a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ee299c700 1 -- 172.21.15.5:0/2264148367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560102faac60 con 0x56010324ebd0 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ee199a700 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ee299c700 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x560103248360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007dc0 tx=0x7f2ed4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d15bb506723ac70 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ed13ff700 1 -- 172.21.15.5:0/2264148367 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4028070 con 0x56010324ebd0 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ed13ff700 1 -- 172.21.15.5:0/2264148367 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed400f040 con 0x56010324ebd0 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.019+0000 7f2ed13ff700 1 -- 172.21.15.5:0/2264148367 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400e070 con 0x56010324ebd0 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2264148367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 msgr2=0x560103248360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x560103248360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007dc0 tx=0x7f2ed4009920 comp rx=0 tx=0).stop 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2264148367 shutdown_connections 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56010324ebd0 0x560103248360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243400 0x56010325a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/2264148367 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560103243d20 0x560103259720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2264148367 >> 172.21.15.5:0/2264148367 conn(0x5601031b8260 msgr2=0x56010324f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2264148367 shutdown_connections 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.020+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/2264148367 wait complete. 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 -- start start 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 Processor -- start 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243400 0x560103251dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243d20 0x560103252300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601032b1630 con 0x560103243d20 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601032b1770 con 0x560103243400 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.021+0000 7f2ef7ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601032b1900 con 0x56010324ebd0 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ef7ddfb80 10 client.0 _ll_register_callbacks cb 0x7f2ecc009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee199a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243d20 0x560103252300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42152/0 (socket says 172.21.15.5:42152) 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 -- 172.21.15.5:0/1540657674 learned_addr learned my addr 172.21.15.5:0/1540657674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243400 0x560103251dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 -- 172.21.15.5:0/1540657674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243400 msgr2=0x560103251dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243400 0x560103251dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 -- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243d20 msgr2=0x560103252300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243d20 0x560103252300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 -- 172.21.15.5:0/1540657674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc021040 con 0x56010324ebd0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ee299c700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005b30 tx=0x7f2ed4005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54e7b2f11bb6fc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.022+0000 7f2ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4028070 con 0x56010324ebd0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ed809b160 con 0x56010324ebd0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2ed4004de0 con 0x56010324ebd0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed400e040 con 0x56010324ebd0 2023-09-23T04:48:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 10 client.26161 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2ed4020040 con 0x56010324ebd0 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed809b160 con 0x56010324ebd0 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2ed402dcf0 con 0x56010324ebd0 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 10 client.26161 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 20 client.26161 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.023+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5601032b30e0 0x5601032b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ef7ddfb80 4 client.26161 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:48:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56010318fb30 con 0x5601032b30e0 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5601032b30e0 0x5601032b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ec5ffb700 10 client.26161 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ee219b700 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5601032b30e0 0x5601032b5500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc08cd529ba0452f server_cookie=8e5ed86ab02b6bdd in_seq=0 out_seq=0 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ec5ffb700 1 -- 172.21.15.5:0/1540657674 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56010318fb30 con 0x5601032b30e0 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.024+0000 7f2ec5ffb700 10 client.26161 handle_command_reply: tid=0 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 client.26161 shutdown 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 msgr2=0x5601032b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005b30 tx=0x7f2ed4005b60 comp rx=0 tx=0).stop 2023-09-23T04:48:20.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 shutdown_connections 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5601032b30e0 0x5601032b5500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56010324ebd0 0x5601032b10f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560103243400 0x560103251dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ec5ffb700 0 client.26161 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 --2- 172.21.15.5:0/1540657674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560103243d20 0x560103252300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 >> 172.21.15.5:0/1540657674 conn(0x5601031b8260 msgr2=0x56010324cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 shutdown_connections 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 1 -- 172.21.15.5:0/1540657674 wait complete. 2023-09-23T04:48:20.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.025+0000 7f2ef7ddfb80 20 client.26161 trim_cache size 0 max 16384 2023-09-23T04:48:20.033 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:48:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:48:20.033 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 msgr2=0x7f2ec80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 0x7f2ec80a3bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009ec0 tx=0x7f2ed400b040 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 msgr2=0x7f2edc199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8005910 tx=0x7f2ed8005740 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 shutdown_connections 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2ec80a1710 0x7f2ec80a3bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2edc19dba0 0x7f2edc19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2edc1032e0 0x7f2edc199200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 --2- 172.21.15.5:0/2033808750 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2edc108400 0x7f2edc199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 >> 172.21.15.5:0/2033808750 conn(0x7f2edc0feb40 msgr2=0x7f2edc106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 shutdown_connections 2023-09-23T04:48:20.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.026+0000 7f2ee43ff700 1 -- 172.21.15.5:0/2033808750 wait complete. 2023-09-23T04:48:20.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:19 smithi005 ceph-mon[97638]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 294 op/s 2023-09-23T04:48:20.045 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:48:20.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:48:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:19 smithi118 ceph-mon[134297]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 294 op/s 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 -- 172.21.15.5:0/2169985640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f987410d680 msgr2=0x7f987410faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 --2- 172.21.15.5:0/2169985640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f987410d680 0x7f987410faf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b0a0 tx=0x7f986c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 -- 172.21.15.5:0/2169985640 shutdown_connections 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 --2- 172.21.15.5:0/2169985640 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f987410d680 0x7f987410faf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 --2- 172.21.15.5:0/2169985640 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874073250 0x7f9874073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 --2- 172.21.15.5:0/2169985640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073b70 0x7f987410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.282+0000 7f987abff700 1 -- 172.21.15.5:0/2169985640 >> 172.21.15.5:0/2169985640 conn(0x7f98740fc930 msgr2=0x7f98740fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.283+0000 7f987abff700 1 -- 172.21.15.5:0/2169985640 shutdown_connections 2023-09-23T04:48:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.283+0000 7f987abff700 1 -- 172.21.15.5:0/2169985640 wait complete. 2023-09-23T04:48:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.284+0000 7f987abff700 1 Processor -- start 2023-09-23T04:48:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.284+0000 7f987abff700 1 -- start start 2023-09-23T04:48:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073250 0x7f9874199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987410d680 0x7f987419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419e240 con 0x7f9874073250 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419e3b0 con 0x7f987410d680 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987abff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419e570 con 0x7f9874073b70 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987899b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073250 0x7f9874199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42166/0 (socket says 172.21.15.5:42166) 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 -- 172.21.15.5:0/246406922 learned_addr learned my addr 172.21.15.5:0/246406922 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f987919c700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987410d680 0x7f987419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 -- 172.21.15.5:0/246406922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987410d680 msgr2=0x7f987419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987410d680 0x7f987419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 -- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073250 msgr2=0x7f9874199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073250 0x7f9874199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.285+0000 7f9873fff700 1 -- 172.21.15.5:0/246406922 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c009d00 con 0x7f9874073b70 2023-09-23T04:48:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f9873fff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f986000e9a0 tx=0x7f986000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860009790 con 0x7f9874073b70 2023-09-23T04:48:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9860010430 con 0x7f9874073b70 2023-09-23T04:48:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860018680 con 0x7f9874073b70 2023-09-23T04:48:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987419e850 con 0x7f9874073b70 2023-09-23T04:48:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.286+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98741a6090 con 0x7f9874073b70 2023-09-23T04:48:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.287+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9874107fa0 con 0x7f9874073b70 2023-09-23T04:48:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.290+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f98600105a0 con 0x7f9874073b70 2023-09-23T04:48:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.291+0000 7f9871ffb700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 0x7f98640a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.291+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9860014070 con 0x7f9874073b70 2023-09-23T04:48:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.291+0000 7f987899b700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 0x7f98640a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.292+0000 7f987899b700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 0x7f98640a3c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9868005950 tx=0x7f986800b410 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.295+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9860096720 con 0x7f9874073b70 2023-09-23T04:48:20.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.583+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9874067ca0 con 0x7f9874073b70 2023-09-23T04:48:20.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.584+0000 7f9871ffb700 1 -- 172.21.15.5:0/246406922 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f9860099d40 con 0x7f9874073b70 2023-09-23T04:48:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:48:20.585 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:47:40.065282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 msgr2=0x7f98640a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 0x7f98640a3c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9868005950 tx=0x7f986800b410 comp rx=0 tx=0).stop 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 msgr2=0x7f98741997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f986000e9a0 tx=0x7f986000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 shutdown_connections 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f98640a1790 0x7f98640a3c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f987410d680 0x7f987419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9874073250 0x7f9874199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 --2- 172.21.15.5:0/246406922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9874073b70 0x7f98741997a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 >> 172.21.15.5:0/246406922 conn(0x7f98740fc930 msgr2=0x7f98741050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 shutdown_connections 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.587+0000 7f987abff700 1 -- 172.21.15.5:0/246406922 wait complete. 2023-09-23T04:48:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2023-09-23T04:48:20.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 -- 172.21.15.5:0/3282805784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91741010d0 msgr2=0x7f91741014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 --2- 172.21.15.5:0/3282805784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91741010d0 0x7f91741014b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9164009990 tx=0x7f9164009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 -- 172.21.15.5:0/3282805784 shutdown_connections 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 --2- 172.21.15.5:0/3282805784 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 0x7f917410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 --2- 172.21.15.5:0/3282805784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91741010d0 0x7f91741014b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 --2- 172.21.15.5:0/3282805784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9174101a80 0x7f9174105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 -- 172.21.15.5:0/3282805784 >> 172.21.15.5:0/3282805784 conn(0x7f91740fc990 msgr2=0x7f91740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 -- 172.21.15.5:0/3282805784 shutdown_connections 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.835+0000 7f91795ff700 1 -- 172.21.15.5:0/3282805784 wait complete. 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.836+0000 7f91795ff700 1 Processor -- start 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.836+0000 7f91795ff700 1 -- start start 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.836+0000 7f91795ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.836+0000 7f91795ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 0x7f9174073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91795ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174073ba0 0x7f91741a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91795ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741a5840 con 0x7f9174073ba0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91795ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741a59b0 con 0x7f91741010d0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91795ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741a5b70 con 0x7f91741061f0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45022/0 (socket says 172.21.15.5:45022) 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 -- 172.21.15.5:0/3316926255 learned_addr learned my addr 172.21.15.5:0/3316926255 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91727fc700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 0x7f9174073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f91737fe700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174073ba0 0x7f91741a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 -- 172.21.15.5:0/3316926255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 msgr2=0x7f9174073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 0x7f9174073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 -- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174073ba0 msgr2=0x7f91741a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174073ba0 0x7f91741a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.837+0000 7f9172ffd700 1 -- 172.21.15.5:0/3316926255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640095f0 con 0x7f91741010d0 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f9172ffd700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9164005910 tx=0x7f9164004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f915bfff700 1 -- 172.21.15.5:0/3316926255 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916401d070 con 0x7f91741010d0 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f915bfff700 1 -- 172.21.15.5:0/3316926255 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9164022470 con 0x7f91741010d0 2023-09-23T04:48:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9174196d30 con 0x7f91741010d0 2023-09-23T04:48:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f915bfff700 1 -- 172.21.15.5:0/3316926255 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164017650 con 0x7f91741010d0 2023-09-23T04:48:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.838+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91741971e0 con 0x7f91741010d0 2023-09-23T04:48:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.841+0000 7f915bfff700 1 -- 172.21.15.5:0/3316926255 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f91640225e0 con 0x7f91741010d0 2023-09-23T04:48:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 Processor -- start 2023-09-23T04:48:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 -- start start 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f915bfff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 0x7f915c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f915bfff700 1 -- 172.21.15.5:0/3316926255 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f91640ce960 con 0x7f91741010d0 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f91737fe700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 0x7f915c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f10fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.843+0000 7f918d006b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96e6c460 con 0x562c96f03400 2023-09-23T04:48:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f918d006b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96e5dc20 con 0x562c96f03d20 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f918d006b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96e5b540 con 0x562c96f0ebd0 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45036/0 (socket says 172.21.15.5:45036) 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 -- 172.21.15.5:0/1017985915 learned_addr learned my addr 172.21.15.5:0/1017985915 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f9172ffd700 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91737fe700 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 -- 172.21.15.5:0/1017985915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 msgr2=0x562c96f10fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f10fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 -- 172.21.15.5:0/1017985915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 msgr2=0x562c96f037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91727fc700 1 -- 172.21.15.5:0/1017985915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c96eb98d0 con 0x562c96f03d20 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.844+0000 7f91737fe700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 0x7f915c0a3bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f916c0093c0 tx=0x7f916c009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.845+0000 7f91727fc700 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9168004200 tx=0x7f9168009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a03bf9570214b3d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.845+0000 7f91597fa700 1 -- 172.21.15.5:0/1017985915 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916801b040 con 0x562c96f03d20 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.845+0000 7f91597fa700 1 -- 172.21.15.5:0/1017985915 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9168009e00 con 0x562c96f03d20 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.845+0000 7f91597fa700 1 -- 172.21.15.5:0/1017985915 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916801f680 con 0x562c96f03d20 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- 172.21.15.5:0/1017985915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 msgr2=0x562c96f0e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9168004200 tx=0x7f9168009260 comp rx=0 tx=0).stop 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- 172.21.15.5:0/1017985915 shutdown_connections 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f10fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 --2- 172.21.15.5:0/1017985915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0e690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- 172.21.15.5:0/1017985915 >> 172.21.15.5:0/1017985915 conn(0x562c96e78260 msgr2=0x562c96eff790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- 172.21.15.5:0/1017985915 shutdown_connections 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- 172.21.15.5:0/1017985915 wait complete. 2023-09-23T04:48:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.846+0000 7f918d006b80 1 -- start start 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 Processor -- start 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f0bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96e56e30 con 0x562c96f03400 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96df3710 con 0x562c96f03d20 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96df33b0 con 0x562c96f0ebd0 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f918d006b80 10 client.0 _ll_register_callbacks cb 0x562c96c50800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f91727fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49730/0 (socket says 172.21.15.5:49730) 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.847+0000 7f9172ffd700 1 -- 172.21.15.5:0/57476859 learned_addr learned my addr 172.21.15.5:0/57476859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f91737fe700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f0bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 -- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 msgr2=0x562c96f0bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f0bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 -- 172.21.15.5:0/57476859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 msgr2=0x562c96f0b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 -- 172.21.15.5:0/57476859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9168016df0 con 0x562c96f03400 2023-09-23T04:48:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f91737fe700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f0bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f91727fc700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9172ffd700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9164097f90 tx=0x7f916409af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3ccb412f98c945 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.848+0000 7f9156ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164097040 con 0x562c96f03400 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c96ef7250 con 0x562c96f03400 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f916409b520 con 0x562c96f03400 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f916409f670 con 0x562c96f03400 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 10 client.16140 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c96c81de0 con 0x562c96f03400 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c96ef7250 con 0x562c96f03400 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f916409f170 con 0x562c96f03400 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 10 client.16140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 20 client.16140 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.849+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f1e370 0x562c96f20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.850+0000 7f918d006b80 4 client.16140 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.850+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c96e4fe10 con 0x562c96f1e370 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.850+0000 7f91727fc700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f1e370 0x562c96f20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.850+0000 7f91727fc700 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f1e370 0x562c96f20760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f650adcc7c49cbd server_cookie=26be58471a328490 in_seq=0 out_seq=0 2023-09-23T04:48:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.850+0000 7f9156ff5700 10 client.16140 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f9156ff5700 1 -- 172.21.15.5:0/57476859 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562c96e4fe10 con 0x562c96f1e370 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f9156ff5700 10 client.16140 handle_command_reply: tid=0 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 client.16140 shutdown 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 msgr2=0x562c96f0e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9164097f90 tx=0x7f916409af00 comp rx=0 tx=0).stop 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 shutdown_connections 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f0ebd0 0x562c96f0bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f1e370 0x562c96f20760 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f03400 0x562c96f0e630 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 --2- 172.21.15.5:0/57476859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03d20 0x562c96f0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 >> 172.21.15.5:0/57476859 conn(0x562c96e78260 msgr2=0x562c96eff320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.851+0000 7f9156ff5700 0 client.16140 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.852+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 shutdown_connections 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.852+0000 7f918d006b80 1 -- 172.21.15.5:0/57476859 wait complete. 2023-09-23T04:48:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.852+0000 7f918d006b80 20 client.16140 trim_cache size 0 max 16384 2023-09-23T04:48:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 Processor -- start 2023-09-23T04:48:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 -- start start 2023-09-23T04:48:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f06bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96cd7920 con 0x562c96f0ebd0 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96f20a90 con 0x562c96f03400 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.866+0000 7f918d006b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96f07760 con 0x562c96f03d20 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45060/0 (socket says 172.21.15.5:45060) 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f91727fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f06bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 -- 172.21.15.5:0/2115226264 learned_addr learned my addr 172.21.15.5:0/2115226264 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f91737fe700 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 -- 172.21.15.5:0/2115226264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 msgr2=0x562c96f06bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f06bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 -- 172.21.15.5:0/2115226264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 msgr2=0x562c96f07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 -- 172.21.15.5:0/2115226264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c96efb480 con 0x562c96f03400 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f91727fc700 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f916409b4b0 tx=0x7f9164097990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a31d8e0f46526be server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9159ffb700 1 -- 172.21.15.5:0/2115226264 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164097040 con 0x562c96f03400 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.867+0000 7f9159ffb700 1 -- 172.21.15.5:0/2115226264 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916409b520 con 0x562c96f03400 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f9159ffb700 1 -- 172.21.15.5:0/2115226264 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916409f300 con 0x562c96f03400 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 -- 172.21.15.5:0/2115226264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 msgr2=0x562c96f0e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f916409b4b0 tx=0x7f9164097990 comp rx=0 tx=0).stop 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 -- 172.21.15.5:0/2115226264 shutdown_connections 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f0e1d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 --2- 172.21.15.5:0/2115226264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 -- 172.21.15.5:0/2115226264 >> 172.21.15.5:0/2115226264 conn(0x562c96e78260 msgr2=0x562c96f20380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 -- 172.21.15.5:0/2115226264 shutdown_connections 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.868+0000 7f918d006b80 1 -- 172.21.15.5:0/2115226264 wait complete. 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.869+0000 7f918d006b80 1 -- start start 2023-09-23T04:48:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.869+0000 7f918d006b80 1 Processor -- start 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96f14490 con 0x562c96f0ebd0 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96f71140 con 0x562c96f03400 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c96f71280 con 0x562c96f03d20 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f918d006b80 10 client.0 _ll_register_callbacks cb 0x562c96c50800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f91727fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f91727fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48262/0 (socket says 172.21.15.5:48262) 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 -- 172.21.15.5:0/2606842801 learned_addr learned my addr 172.21.15.5:0/2606842801 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f91737fe700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 -- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 msgr2=0x562c96f13700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 -- 172.21.15.5:0/2606842801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 msgr2=0x562c96f13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.870+0000 7f9172ffd700 1 -- 172.21.15.5:0/2606842801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c96eb98d0 con 0x562c96f03400 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f91727fc700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9172ffd700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f916402f040 tx=0x7f916409ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23339fef97c47123 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9156ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164097040 con 0x562c96f03400 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f916409f670 con 0x562c96f03400 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9168009e00 con 0x562c96f03400 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91640a3820 con 0x562c96f03400 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f918d006b80 10 client.26129 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c96c81de0 con 0x562c96f03400 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.871+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9168009e00 con 0x562c96f03400 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9164097430 con 0x562c96f03400 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 10 client.26129 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 10 client.26129 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 20 client.26129 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f72450 0x562c96f74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 4 client.26129 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c96e4fb30 con 0x562c96f72450 2023-09-23T04:48:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.872+0000 7f91727fc700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f72450 0x562c96f74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f91727fc700 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f72450 0x562c96f74840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3048b28d2899dcb server_cookie=f3678faa5a779298 in_seq=0 out_seq=0 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f9156ff5700 10 client.26129 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f9156ff5700 1 -- 172.21.15.5:0/2606842801 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c96e4fb30 con 0x562c96f72450 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f9156ff5700 10 client.26129 handle_command_reply: tid=0 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 client.26129 shutdown 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 msgr2=0x562c96f15910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f916402f040 tx=0x7f916409ac40 comp rx=0 tx=0).stop 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 shutdown_connections 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x562c96f72450 0x562c96f74840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96f0ebd0 0x562c96f13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c96f03400 0x562c96f15910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 --2- 172.21.15.5:0/2606842801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562c96f03d20 0x562c96f13700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f9156ff5700 0 client.26129 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 >> 172.21.15.5:0/2606842801 conn(0x562c96e78260 msgr2=0x562c96f06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 shutdown_connections 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 1 -- 172.21.15.5:0/2606842801 wait complete. 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.873+0000 7f918d006b80 20 client.26129 trim_cache size 0 max 16384 2023-09-23T04:48:20.883 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 msgr2=0x7f915c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 0x7f915c0a3bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f916c0093c0 tx=0x7f916c009920 comp rx=0 tx=0).stop 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 msgr2=0x7f9174073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9164005910 tx=0x7f9164004730 comp rx=0 tx=0).stop 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 shutdown_connections 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f915c0a1710 0x7f915c0a3bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174073ba0 0x7f91741a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91741010d0 0x7f9174073120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 --2- 172.21.15.5:0/3316926255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f91741061f0 0x7f9174073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 >> 172.21.15.5:0/3316926255 conn(0x7f91740fc990 msgr2=0x7f91740fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 shutdown_connections 2023-09-23T04:48:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:20.874+0000 7f91795ff700 1 -- 172.21.15.5:0/3316926255 wait complete. 2023-09-23T04:48:20.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:48:20.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 72221 7.24 MB/sec warmup 56 sec latency 28.021 ms 2023-09-23T04:48:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.133+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/3677296875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c8073250 msgr2=0x7fe2c8073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.133+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/3677296875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c8073250 0x7fe2c8073630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009990 tx=0x7fe2b8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/3677296875 shutdown_connections 2023-09-23T04:48:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/3677296875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c810d6a0 0x7fe2c810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/3677296875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c8073250 0x7fe2c8073630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/3677296875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073b70 0x7fe2c810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/3677296875 >> 172.21.15.5:0/3677296875 conn(0x7fe2c80fc990 msgr2=0x7fe2c80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/3677296875 shutdown_connections 2023-09-23T04:48:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.134+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/3677296875 wait complete. 2023-09-23T04:48:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.135+0000 7fe2cd1ff700 1 Processor -- start 2023-09-23T04:48:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.135+0000 7fe2cd1ff700 1 -- start start 2023-09-23T04:48:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.135+0000 7fe2cd1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.135+0000 7fe2cd1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 0x7fe2c8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.135+0000 7fe2cd1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c810d6a0 0x7fe2c819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2cd1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819e120 con 0x7fe2c8073b70 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2cd1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819e290 con 0x7fe2c810d6a0 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2cd1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819e450 con 0x7fe2c8073250 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 0x7fe2c8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48270/0 (socket says 172.21.15.5:48270) 2023-09-23T04:48:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/1446573733 learned_addr learned my addr 172.21.15.5:0/1446573733 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c759e700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c810d6a0 0x7fe2c819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/1446573733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c810d6a0 msgr2=0x7fe2c819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c810d6a0 0x7fe2c819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 msgr2=0x7fe2c8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 0x7fe2c8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/1446573733 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80095f0 con 0x7fe2c8073250 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.136+0000 7fe2c659c700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 0x7fe2c8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000c00 tx=0x7fe2b8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2affff700 1 -- 172.21.15.5:0/1446573733 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b801d070 con 0x7fe2c8073250 2023-09-23T04:48:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2affff700 1 -- 172.21.15.5:0/1446573733 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b8022470 con 0x7fe2c8073250 2023-09-23T04:48:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c819e6d0 con 0x7fe2c8073250 2023-09-23T04:48:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2affff700 1 -- 172.21.15.5:0/1446573733 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8017650 con 0x7fe2c8073250 2023-09-23T04:48:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.137+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c81a5f70 con 0x7fe2c8073250 2023-09-23T04:48:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.141+0000 7fe2affff700 1 -- 172.21.15.5:0/1446573733 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe2b80225e0 con 0x7fe2c8073250 2023-09-23T04:48:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.142+0000 7fe2affff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 0x7fe2b00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.142+0000 7fe2e0aedb80 1 Processor -- start 2023-09-23T04:48:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.142+0000 7fe2affff700 1 -- 172.21.15.5:0/1446573733 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe2b80cead0 con 0x7fe2c8073250 2023-09-23T04:48:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2c659c700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 0x7fe2b00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 -- start start 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207970460 con 0x559207a07d20 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207961c20 con 0x559207a07400 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920795f540 con 0x559207a12bd0 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2c759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2c759e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48274/0 (socket says 172.21.15.5:48274) 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.143+0000 7fe2c759e700 1 -- 172.21.15.5:0/1521394296 learned_addr learned my addr 172.21.15.5:0/1521394296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c659c700 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c659c700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 0x7fe2b00a3c50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2c819a750 tx=0x7fe2bc009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 -- 172.21.15.5:0/1521394296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 msgr2=0x559207a077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 -- 172.21.15.5:0/1521394296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 msgr2=0x559207a12690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a12690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 -- 172.21.15.5:0/1521394296 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592079bd8d0 con 0x559207a12bd0 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2c759e700 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0011680 tx=0x7fe2c0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42affbb954afb97a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2ad7fa700 1 -- 172.21.15.5:0/1521394296 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000e040 con 0x559207a12bd0 2023-09-23T04:48:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2ad7fa700 1 -- 172.21.15.5:0/1521394296 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c000f040 con 0x559207a12bd0 2023-09-23T04:48:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.144+0000 7fe2ad7fa700 1 -- 172.21.15.5:0/1521394296 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0004860 con 0x559207a12bd0 2023-09-23T04:48:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/1521394296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 msgr2=0x559207a14fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0011680 tx=0x7fe2c0009920 comp rx=0 tx=0).stop 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/1521394296 shutdown_connections 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a12bd0 0x559207a14fc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/1521394296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/1521394296 >> 172.21.15.5:0/1521394296 conn(0x55920797c260 msgr2=0x559207a03790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/1521394296 shutdown_connections 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.145+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/1521394296 wait complete. 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.146+0000 7fe2e0aedb80 1 -- start start 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.146+0000 7fe2e0aedb80 1 Processor -- start 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a0f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55920795ae30 con 0x559207a07d20 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592078f7710 con 0x559207a12bd0 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592078f73b0 con 0x559207a07400 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2e0aedb80 10 client.0 _ll_register_callbacks cb 0x559207754800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48276/0 (socket says 172.21.15.5:48276) 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/3562058035 learned_addr learned my addr 172.21.15.5:0/3562058035 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2c759e700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.147+0000 7fe2c659c700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a0f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/3562058035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 msgr2=0x559207a0ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a0ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/3562058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 msgr2=0x559207a0f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a0f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/3562058035 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c0011330 con 0x559207a07400 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8097b90 tx=0x7fe2b809b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2e4e58cd7cfdeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b809bd90 con 0x559207a07400 2023-09-23T04:48:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592079fb250 con 0x559207a07400 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe2b809b690 con 0x559207a07400 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b809f7e0 con 0x559207a07400 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.148+0000 7fe2e0aedb80 10 client.26197 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559207785de0 con 0x559207a07400 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592079fb250 con 0x559207a07400 2023-09-23T04:48:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe2b80a3500 con 0x559207a07400 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 10 client.26197 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 10 client.26197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 20 client.26197 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a22280 0x559207a24670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 4 client.26197 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.149+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559207953e10 con 0x559207a22280 2023-09-23T04:48:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.150+0000 7fe2c659c700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a22280 0x559207a24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.150+0000 7fe2aaff5700 10 client.26197 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.150+0000 7fe2c659c700 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a22280 0x559207a24670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8ce8893bb9a539a server_cookie=7c0e50a3af6902e7 in_seq=0 out_seq=0 2023-09-23T04:48:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2aaff5700 1 -- 172.21.15.5:0/3562058035 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559207953e10 con 0x559207a22280 2023-09-23T04:48:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2aaff5700 10 client.26197 handle_command_reply: tid=0 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 client.26197 shutdown 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 msgr2=0x559207a12600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8097b90 tx=0x7fe2b809b4e0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 shutdown_connections 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a0ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07400 0x559207a12600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a22280 0x559207a24670 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/3562058035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07d20 0x559207a0f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 >> 172.21.15.5:0/3562058035 conn(0x55920797c260 msgr2=0x559207a03320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.151+0000 7fe2aaff5700 0 client.26197 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.152+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 shutdown_connections 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.152+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/3562058035 wait complete. 2023-09-23T04:48:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.152+0000 7fe2e0aedb80 20 client.26197 trim_cache size 0 max 16384 2023-09-23T04:48:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 Processor -- start 2023-09-23T04:48:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 -- start start 2023-09-23T04:48:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592077db920 con 0x559207a07400 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592079fd860 con 0x559207a12bd0 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.167+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207a19c50 con 0x559207a07d20 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c759e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49790/0 (socket says 172.21.15.5:49790) 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/2033164468 learned_addr learned my addr 172.21.15.5:0/2033164468 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 -- 172.21.15.5:0/2033164468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 msgr2=0x559207a12130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a12130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 -- 172.21.15.5:0/2033164468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 msgr2=0x559207a0a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 -- 172.21.15.5:0/2033164468 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592079ff480 con 0x559207a07d20 2023-09-23T04:48:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.168+0000 7fe2c659c700 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a19710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x559207a24ee0 tx=0x7fe2bc015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3782e6ce0b70d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.169+0000 7fe2adffb700 1 -- 172.21.15.5:0/2033164468 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc024030 con 0x559207a07d20 2023-09-23T04:48:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.169+0000 7fe2adffb700 1 -- 172.21.15.5:0/2033164468 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2bc025040 con 0x559207a07d20 2023-09-23T04:48:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.169+0000 7fe2adffb700 1 -- 172.21.15.5:0/2033164468 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc028730 con 0x559207a07d20 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/2033164468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 msgr2=0x559207a19710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a19710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x559207a24ee0 tx=0x7fe2bc015000 comp rx=0 tx=0).stop 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/2033164468 shutdown_connections 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a12bd0 0x559207a12130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a07400 0x559207a0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/2033164468 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a19710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/2033164468 >> 172.21.15.5:0/2033164468 conn(0x55920797c260 msgr2=0x559207a0ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/2033164468 shutdown_connections 2023-09-23T04:48:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.170+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/2033164468 wait complete. 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.171+0000 7fe2e0aedb80 1 -- start start 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.171+0000 7fe2e0aedb80 1 Processor -- start 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.171+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.171+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a15790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.171+0000 7fe2e0aedb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a12bd0 0x559207a15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207a16410 con 0x559207a12bd0 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207a16580 con 0x559207a07400 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2e0aedb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559207a76040 con 0x559207a07d20 2023-09-23T04:48:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2e0aedb80 10 client.0 _ll_register_callbacks cb 0x7fe2b80bb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a15790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48294/0 (socket says 172.21.15.5:48294) 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c659c700 1 -- 172.21.15.5:0/730485502 learned_addr learned my addr 172.21.15.5:0/730485502 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c759e700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a12bd0 0x559207a15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 msgr2=0x559207a15790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a15790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/730485502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a12bd0 msgr2=0x559207a15cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a12bd0 0x559207a15cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.172+0000 7fe2c6d9d700 1 -- 172.21.15.5:0/730485502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559207785de0 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2c6d9d700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a19480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80bb1a0 tx=0x7fe2b8097aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2cc844d56d4ab7f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b809ad80 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe2b809bd90 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c000f040 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b809b690 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 10 client.26147 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe2bc017070 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2c000f040 con 0x559207a07400 2023-09-23T04:48:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe2b809a5f0 con 0x559207a07400 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 10 client.26147 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 10 client.26147 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 20 client.26147 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a77300 0x559207a79720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 4 client.26147 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.173+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559207953b30 con 0x559207a77300 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2c659c700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a77300 0x559207a79720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2c659c700 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a77300 0x559207a79720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e76479b14d608d2 server_cookie=fb147b958f2d0d65 in_seq=0 out_seq=0 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2aaff5700 10 client.26147 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2aaff5700 1 -- 172.21.15.5:0/730485502 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559207953b30 con 0x559207a77300 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2aaff5700 10 client.26147 handle_command_reply: tid=0 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.174+0000 7fe2e0aedb80 1 client.26147 shutdown 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 msgr2=0x559207a19480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a19480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80bb1a0 tx=0x7fe2b8097aa0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 shutdown_connections 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x559207a77300 0x559207a79720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559207a12bd0 0x559207a15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559207a07400 0x559207a19480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 --2- 172.21.15.5:0/730485502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559207a07d20 0x559207a15790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 >> 172.21.15.5:0/730485502 conn(0x55920797c260 msgr2=0x559207a029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2aaff5700 0 client.26147 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 shutdown_connections 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 1 -- 172.21.15.5:0/730485502 wait complete. 2023-09-23T04:48:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.175+0000 7fe2e0aedb80 20 client.26147 trim_cache size 0 max 16384 2023-09-23T04:48:21.185 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:21.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 msgr2=0x7fe2b00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 0x7fe2b00a3c50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2c819a750 tx=0x7fe2bc009260 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 msgr2=0x7fe2c8199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000c00 tx=0x7fe2b8005740 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 shutdown_connections 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2b00a1790 0x7fe2b00a3c50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2c810d6a0 0x7fe2c819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe2c8073250 0x7fe2c8199140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 --2- 172.21.15.5:0/1446573733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c8073b70 0x7fe2c8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 >> 172.21.15.5:0/1446573733 conn(0x7fe2c80fc990 msgr2=0x7fe2c810f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 shutdown_connections 2023-09-23T04:48:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.176+0000 7fe2cd1ff700 1 -- 172.21.15.5:0/1446573733 wait complete. 2023-09-23T04:48:21.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:48:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 -- 172.21.15.5:0/1672990916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 msgr2=0x7f4700111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 --2- 172.21.15.5:0/1672990916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f4700111930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009990 tx=0x7f46f4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 -- 172.21.15.5:0/1672990916 shutdown_connections 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 --2- 172.21.15.5:0/1672990916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f4700114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 --2- 172.21.15.5:0/1672990916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f47000735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 --2- 172.21.15.5:0/1672990916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f4700111930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.465+0000 7f47061ff700 1 -- 172.21.15.5:0/1672990916 >> 172.21.15.5:0/1672990916 conn(0x7f47000fc990 msgr2=0x7f47000fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.466+0000 7f47061ff700 1 -- 172.21.15.5:0/1672990916 shutdown_connections 2023-09-23T04:48:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.466+0000 7f47061ff700 1 -- 172.21.15.5:0/1672990916 wait complete. 2023-09-23T04:48:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.467+0000 7f47061ff700 1 Processor -- start 2023-09-23T04:48:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.467+0000 7f47061ff700 1 -- start start 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.467+0000 7f47061ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f47061ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f470019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f47061ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f47001a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f47061ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001a2590 con 0x7f4700111e70 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f47061ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001a2700 con 0x7f4700073ae0 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f47061ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001a28c0 con 0x7f47000731c0 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48312/0 (socket says 172.21.15.5:48312) 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 -- 172.21.15.5:0/3296536141 learned_addr learned my addr 172.21.15.5:0/3296536141 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46feffd700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f470019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46fffff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f47001a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 -- 172.21.15.5:0/3296536141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 msgr2=0x7f470019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f470019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 -- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 msgr2=0x7f47001a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f47001a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 -- 172.21.15.5:0/3296536141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40095f0 con 0x7f47000731c0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46fffff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f47001a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.468+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9a0 tx=0x7f46f000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.469+0000 7f46fcff9700 1 -- 172.21.15.5:0/3296536141 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0009790 con 0x7f47000731c0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.469+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47001a2ba0 con 0x7f47000731c0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.469+0000 7f46fcff9700 1 -- 172.21.15.5:0/3296536141 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f000cbd0 con 0x7f47000731c0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.469+0000 7f46fcff9700 1 -- 172.21.15.5:0/3296536141 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0018680 con 0x7f47000731c0 2023-09-23T04:48:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.469+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001aa3e0 con 0x7f47000731c0 2023-09-23T04:48:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.472+0000 7f46fcff9700 1 -- 172.21.15.5:0/3296536141 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f46f000cd40 con 0x7f47000731c0 2023-09-23T04:48:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.473+0000 7f4719ad9b80 1 Processor -- start 2023-09-23T04:48:21.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 -- start start 2023-09-23T04:48:21.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46fcff9700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 0x7f46e80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46fcff9700 1 -- 172.21.15.5:0/3296536141 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f46f0014070 con 0x7f47000731c0 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca34bd0 0x557faca36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac992460 con 0x557faca34bd0 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac983c20 con 0x557faca29d20 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac981540 con 0x557faca29400 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46fffff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 0x7f46e80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48316/0 (socket says 172.21.15.5:48316) 2023-09-23T04:48:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46ff7fe700 1 -- 172.21.15.5:0/415361928 learned_addr learned my addr 172.21.15.5:0/415361928 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.474+0000 7f46feffd700 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 -- 172.21.15.5:0/415361928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 msgr2=0x557faca34690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 -- 172.21.15.5:0/415361928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca34bd0 msgr2=0x557faca36fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca34bd0 0x557faca36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 -- 172.21.15.5:0/415361928 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fac9df8d0 con 0x557faca29400 2023-09-23T04:48:21.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46f0096ee0 tx=0x7f46f009a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c50dc5a9035ddb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46fffff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 0x7f46e80a3c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fd0 tx=0x7f46f8009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46e5ffb700 1 -- 172.21.15.5:0/415361928 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f009a470 con 0x557faca29400 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46e5ffb700 1 -- 172.21.15.5:0/415361928 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f009e5c0 con 0x557faca29400 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.475+0000 7f46e5ffb700 1 -- 172.21.15.5:0/415361928 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0099bf0 con 0x557faca29400 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 -- 172.21.15.5:0/415361928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 msgr2=0x557faca297e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46f0096ee0 tx=0x7f46f009a080 comp rx=0 tx=0).stop 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 -- 172.21.15.5:0/415361928 shutdown_connections 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca34bd0 0x557faca36fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29400 0x557faca297e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/415361928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 -- 172.21.15.5:0/415361928 >> 172.21.15.5:0/415361928 conn(0x557fac99e260 msgr2=0x557faca25790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 -- 172.21.15.5:0/415361928 shutdown_connections 2023-09-23T04:48:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.476+0000 7f4719ad9b80 1 -- 172.21.15.5:0/415361928 wait complete. 2023-09-23T04:48:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.477+0000 7f4719ad9b80 1 -- start start 2023-09-23T04:48:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.477+0000 7f4719ad9b80 1 Processor -- start 2023-09-23T04:48:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca31fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac97ce30 con 0x557faca29400 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac919710 con 0x557faca34bd0 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac9193b0 con 0x557faca29d20 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f4719ad9b80 10 client.0 _ll_register_callbacks cb 0x557fac776800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46feffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca31fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49844/0 (socket says 172.21.15.5:49844) 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46feffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48324/0 (socket says 172.21.15.5:48324) 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.478+0000 7f46ff7fe700 1 -- 172.21.15.5:0/2246939472 learned_addr learned my addr 172.21.15.5:0/2246939472 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 -- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 msgr2=0x557faca31fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca31fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 -- 172.21.15.5:0/2246939472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 msgr2=0x557faca34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 -- 172.21.15.5:0/2246939472 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001aa3e0 con 0x557faca29d20 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46feffd700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b2e0 tx=0x7f46f4005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c6a2df84c3cd25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.479+0000 7f46deffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f401d070 con 0x557faca29d20 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557faca1d250 con 0x557faca29d20 2023-09-23T04:48:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f46f4004500 con 0x557faca29d20 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f40046c0 con 0x557faca29d20 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f4719ad9b80 10 client.26215 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557fac7a7de0 con 0x557faca29d20 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557faca1d250 con 0x557faca29d20 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.480+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f46f400f840 con 0x557faca29d20 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 10 client.26215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 20 client.26215 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca445e0 0x557faca46a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 4 client.26215 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fac975e10 con 0x557faca445e0 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca445e0 0x557faca46a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca445e0 0x557faca46a00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a482b52df0819f3c server_cookie=8c9773baa4023f2a in_seq=0 out_seq=0 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.481+0000 7f46deffd700 10 client.26215 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f46deffd700 1 -- 172.21.15.5:0/2246939472 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557fac975e10 con 0x557faca445e0 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f46deffd700 10 client.26215 handle_command_reply: tid=0 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 client.26215 shutdown 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 msgr2=0x557faca31870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b2e0 tx=0x7f46f4005ad0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 shutdown_connections 2023-09-23T04:48:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca445e0 0x557faca46a00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca31fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2246939472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca31870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 >> 172.21.15.5:0/2246939472 conn(0x557fac99e260 msgr2=0x557faca25320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f46deffd700 0 client.26215 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.482+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 shutdown_connections 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.483+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2246939472 wait complete. 2023-09-23T04:48:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.483+0000 7f4719ad9b80 20 client.26215 trim_cache size 0 max 16384 2023-09-23T04:48:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.498+0000 7f4719ad9b80 1 Processor -- start 2023-09-23T04:48:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.498+0000 7f4719ad9b80 1 -- start start 2023-09-23T04:48:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fac7fd920 con 0x557faca29400 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faca2a620 con 0x557faca29d20 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faca3bc60 con 0x557faca34bd0 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46fffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46fffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48328/0 (socket says 172.21.15.5:48328) 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46fffff700 1 -- 172.21.15.5:0/2538512925 learned_addr learned my addr 172.21.15.5:0/2538512925 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46feffd700 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.499+0000 7f46ff7fe700 1 -- 172.21.15.5:0/2538512925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 msgr2=0x557faca252a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46ff7fe700 1 -- 172.21.15.5:0/2538512925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 msgr2=0x557faca34190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46ff7fe700 1 -- 172.21.15.5:0/2538512925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fac790c60 con 0x557faca29400 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46fffff700 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3b720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c5b0 tx=0x7f46f0099e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9b8a01da959eb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46e67fc700 1 -- 172.21.15.5:0/2538512925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f009a470 con 0x557faca29400 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46e67fc700 1 -- 172.21.15.5:0/2538512925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f0098810 con 0x557faca29400 2023-09-23T04:48:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.500+0000 7f46e67fc700 1 -- 172.21.15.5:0/2538512925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00a25d0 con 0x557faca29400 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2538512925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 msgr2=0x557faca3b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3b720 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c5b0 tx=0x7f46f0099e50 comp rx=0 tx=0).stop 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2538512925 shutdown_connections 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca34bd0 0x557faca252a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3b720 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/2538512925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca29d20 0x557faca34190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2538512925 >> 172.21.15.5:0/2538512925 conn(0x557fac99e260 msgr2=0x557faca40970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2538512925 shutdown_connections 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- 172.21.15.5:0/2538512925 wait complete. 2023-09-23T04:48:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.501+0000 7f4719ad9b80 1 -- start start 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 Processor -- start 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faca97970 con 0x557faca29400 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faca97ab0 con 0x557faca34bd0 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faca97bf0 con 0x557faca29d20 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.502+0000 7f4719ad9b80 10 client.0 _ll_register_callbacks cb 0x7f46f400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46feffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46feffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48332/0 (socket says 172.21.15.5:48332) 2023-09-23T04:48:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46feffd700 1 -- 172.21.15.5:0/261564892 learned_addr learned my addr 172.21.15.5:0/261564892 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46fffff700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 -- 172.21.15.5:0/261564892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 msgr2=0x557faca3b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 -- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 msgr2=0x557faca97430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca97430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 -- 172.21.15.5:0/261564892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f8018440 con 0x557faca29400 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46feffd700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46ff7fe700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3aac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46f0099fd0 tx=0x7f46f0098a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff88f32af20923ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46deffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0099b70 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f46f00a2370 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.503+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46f009e5c0 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f00a2530 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 10 client.16179 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557fac9df8d0 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f009e5c0 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f46f00a9020 con 0x557faca29400 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 20 client.16179 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca98e90 0x557faca9b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 4 client.16179 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fac975b30 con 0x557faca98e90 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.504+0000 7f46feffd700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca98e90 0x557faca9b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f46feffd700 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca98e90 0x557faca9b2b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5baa61ec2f5db87d server_cookie=95321939c3665d59 in_seq=0 out_seq=0 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f46deffd700 10 client.16179 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f46deffd700 1 -- 172.21.15.5:0/261564892 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fac975b30 con 0x557faca98e90 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f46deffd700 10 client.16179 handle_command_reply: tid=0 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 client.16179 shutdown 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 msgr2=0x557faca3aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3aac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46f0099fd0 tx=0x7f46f0098a00 comp rx=0 tx=0).stop 2023-09-23T04:48:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 shutdown_connections 2023-09-23T04:48:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x557faca98e90 0x557faca9b2b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557faca34bd0 0x557faca97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faca29400 0x557faca3aac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 --2- 172.21.15.5:0/261564892 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557faca29d20 0x557faca3b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f46deffd700 0 client.16179 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.505+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 >> 172.21.15.5:0/261564892 conn(0x557fac99e260 msgr2=0x557faca252a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 shutdown_connections 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f4719ad9b80 1 -- 172.21.15.5:0/261564892 wait complete. 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f4719ad9b80 20 client.16179 trim_cache size 0 max 16384 2023-09-23T04:48:21.512 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 msgr2=0x7f46e80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 0x7f46e80a3c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fd0 tx=0x7f46f8009920 comp rx=0 tx=0).stop 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 msgr2=0x7f470019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.506+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9a0 tx=0x7f46f000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 shutdown_connections 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f46e80a1790 0x7f46e80a3c50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4700111e70 0x7f47001a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f47000731c0 0x7f470019d5b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 --2- 172.21.15.5:0/3296536141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4700073ae0 0x7f470019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 >> 172.21.15.5:0/3296536141 conn(0x7f47000fc990 msgr2=0x7f47000fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 shutdown_connections 2023-09-23T04:48:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.507+0000 7f47061ff700 1 -- 172.21.15.5:0/3296536141 wait complete. 2023-09-23T04:48:21.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:48:21.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 -- 172.21.15.5:0/4087813280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 msgr2=0x7fd63010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 --2- 172.21.15.5:0/4087813280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010dd70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd628009f90 tx=0x7fd62800b330 comp rx=0 tx=0).stop 2023-09-23T04:48:21.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 -- 172.21.15.5:0/4087813280 shutdown_connections 2023-09-23T04:48:21.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 --2- 172.21.15.5:0/4087813280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010dd70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 --2- 172.21.15.5:0/4087813280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd6301014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 --2- 172.21.15.5:0/4087813280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd630101a80 0x7fd630105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 -- 172.21.15.5:0/4087813280 >> 172.21.15.5:0/4087813280 conn(0x7fd6300fc990 msgr2=0x7fd6300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 -- 172.21.15.5:0/4087813280 shutdown_connections 2023-09-23T04:48:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.760+0000 7fd6359ff700 1 -- 172.21.15.5:0/4087813280 wait complete. 2023-09-23T04:48:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.761+0000 7fd6359ff700 1 Processor -- start 2023-09-23T04:48:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.761+0000 7fd6359ff700 1 -- start start 2023-09-23T04:48:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6301a2080 0x7fd6301a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301a2b00 con 0x7fd6301061f0 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301a2c70 con 0x7fd6301010d0 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd6359ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301a2e30 con 0x7fd6301a2080 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45158/0 (socket says 172.21.15.5:45158) 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 -- 172.21.15.5:0/897326953 learned_addr learned my addr 172.21.15.5:0/897326953 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6301a2080 0x7fd6301a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 -- 172.21.15.5:0/897326953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6301a2080 msgr2=0x7fd6301a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6301a2080 0x7fd6301a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 -- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 msgr2=0x7fd63010d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62effd700 1 -- 172.21.15.5:0/897326953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6200095f0 con 0x7fd6301010d0 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.762+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd62effd700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd62000ec10 tx=0x7fd62000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd6349fd700 1 -- 172.21.15.5:0/897326953 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000cc90 con 0x7fd6301010d0 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd628009c40 con 0x7fd6301010d0 2023-09-23T04:48:21.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd6349fd700 1 -- 172.21.15.5:0/897326953 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6200044e0 con 0x7fd6301010d0 2023-09-23T04:48:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd6349fd700 1 -- 172.21.15.5:0/897326953 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620010430 con 0x7fd6301010d0 2023-09-23T04:48:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.763+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6301aa6d0 con 0x7fd6301010d0 2023-09-23T04:48:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.767+0000 7fd6349fd700 1 -- 172.21.15.5:0/897326953 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd620003fb0 con 0x7fd6301010d0 2023-09-23T04:48:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.768+0000 7fd649328b80 1 Processor -- start 2023-09-23T04:48:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.768+0000 7fd649328b80 1 -- start start 2023-09-23T04:48:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc4e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc59690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc5bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cbb7460 con 0x56008cc59bd0 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cba8c20 con 0x56008cc4e400 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd649328b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cba6540 con 0x56008cc4ed20 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd62f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc4e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd6349fd700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 0x7fd6180a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd62f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc4e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45168/0 (socket says 172.21.15.5:45168) 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd62f7fe700 1 -- 172.21.15.5:0/1752135175 learned_addr learned my addr 172.21.15.5:0/1752135175 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd62effd700 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc59690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc5bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.769+0000 7fd6349fd700 1 -- 172.21.15.5:0/897326953 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd620014070 con 0x7fd6301010d0 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 0x7fd6180a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 -- 172.21.15.5:0/1752135175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 msgr2=0x56008cc4e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc4e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 -- 172.21.15.5:0/1752135175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 msgr2=0x56008cc5bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc5bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 -- 172.21.15.5:0/1752135175 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56008cc048d0 con 0x56008cc4ed20 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd62effd700 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc59690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6200caff0 tx=0x7fd62008ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7189d30e58d92a66 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd615ffb700 1 -- 172.21.15.5:0/1752135175 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62009a350 con 0x56008cc4ed20 2023-09-23T04:48:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd615ffb700 1 -- 172.21.15.5:0/1752135175 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62009a170 con 0x56008cc4ed20 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.770+0000 7fd615ffb700 1 -- 172.21.15.5:0/1752135175 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200a2ba0 con 0x56008cc4ed20 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 -- 172.21.15.5:0/1752135175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 msgr2=0x56008cc59690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 0x7fd6180a3bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd628015000 tx=0x7fd62800b330 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc59690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd6200caff0 tx=0x7fd62008ff80 comp rx=0 tx=0).stop 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 -- 172.21.15.5:0/1752135175 shutdown_connections 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc4e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 --2- 172.21.15.5:0/1752135175 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc59690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 -- 172.21.15.5:0/1752135175 >> 172.21.15.5:0/1752135175 conn(0x56008cbc3260 msgr2=0x56008cc4a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 -- 172.21.15.5:0/1752135175 shutdown_connections 2023-09-23T04:48:21.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.771+0000 7fd649328b80 1 -- 172.21.15.5:0/1752135175 wait complete. 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.772+0000 7fd649328b80 1 -- start start 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.772+0000 7fd649328b80 1 Processor -- start 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc56fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cba1e30 con 0x56008cc59bd0 2023-09-23T04:48:21.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cb3e710 con 0x56008cc4e400 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cb3e3b0 con 0x56008cc4ed20 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd649328b80 10 client.0 _ll_register_callbacks cb 0x56008c99b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45172/0 (socket says 172.21.15.5:45172) 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd62effd700 1 -- 172.21.15.5:0/2588043919 learned_addr learned my addr 172.21.15.5:0/2588043919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.773+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc56fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 -- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 msgr2=0x56008cc56890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc56890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 -- 172.21.15.5:0/2588043919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 msgr2=0x56008cc56fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc56fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 -- 172.21.15.5:0/2588043919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6301aa6d0 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd62effd700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd62009a510 tx=0x7fd62001fde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ab99f6316568c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd60effd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200a2ba0 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd620098bb0 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56008cc42250 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6200850c0 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd649328b80 10 client.26177 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.774+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56008c9ccde0 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56008cc42250 con 0x56008cc4e400 2023-09-23T04:48:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd62009aa70 con 0x56008cc4e400 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 10 client.26177 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 10 client.26177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 20 client.26177 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008cc69360 0x56008cc6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 4 client.26177 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56008cb9ae10 con 0x56008cc69360 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.775+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008cc69360 0x56008cc6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.776+0000 7fd60effd700 10 client.26177 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.776+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008cc69360 0x56008cc6b780 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbfa937e8feb2c13 server_cookie=eff121cc4002f139 in_seq=0 out_seq=0 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.776+0000 7fd60effd700 1 -- 172.21.15.5:0/2588043919 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56008cb9ae10 con 0x56008cc69360 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.776+0000 7fd60effd700 10 client.26177 handle_command_reply: tid=0 2023-09-23T04:48:21.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.776+0000 7fd649328b80 1 client.26177 shutdown 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 msgr2=0x56008cc59610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd62009a510 tx=0x7fd62001fde0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 shutdown_connections 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008cc56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008cc69360 0x56008cc6b780 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc59610 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 --2- 172.21.15.5:0/2588043919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc56890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 >> 172.21.15.5:0/2588043919 conn(0x56008cbc3260 msgr2=0x56008cc4a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd60effd700 0 client.26177 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 shutdown_connections 2023-09-23T04:48:21.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 1 -- 172.21.15.5:0/2588043919 wait complete. 2023-09-23T04:48:21.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.777+0000 7fd649328b80 20 client.26177 trim_cache size 0 max 16384 2023-09-23T04:48:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 Processor -- start 2023-09-23T04:48:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 -- start start 2023-09-23T04:48:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc4ed20 0x56008cc591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc59bd0 0x56008cc560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008ca22920 con 0x56008cc4ed20 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cc54e90 con 0x56008cc4e400 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.792+0000 7fd649328b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008cc527a0 con 0x56008cc59bd0 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45184/0 (socket says 172.21.15.5:45184) 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 -- 172.21.15.5:0/3682587118 learned_addr learned my addr 172.21.15.5:0/3682587118 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc4ed20 0x56008cc591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62f7fe700 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc59bd0 0x56008cc560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 -- 172.21.15.5:0/3682587118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc59bd0 msgr2=0x56008cc560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc59bd0 0x56008cc560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 -- 172.21.15.5:0/3682587118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc4ed20 msgr2=0x56008cc591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc4ed20 0x56008cc591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 -- 172.21.15.5:0/3682587118 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56008c9b5c60 con 0x56008cc4e400 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd62effd700 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c5b0 tx=0x7fd62009ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5970547fd5b74f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd6167fc700 1 -- 172.21.15.5:0/3682587118 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200b4080 con 0x56008cc4e400 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd6167fc700 1 -- 172.21.15.5:0/3682587118 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6200cdd60 con 0x56008cc4e400 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.793+0000 7fd6167fc700 1 -- 172.21.15.5:0/3682587118 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200a2ba0 con 0x56008cc4e400 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 -- 172.21.15.5:0/3682587118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 msgr2=0x56008cc56f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd62000c5b0 tx=0x7fd62009ae30 comp rx=0 tx=0).stop 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 -- 172.21.15.5:0/3682587118 shutdown_connections 2023-09-23T04:48:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc59bd0 0x56008cc560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc56f90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 --2- 172.21.15.5:0/3682587118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc4ed20 0x56008cc591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 -- 172.21.15.5:0/3682587118 >> 172.21.15.5:0/3682587118 conn(0x56008cbc3260 msgr2=0x56008cc51c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 -- 172.21.15.5:0/3682587118 shutdown_connections 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.794+0000 7fd649328b80 1 -- 172.21.15.5:0/3682587118 wait complete. 2023-09-23T04:48:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.795+0000 7fd649328b80 1 -- start start 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.795+0000 7fd649328b80 1 Processor -- start 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc5e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008ccbc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008ccbc6b0 con 0x56008cc59bd0 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008ccbc7f0 con 0x56008cc4e400 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56008ccbc930 con 0x56008cc4ed20 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd649328b80 10 client.0 _ll_register_callbacks cb 0x7fd624009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:21.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008ccbc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc5e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48398/0 (socket says 172.21.15.5:48398) 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc5e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45198/0 (socket says 172.21.15.5:45198) 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.796+0000 7fd62e7fc700 1 -- 172.21.15.5:0/1065174025 learned_addr learned my addr 172.21.15.5:0/1065174025 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 -- 172.21.15.5:0/1065174025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 msgr2=0x56008cc5e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc5e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 -- 172.21.15.5:0/1065174025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 msgr2=0x56008ccbc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008ccbc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 -- 172.21.15.5:0/1065174025 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56008c9ccde0 con 0x56008cc4ed20 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd62e7fc700 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd62401fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3a8edbde0a3e0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd60effd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624015070 con 0x56008cc4ed20 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd62009a170 con 0x56008cc4ed20 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.797+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd624003680 con 0x56008cc4ed20 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd624003840 con 0x56008cc4ed20 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 10 client.26239 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56008c9f3250 con 0x56008cc4ed20 2023-09-23T04:48:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62009a170 con 0x56008cc4ed20 2023-09-23T04:48:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd624010ad0 con 0x56008cc4ed20 2023-09-23T04:48:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 10 client.26239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:48:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 20 client.26239 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008ccbe0e0 0x56008ccc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 4 client.26239 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56008cb9ab30 con 0x56008ccbe0e0 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.798+0000 7fd62effd700 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008ccbe0e0 0x56008ccc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd62effd700 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008ccbe0e0 0x56008ccc0500 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccfc7c9274be5c9a server_cookie=182e14b491eab421 in_seq=0 out_seq=0 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd60effd700 10 client.26239 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd60effd700 1 -- 172.21.15.5:0/1065174025 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56008cb9ab30 con 0x56008ccbe0e0 2023-09-23T04:48:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd60effd700 10 client.26239 handle_command_reply: tid=0 2023-09-23T04:48:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd649328b80 1 client.26239 shutdown 2023-09-23T04:48:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 msgr2=0x56008cc5ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.799+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd62401fcd0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 shutdown_connections 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008cc59bd0 0x56008ccbc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x56008ccbe0e0 0x56008ccc0500 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56008cc4e400 0x56008cc5e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 --2- 172.21.15.5:0/1065174025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56008cc4ed20 0x56008cc5ee30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 >> 172.21.15.5:0/1065174025 conn(0x56008cbc3260 msgr2=0x56008cc499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd60effd700 0 client.26239 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 shutdown_connections 2023-09-23T04:48:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.800+0000 7fd649328b80 1 -- 172.21.15.5:0/1065174025 wait complete. 2023-09-23T04:48:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd649328b80 20 client.26239 trim_cache size 0 max 16384 2023-09-23T04:48:21.810 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 msgr2=0x7fd6180a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 0x7fd6180a3bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd628015000 tx=0x7fd62800b330 comp rx=0 tx=0).stop 2023-09-23T04:48:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 msgr2=0x7fd63010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd62000ec10 tx=0x7fd62000ef20 comp rx=0 tx=0).stop 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 shutdown_connections 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd6180a1710 0x7fd6180a3bd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd6301a2080 0x7fd6301a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6301010d0 0x7fd63010cd00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 --2- 172.21.15.5:0/897326953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6301061f0 0x7fd63010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 >> 172.21.15.5:0/897326953 conn(0x7fd6300fc990 msgr2=0x7fd6300fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 shutdown_connections 2023-09-23T04:48:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:21.801+0000 7fd6359ff700 1 -- 172.21.15.5:0/897326953 wait complete. 2023-09-23T04:48:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:21 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/246406922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:21 smithi005 ceph-mon[97638]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2023-09-23T04:48:21.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:48:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:21 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/246406922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:21 smithi118 ceph-mon[134297]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2023-09-23T04:48:21.925 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74686 7.39 MB/sec warmup 57 sec latency 22.861 ms 2023-09-23T04:48:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:21 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/246406922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:21 smithi069 ceph-mon[133931]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.072+0000 7fd41b7ff700 1 -- 172.21.15.5:0/3866415438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4140ffeb0 msgr2=0x7fd414100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.072+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/3866415438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4140ffeb0 0x7fd414100290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd410009990 tx=0x7fd410009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.072+0000 7fd41b7ff700 1 -- 172.21.15.5:0/3866415438 shutdown_connections 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.072+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/3866415438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4141052b0 0x7fd414105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.072+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/3866415438 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4140ffeb0 0x7fd414100290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.073+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/3866415438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141007d0 0x7fd414104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.073+0000 7fd41b7ff700 1 -- 172.21.15.5:0/3866415438 >> 172.21.15.5:0/3866415438 conn(0x7fd414076f30 msgr2=0x7fd414077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.073+0000 7fd41b7ff700 1 -- 172.21.15.5:0/3866415438 shutdown_connections 2023-09-23T04:48:22.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.073+0000 7fd41b7ff700 1 -- 172.21.15.5:0/3866415438 wait complete. 2023-09-23T04:48:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.074+0000 7fd41b7ff700 1 Processor -- start 2023-09-23T04:48:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.074+0000 7fd41b7ff700 1 -- start start 2023-09-23T04:48:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.074+0000 7fd41b7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 0x7fd414110990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.074+0000 7fd41b7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.074+0000 7fd41b7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141052b0 0x7fd41410cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41b7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41410d240 con 0x7fd4140ffeb0 2023-09-23T04:48:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41b7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41410d3b0 con 0x7fd4141052b0 2023-09-23T04:48:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41b7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41410d570 con 0x7fd4141007d0 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd418d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 0x7fd414110990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd418d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48414/0 (socket says 172.21.15.5:48414) 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd418d9a700 1 -- 172.21.15.5:0/1032087655 learned_addr learned my addr 172.21.15.5:0/1032087655 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd419d9c700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141052b0 0x7fd41410cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 -- 172.21.15.5:0/1032087655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 msgr2=0x7fd414110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 -- 172.21.15.5:0/1032087655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141052b0 msgr2=0x7fd41410cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141052b0 0x7fd41410cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd41959b700 1 -- 172.21.15.5:0/1032087655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009520 con 0x7fd4140ffeb0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.075+0000 7fd418d9a700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd41959b700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 0x7fd414110990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd410006010 tx=0x7fd410005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd40a7fc700 1 -- 172.21.15.5:0/1032087655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41001d070 con 0x7fd4140ffeb0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4100095f0 con 0x7fd4140ffeb0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd40a7fc700 1 -- 172.21.15.5:0/1032087655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd410022470 con 0x7fd4140ffeb0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd40a7fc700 1 -- 172.21.15.5:0/1032087655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410017650 con 0x7fd4140ffeb0 2023-09-23T04:48:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.076+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd414111850 con 0x7fd4140ffeb0 2023-09-23T04:48:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.080+0000 7fd40a7fc700 1 -- 172.21.15.5:0/1032087655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd410022850 con 0x7fd4140ffeb0 2023-09-23T04:48:22.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd42f1c3b80 1 Processor -- start 2023-09-23T04:48:22.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd42f1c3b80 1 -- start start 2023-09-23T04:48:22.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd40a7fc700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 0x7fd4000a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0ea17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.082+0000 7fd40a7fc700 1 -- 172.21.15.5:0/1032087655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd4100cea00 con 0x7fd4140ffeb0 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eaefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0e0a460 con 0x5645a0eacbd0 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0dfbc20 con 0x5645a0ea1400 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0df9540 con 0x5645a0ea1d20 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd419d9c700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 0x7fd4000a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48418/0 (socket says 172.21.15.5:48418) 2023-09-23T04:48:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 -- 172.21.15.5:0/716324354 learned_addr learned my addr 172.21.15.5:0/716324354 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd41959b700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0ea17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd419d9c700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eaefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 -- 172.21.15.5:0/716324354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 msgr2=0x5645a0ea17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0ea17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 -- 172.21.15.5:0/716324354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 msgr2=0x5645a0eaefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eaefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd418d9a700 1 -- 172.21.15.5:0/716324354 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645a0e578d0 con 0x5645a0ea1d20 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.083+0000 7fd419d9c700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eaefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.084+0000 7fd418d9a700 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd404011810 tx=0x7fd404011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6a065901e83035 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.084+0000 7fd419d9c700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 0x7fd4000a3cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009ec0 tx=0x7fd40c0089d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.084+0000 7fd3ff7fe700 1 -- 172.21.15.5:0/716324354 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404004020 con 0x5645a0ea1d20 2023-09-23T04:48:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.084+0000 7fd3ff7fe700 1 -- 172.21.15.5:0/716324354 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd404003680 con 0x5645a0ea1d20 2023-09-23T04:48:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.084+0000 7fd3ff7fe700 1 -- 172.21.15.5:0/716324354 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400f900 con 0x5645a0ea1d20 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/716324354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 msgr2=0x5645a0eac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd404011810 tx=0x7fd404011b20 comp rx=0 tx=0).stop 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/716324354 shutdown_connections 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eaefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0ea17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/716324354 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0eac690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/716324354 >> 172.21.15.5:0/716324354 conn(0x5645a0e16260 msgr2=0x5645a0e9d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/716324354 shutdown_connections 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.085+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/716324354 wait complete. 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.086+0000 7fd42f1c3b80 1 -- start start 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.086+0000 7fd42f1c3b80 1 Processor -- start 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0ea9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0df4e30 con 0x5645a0eacbd0 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0d91710 con 0x5645a0ea1400 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0d913b0 con 0x5645a0ea1d20 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd42f1c3b80 10 client.0 _ll_register_callbacks cb 0x5645a0bee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45234/0 (socket says 172.21.15.5:45234) 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 -- 172.21.15.5:0/3745782332 learned_addr learned my addr 172.21.15.5:0/3745782332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd418d9a700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd419d9c700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0ea9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 -- 172.21.15.5:0/3745782332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 msgr2=0x5645a0ea97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 -- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 msgr2=0x5645a0ea9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0ea9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.087+0000 7fd41959b700 1 -- 172.21.15.5:0/3745782332 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd404011440 con 0x5645a0ea1400 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd41959b700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd410098030 tx=0x7fd410009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0a6a71ed7ddbb1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd3fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410086070 con 0x5645a0ea1400 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645a0e95250 con 0x5645a0ea1400 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd410005790 con 0x5645a0ea1400 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41009bcc0 con 0x5645a0ea1400 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd42f1c3b80 10 client.26195 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5645a0c1fde0 con 0x5645a0ea1400 2023-09-23T04:48:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.088+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645a0e95250 con 0x5645a0ea1400 2023-09-23T04:48:22.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd41009abd0 con 0x5645a0ea1400 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 10 client.26195 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 20 client.26195 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0ebc1c0 0x5645a0ebe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 4 client.26195 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645a0dede10 con 0x5645a0ebc1c0 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd418d9a700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0ebc1c0 0x5645a0ebe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd418d9a700 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0ebc1c0 0x5645a0ebe5e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe780c3efd04981 server_cookie=8a04cb66d97ad9e4 in_seq=0 out_seq=0 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.089+0000 7fd3fcff9700 10 client.26195 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd3fcff9700 1 -- 172.21.15.5:0/3745782332 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5645a0dede10 con 0x5645a0ebc1c0 2023-09-23T04:48:22.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd3fcff9700 10 client.26195 handle_command_reply: tid=0 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd42f1c3b80 1 client.26195 shutdown 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 msgr2=0x5645a0eac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd410098030 tx=0x7fd410009310 comp rx=0 tx=0).stop 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 shutdown_connections 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.090+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0ebc1c0 0x5645a0ebe5e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0ea9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eac5e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/3745782332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd3fcff9700 0 client.26195 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 >> 172.21.15.5:0/3745782332 conn(0x5645a0e16260 msgr2=0x5645a0e9d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 shutdown_connections 2023-09-23T04:48:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/3745782332 wait complete. 2023-09-23T04:48:22.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.091+0000 7fd42f1c3b80 20 client.26195 trim_cache size 0 max 16384 2023-09-23T04:48:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 Processor -- start 2023-09-23T04:48:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 -- start start 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0c75920 con 0x5645a0eacbd0 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0eaa800 con 0x5645a0ea1d20 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.107+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0e9e5b0 con 0x5645a0ea1400 2023-09-23T04:48:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48434/0 (socket says 172.21.15.5:48434) 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 -- 172.21.15.5:0/1093591209 learned_addr learned my addr 172.21.15.5:0/1093591209 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd418d9a700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 -- 172.21.15.5:0/1093591209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1d20 msgr2=0x5645a0ea97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd419d9c700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 -- 172.21.15.5:0/1093591209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 msgr2=0x5645a0eac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 -- 172.21.15.5:0/1093591209 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645a0c08c60 con 0x5645a0ea1400 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd419d9c700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:48:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.108+0000 7fd41959b700 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd41009b550 tx=0x7fd41009b1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe34f1f0135800f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd3fffff700 1 -- 172.21.15.5:0/1093591209 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410005790 con 0x5645a0ea1400 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd3fffff700 1 -- 172.21.15.5:0/1093591209 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41009bcc0 con 0x5645a0ea1400 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd3fffff700 1 -- 172.21.15.5:0/1093591209 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41009ad10 con 0x5645a0ea1400 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/1093591209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 msgr2=0x5645a0ea42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd41009b550 tx=0x7fd41009b1e0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/1093591209 shutdown_connections 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0eacbd0 0x5645a0eac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0ea1400 0x5645a0ea42a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/1093591209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1d20 0x5645a0ea97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.109+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/1093591209 >> 172.21.15.5:0/1093591209 conn(0x5645a0e16260 msgr2=0x5645a0ea90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.110+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/1093591209 shutdown_connections 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.110+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/1093591209 wait complete. 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.110+0000 7fd42f1c3b80 1 -- start start 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.110+0000 7fd42f1c3b80 1 Processor -- start 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eb33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0ea1d20 0x5645a0eb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0eb3cc0 con 0x5645a0ea1d20 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0eb2180 con 0x5645a0ea1400 2023-09-23T04:48:22.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a0eb2340 con 0x5645a0eacbd0 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd42f1c3b80 10 client.0 _ll_register_callbacks cb 0x5645a0bee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd41959b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eb33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48450/0 (socket says 172.21.15.5:48450) 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 -- 172.21.15.5:0/628072369 learned_addr learned my addr 172.21.15.5:0/628072369 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd418d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0ea1d20 0x5645a0eb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 -- 172.21.15.5:0/628072369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 msgr2=0x5645a0eb33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eb33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 -- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0ea1d20 msgr2=0x5645a0eb1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0ea1d20 0x5645a0eb1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.111+0000 7fd419d9c700 1 -- 172.21.15.5:0/628072369 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645a0c1fde0 con 0x5645a0eacbd0 2023-09-23T04:48:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.112+0000 7fd419d9c700 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd40c004510 tx=0x7fd40c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f41506d7547358 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.112+0000 7fd3fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.112+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00e070 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.112+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd40c004820 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.112+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd404003680 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd40c0188e0 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 10 client.26257 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5645a0c46250 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd404003680 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd40c02c070 con 0x5645a0eacbd0 2023-09-23T04:48:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 10 client.26257 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 20 client.26257 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0f11330 0x5645a0f13750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 4 client.26257 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.113+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645a0dedb30 con 0x5645a0f11330 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd41959b700 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0f11330 0x5645a0f13750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd41959b700 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0f11330 0x5645a0f13750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77c44a6810fc49e server_cookie=38272110c13bfed0 in_seq=0 out_seq=0 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd3fcff9700 10 client.26257 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd3fcff9700 1 -- 172.21.15.5:0/628072369 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645a0dedb30 con 0x5645a0f11330 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd3fcff9700 10 client.26257 handle_command_reply: tid=0 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.114+0000 7fd42f1c3b80 1 client.26257 shutdown 2023-09-23T04:48:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 msgr2=0x5645a0eb1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd40c004510 tx=0x7fd40c00b040 comp rx=0 tx=0).stop 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 shutdown_connections 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645a0f11330 0x5645a0f13750 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5645a0eacbd0 0x5645a0eb1c40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5645a0ea1400 0x5645a0eb33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd3fcff9700 0 client.26257 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 --2- 172.21.15.5:0/628072369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645a0ea1d20 0x5645a0eb1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 >> 172.21.15.5:0/628072369 conn(0x5645a0e16260 msgr2=0x5645a0ea8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 shutdown_connections 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 1 -- 172.21.15.5:0/628072369 wait complete. 2023-09-23T04:48:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.115+0000 7fd42f1c3b80 20 client.26257 trim_cache size 0 max 16384 2023-09-23T04:48:22.121 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:22.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 msgr2=0x7fd4000a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 0x7fd4000a3cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009ec0 tx=0x7fd40c0089d0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 msgr2=0x7fd414110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 0x7fd414110990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd410006010 tx=0x7fd410005720 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 shutdown_connections 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd4000a1810 0x7fd4000a3cd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4141052b0 0x7fd41410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4140ffeb0 0x7fd414110990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 --2- 172.21.15.5:0/1032087655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd4141007d0 0x7fd414110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 >> 172.21.15.5:0/1032087655 conn(0x7fd414076f30 msgr2=0x7fd414102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 shutdown_connections 2023-09-23T04:48:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:22.116+0000 7fd41b7ff700 1 -- 172.21.15.5:0/1032087655 wait complete. 2023-09-23T04:48:22.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 75727 7.32 MB/sec warmup 58 sec latency 41.085 ms 2023-09-23T04:48:23.138 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:48:23.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:48:23.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 -- 172.21.15.5:0/393825441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa8101970 msgr2=0x7f7aa81059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/393825441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa8101970 0x7f7aa81059b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009990 tx=0x7f7a9c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 -- 172.21.15.5:0/393825441 shutdown_connections 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/393825441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa81060e0 0x7f7aa810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/393825441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/393825441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa8101970 0x7f7aa81059b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.379+0000 7f7aad9ff700 1 -- 172.21.15.5:0/393825441 >> 172.21.15.5:0/393825441 conn(0x7f7aa80fc860 msgr2=0x7f7aa80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.380+0000 7f7aad9ff700 1 -- 172.21.15.5:0/393825441 shutdown_connections 2023-09-23T04:48:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.380+0000 7f7aad9ff700 1 -- 172.21.15.5:0/393825441 wait complete. 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 Processor -- start 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 -- start start 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa8196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 0x7f7aa819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa819c270 con 0x7f7aa8100fc0 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa819c3e0 con 0x7f7aa81060e0 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.381+0000 7f7aad9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa819c5a0 con 0x7f7aa819b7f0 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 0x7f7aa819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45280/0 (socket says 172.21.15.5:45280) 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 -- 172.21.15.5:0/934065518 learned_addr learned my addr 172.21.15.5:0/934065518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa8196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 -- 172.21.15.5:0/934065518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 msgr2=0x7f7aa819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 0x7f7aa819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 -- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 msgr2=0x7f7aa8196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa8196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 -- 172.21.15.5:0/934065518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0095f0 con 0x7f7aa81060e0 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa8196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 0x7f7aa819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.382+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c004730 tx=0x7f7a9c004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.383+0000 7f7aac9fd700 1 -- 172.21.15.5:0/934065518 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01d070 con 0x7f7aa81060e0 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.383+0000 7f7aac9fd700 1 -- 172.21.15.5:0/934065518 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c004c90 con 0x7f7aa81060e0 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.383+0000 7f7aac9fd700 1 -- 172.21.15.5:0/934065518 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0054b0 con 0x7f7aa81060e0 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.383+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa81a3aa0 con 0x7f7aa81060e0 2023-09-23T04:48:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.383+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81a3ec0 con 0x7f7aa81060e0 2023-09-23T04:48:23.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.386+0000 7f7aac9fd700 1 -- 172.21.15.5:0/934065518 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7a9c005610 con 0x7f7aa81060e0 2023-09-23T04:48:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.388+0000 7f7aac9fd700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 0x7f7a900a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.388+0000 7f7ac13ebb80 1 Processor -- start 2023-09-23T04:48:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7aac9fd700 1 -- 172.21.15.5:0/934065518 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7a9c0ce7f0 con 0x7f7aa81060e0 2023-09-23T04:48:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 -- start start 2023-09-23T04:48:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 0x7f7a900a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d974860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d982040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d8dd4e0 con 0x56001d97fc50 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d8ceca0 con 0x56001d974da0 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d8cc5c0 con 0x56001d974480 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45286/0 (socket says 172.21.15.5:45286) 2023-09-23T04:48:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.389+0000 7f7aa67fc700 1 -- 172.21.15.5:0/2379314834 learned_addr learned my addr 172.21.15.5:0/2379314834 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d974860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d982040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 -- 172.21.15.5:0/2379314834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 msgr2=0x56001d974860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d974860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 -- 172.21.15.5:0/2379314834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 msgr2=0x56001d982040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d982040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 -- 172.21.15.5:0/2379314834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001d92a950 con 0x56001d974da0 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d974860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 0x7f7a900a3c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009d00 tx=0x7f7a98009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c097e80 tx=0x7f7a9c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0901f6ea9c259ec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7a8dffb700 1 -- 172.21.15.5:0/2379314834 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c09b410 con 0x56001d974da0 2023-09-23T04:48:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7a8dffb700 1 -- 172.21.15.5:0/2379314834 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c09f560 con 0x56001d974da0 2023-09-23T04:48:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.390+0000 7f7a8dffb700 1 -- 172.21.15.5:0/2379314834 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c09a900 con 0x56001d974da0 2023-09-23T04:48:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2379314834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 msgr2=0x56001d97f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c097e80 tx=0x7f7a9c009310 comp rx=0 tx=0).stop 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2379314834 shutdown_connections 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d982040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d974860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.391+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2379314834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d97f710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.392+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2379314834 >> 172.21.15.5:0/2379314834 conn(0x56001d8e92e0 msgr2=0x56001d970810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.392+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2379314834 shutdown_connections 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.392+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2379314834 wait complete. 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.392+0000 7f7ac13ebb80 1 -- start start 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.392+0000 7f7ac13ebb80 1 Processor -- start 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d8c7eb0 con 0x56001d974da0 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d864790 con 0x56001d97fc50 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d864430 con 0x56001d974480 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7ac13ebb80 10 client.0 _ll_register_callbacks cb 0x56001d6c1880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45290/0 (socket says 172.21.15.5:45290) 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7aa77fe700 1 -- 172.21.15.5:0/3132651970 learned_addr learned my addr 172.21.15.5:0/3132651970 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.393+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 -- 172.21.15.5:0/3132651970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 msgr2=0x56001d980030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 -- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 msgr2=0x56001d980570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 -- 172.21.15.5:0/3132651970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81a3ec0 con 0x56001d97fc50 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004630 tx=0x7f7aa00049f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283c2b62eb98e030 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7a86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0007dd0 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56001d9682d0 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.394+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7aa0016460 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa0016620 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 10 client.26213 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56001d6f2e60 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56001d9682d0 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7aa0014040 con 0x56001d97fc50 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 10 client.26213 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 20 client.26213 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d993800 0x56001d995c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 4 client.26213 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.395+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56001d8c0e90 con 0x56001d993800 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.396+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d993800 0x56001d995c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.396+0000 7f7a86ffd700 10 client.26213 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.396+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d993800 0x56001d995c20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc67e4a1a030c5e server_cookie=31af0cd3702aa5e6 in_seq=0 out_seq=0 2023-09-23T04:48:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.397+0000 7f7a86ffd700 1 -- 172.21.15.5:0/3132651970 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56001d8c0e90 con 0x56001d993800 2023-09-23T04:48:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.397+0000 7f7a86ffd700 10 client.26213 handle_command_reply: tid=0 2023-09-23T04:48:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.397+0000 7f7ac13ebb80 1 client.26213 shutdown 2023-09-23T04:48:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.397+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 msgr2=0x56001d98b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.397+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004630 tx=0x7f7aa00049f0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 shutdown_connections 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d993800 0x56001d995c20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d97fc50 0x56001d98b0b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/3132651970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7a86ffd700 0 client.26213 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 >> 172.21.15.5:0/3132651970 conn(0x56001d8e92e0 msgr2=0x56001d9703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 shutdown_connections 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/3132651970 wait complete. 2023-09-23T04:48:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.398+0000 7f7ac13ebb80 20 client.26213 trim_cache size 0 max 16384 2023-09-23T04:48:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 Processor -- start 2023-09-23T04:48:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 -- start start 2023-09-23T04:48:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d98a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d7489a0 con 0x56001d97fc50 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d97ee30 con 0x56001d974da0 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.417+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d97f9a0 con 0x56001d974480 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d98a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45304/0 (socket says 172.21.15.5:45304) 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 -- 172.21.15.5:0/1155949772 learned_addr learned my addr 172.21.15.5:0/1155949772 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 -- 172.21.15.5:0/1155949772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 msgr2=0x56001d980030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 -- 172.21.15.5:0/1155949772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 msgr2=0x56001d98a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d98a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 -- 172.21.15.5:0/1155949772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001d99a850 con 0x56001d974da0 2023-09-23T04:48:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.418+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c09b3a0 tx=0x7f7a9c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1350f19a81083ba0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7a8e7fc700 1 -- 172.21.15.5:0/1155949772 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0a3710 con 0x56001d974da0 2023-09-23T04:48:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7a8e7fc700 1 -- 172.21.15.5:0/1155949772 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c00fbc0 con 0x56001d974da0 2023-09-23T04:48:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7a8e7fc700 1 -- 172.21.15.5:0/1155949772 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0ce620 con 0x56001d974da0 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/1155949772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 msgr2=0x56001d980570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c09b3a0 tx=0x7f7a9c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/1155949772 shutdown_connections 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d97fc50 0x56001d98a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d974480 0x56001d980030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/1155949772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974da0 0x56001d980570 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.419+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/1155949772 >> 172.21.15.5:0/1155949772 conn(0x56001d8e92e0 msgr2=0x56001d979630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.420+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/1155949772 shutdown_connections 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.420+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/1155949772 wait complete. 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.420+0000 7f7ac13ebb80 1 -- start start 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.420+0000 7f7ac13ebb80 1 Processor -- start 2023-09-23T04:48:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d981320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d97fc50 0x56001d981860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d981e30 con 0x56001d974da0 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d981fa0 con 0x56001d974480 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001d982160 con 0x56001d97fc50 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7ac13ebb80 10 client.0 _ll_register_callbacks cb 0x7f7aa0003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7aa6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7aa6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45320/0 (socket says 172.21.15.5:45320) 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7aa6ffd700 1 -- 172.21.15.5:0/2866591513 learned_addr learned my addr 172.21.15.5:0/2866591513 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.421+0000 7f7aa77fe700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d97fc50 0x56001d981860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d981320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 -- 172.21.15.5:0/2866591513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d97fc50 msgr2=0x56001d981860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d97fc50 0x56001d981860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 -- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 msgr2=0x56001d981320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d981320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 -- 172.21.15.5:0/2866591513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a98012450 con 0x56001d974480 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7aa6ffd700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a98011a50 tx=0x7f7a980047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4aca75bd3c5b97 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7a86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98025050 con 0x56001d974480 2023-09-23T04:48:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a9c09f560 con 0x56001d974480 2023-09-23T04:48:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.422+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7a98018030 con 0x56001d974480 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a98004030 con 0x56001d974480 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 10 client.26219 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56001d92a950 con 0x56001d974480 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c09f560 con 0x56001d974480 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7a9802f030 con 0x56001d974480 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 10 client.26219 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 20 client.26219 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d9e8d50 0x56001d9eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 4 client.26219 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.423+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56001d8c0bb0 con 0x56001d9e8d50 2023-09-23T04:48:23.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.424+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d9e8d50 0x56001d9eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.424+0000 7f7aa67fc700 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d9e8d50 0x56001d9eb170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6565d5f46c258dd server_cookie=de353670f6410415 in_seq=0 out_seq=0 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.424+0000 7f7a86ffd700 10 client.26219 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.425+0000 7f7a86ffd700 1 -- 172.21.15.5:0/2866591513 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56001d8c0bb0 con 0x56001d9e8d50 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.425+0000 7f7a86ffd700 10 client.26219 handle_command_reply: tid=0 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.425+0000 7f7ac13ebb80 1 client.26219 shutdown 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 msgr2=0x56001d984f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a98011a50 tx=0x7f7a980047c0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 shutdown_connections 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56001d9e8d50 0x56001d9eb170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56001d97fc50 0x56001d981860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56001d974480 0x56001d984f10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7a86ffd700 0 client.26219 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 --2- 172.21.15.5:0/2866591513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56001d974da0 0x56001d981320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 >> 172.21.15.5:0/2866591513 conn(0x56001d8e92e0 msgr2=0x56001d979630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 shutdown_connections 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.426+0000 7f7ac13ebb80 1 -- 172.21.15.5:0/2866591513 wait complete. 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.427+0000 7f7ac13ebb80 20 client.26219 trim_cache size 0 max 16384 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:48:23.430 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "86dcab4b-2f53-4435-9462-b11839c883cc", 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.427+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 msgr2=0x7f7a900a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.427+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 0x7f7a900a3c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009d00 tx=0x7f7a98009260 comp rx=0 tx=0).stop 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.427+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 msgr2=0x7f7aa8197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.427+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c004730 tx=0x7f7a9c004760 comp rx=0 tx=0).stop 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 shutdown_connections 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a900a1790 0x7f7a900a3c50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7aa819b7f0 0x7f7aa819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8100fc0 0x7f7aa8196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 --2- 172.21.15.5:0/934065518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7aa81060e0 0x7f7aa8197390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 >> 172.21.15.5:0/934065518 conn(0x7f7aa80fc860 msgr2=0x7f7aa80fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 shutdown_connections 2023-09-23T04:48:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.428+0000 7f7aad9ff700 1 -- 172.21.15.5:0/934065518 wait complete. 2023-09-23T04:48:23.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.686+0000 7ff35a9ff700 1 -- 172.21.15.5:0/3908998632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff354073250 msgr2=0x7ff354073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.686+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/3908998632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff354073250 0x7ff354073630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff344009990 tx=0x7ff344009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 -- 172.21.15.5:0/3908998632 shutdown_connections 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/3908998632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff35410d580 0x7ff35410f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/3908998632 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff354073250 0x7ff354073630 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/3908998632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073b70 0x7ff35410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 -- 172.21.15.5:0/3908998632 >> 172.21.15.5:0/3908998632 conn(0x7ff3540fc850 msgr2=0x7ff3540fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 -- 172.21.15.5:0/3908998632 shutdown_connections 2023-09-23T04:48:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.687+0000 7ff35a9ff700 1 -- 172.21.15.5:0/3908998632 wait complete. 2023-09-23T04:48:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.688+0000 7ff35a9ff700 1 Processor -- start 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.688+0000 7ff35a9ff700 1 -- start start 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.688+0000 7ff35a9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.688+0000 7ff35a9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 0x7ff3541995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.688+0000 7ff35a9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 0x7ff35419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff35a9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35419e080 con 0x7ff354073250 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff35a9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35419e1f0 con 0x7ff354073b70 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff35a9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35419e3b0 con 0x7ff35410d580 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50056/0 (socket says 172.21.15.5:50056) 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 -- 172.21.15.5:0/1491025620 learned_addr learned my addr 172.21.15.5:0/1491025620 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff358f9c700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 0x7ff35419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 0x7ff3541995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 -- 172.21.15.5:0/1491025620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 msgr2=0x7ff35419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 0x7ff35419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 -- 172.21.15.5:0/1491025620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 msgr2=0x7ff3541995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 0x7ff3541995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff353fff700 1 -- 172.21.15.5:0/1491025620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440095f0 con 0x7ff354073250 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff358f9c700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 0x7ff35419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.689+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 0x7ff3541995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff353fff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff344005910 tx=0x7ff344004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff3517fa700 1 -- 172.21.15.5:0/1491025620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34401d070 con 0x7ff354073250 2023-09-23T04:48:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff3517fa700 1 -- 172.21.15.5:0/1491025620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff344022470 con 0x7ff354073250 2023-09-23T04:48:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35419e630 con 0x7ff354073250 2023-09-23T04:48:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff3517fa700 1 -- 172.21.15.5:0/1491025620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344017650 con 0x7ff354073250 2023-09-23T04:48:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.690+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3541a5ed0 con 0x7ff354073250 2023-09-23T04:48:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.695+0000 7ff3517fa700 1 -- 172.21.15.5:0/1491025620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff3440225e0 con 0x7ff354073250 2023-09-23T04:48:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.696+0000 7ff36e2f3b80 1 Processor -- start 2023-09-23T04:48:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 -- start start 2023-09-23T04:48:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484b400 0x5599a484b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a4856bd0 0x5599a4858fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a47b4460 con 0x5599a484b400 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a47a5c20 con 0x5599a4856bd0 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a47a3540 con 0x5599a484bd20 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff353fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff353fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48498/0 (socket says 172.21.15.5:48498) 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff353fff700 1 -- 172.21.15.5:0/4052489532 learned_addr learned my addr 172.21.15.5:0/4052489532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.697+0000 7ff358f9c700 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484b400 0x5599a484b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 -- 172.21.15.5:0/4052489532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a4856bd0 msgr2=0x5599a4858fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a4856bd0 0x5599a4858fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 -- 172.21.15.5:0/4052489532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484b400 msgr2=0x5599a484b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484b400 0x5599a484b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 -- 172.21.15.5:0/4052489532 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a48018d0 con 0x5599a484bd20 2023-09-23T04:48:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff3517fa700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 0x7ff33c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff353fff700 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3440cbb90 tx=0x7ff3440d4d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5cc78b92ee5a063 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff3517fa700 1 -- 172.21.15.5:0/1491025620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff3440ce8b0 con 0x7ff354073250 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 0x7ff33c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff33a7fc700 1 -- 172.21.15.5:0/4052489532 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34409b2a0 con 0x5599a484bd20 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.698+0000 7ff33a7fc700 1 -- 172.21.15.5:0/4052489532 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff34409b010 con 0x5599a484bd20 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff33a7fc700 1 -- 172.21.15.5:0/4052489532 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34409ae30 con 0x5599a484bd20 2023-09-23T04:48:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/4052489532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 msgr2=0x5599a4856690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 0x7ff33c0a3bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097c0 tx=0x7ff348009750 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3440cbb90 tx=0x7ff3440d4d60 comp rx=0 tx=0).stop 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/4052489532 shutdown_connections 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a4856bd0 0x5599a4858fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484b400 0x5599a484b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/4052489532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a4856690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.699+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/4052489532 >> 172.21.15.5:0/4052489532 conn(0x5599a47c0260 msgr2=0x5599a4847790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.700+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/4052489532 shutdown_connections 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.700+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/4052489532 wait complete. 2023-09-23T04:48:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.700+0000 7ff36e2f3b80 1 -- start start 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.700+0000 7ff36e2f3b80 1 Processor -- start 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484bd20 0x5599a48574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a4856bd0 0x5599a4862030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a479ee30 con 0x5599a484bd20 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a473b710 con 0x5599a484b400 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a473b3b0 con 0x5599a4856bd0 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff36e2f3b80 10 client.0 _ll_register_callbacks cb 0x5599a4598800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff353fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff353fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45346/0 (socket says 172.21.15.5:45346) 2023-09-23T04:48:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff353fff700 1 -- 172.21.15.5:0/1066092760 learned_addr learned my addr 172.21.15.5:0/1066092760 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484bd20 0x5599a48574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.701+0000 7ff358f9c700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a4856bd0 0x5599a4862030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 -- 172.21.15.5:0/1066092760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a4856bd0 msgr2=0x5599a4862030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a4856bd0 0x5599a4862030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 -- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484bd20 msgr2=0x5599a48574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484bd20 0x5599a48574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 -- 172.21.15.5:0/1066092760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3541a5ed0 con 0x5599a484b400 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff353fff700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff3440228b0 tx=0x7ff344022b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697e5bf0279417d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff3337fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34409b2a0 con 0x5599a484b400 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff3440d4e30 con 0x5599a484b400 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a483f250 con 0x5599a484b400 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34409ae30 con 0x5599a484b400 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff36e2f3b80 10 client.26228 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.702+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5599a45c9de0 con 0x5599a484b400 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599a483f250 con 0x5599a484b400 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff3440ced20 con 0x5599a484b400 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 20 client.26228 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a4865f00 0x5599a4868320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 4 client.26228 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599a4797e10 con 0x5599a4865f00 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.703+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a4865f00 0x5599a4868320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.704+0000 7ff3537fe700 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a4865f00 0x5599a4868320 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef459013c31150 server_cookie=31f337026574e4c2 in_seq=0 out_seq=0 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.704+0000 7ff3337fe700 10 client.26228 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.704+0000 7ff3337fe700 1 -- 172.21.15.5:0/1066092760 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599a4797e10 con 0x5599a4865f00 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.704+0000 7ff3337fe700 10 client.26228 handle_command_reply: tid=0 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 client.26228 shutdown 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 msgr2=0x5599a4856fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff3440228b0 tx=0x7ff344022b30 comp rx=0 tx=0).stop 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 shutdown_connections 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a4865f00 0x5599a4868320 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a4856bd0 0x5599a4862030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4856fb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff3337fe700 0 client.26228 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/1066092760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a484bd20 0x5599a48574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 >> 172.21.15.5:0/1066092760 conn(0x5599a47c0260 msgr2=0x5599a4847380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 shutdown_connections 2023-09-23T04:48:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/1066092760 wait complete. 2023-09-23T04:48:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.705+0000 7ff36e2f3b80 20 client.26228 trim_cache size 0 max 16384 2023-09-23T04:48:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.723+0000 7ff36e2f3b80 1 Processor -- start 2023-09-23T04:48:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.723+0000 7ff36e2f3b80 1 -- start start 2023-09-23T04:48:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4851090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a4852ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a461f920 con 0x5599a4856bd0 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a483f140 con 0x5599a484b400 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a4855ea0 con 0x5599a484bd20 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff358f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a4852ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48520/0 (socket says 172.21.15.5:48520) 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 -- 172.21.15.5:0/3551909847 learned_addr learned my addr 172.21.15.5:0/3551909847 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff353fff700 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4851090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 -- 172.21.15.5:0/3551909847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 msgr2=0x5599a4851090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4851090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 -- 172.21.15.5:0/3551909847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 msgr2=0x5599a4852ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a4852ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.724+0000 7ff3537fe700 1 -- 172.21.15.5:0/3551909847 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a4856690 con 0x5599a484bd20 2023-09-23T04:48:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.725+0000 7ff3537fe700 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff348009f70 tx=0x7ff348017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c766fc3357558f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.725+0000 7ff33affd700 1 -- 172.21.15.5:0/3551909847 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348022040 con 0x5599a484bd20 2023-09-23T04:48:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.725+0000 7ff33affd700 1 -- 172.21.15.5:0/3551909847 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348024040 con 0x5599a484bd20 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.725+0000 7ff33affd700 1 -- 172.21.15.5:0/3551909847 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348032a40 con 0x5599a484bd20 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/3551909847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 msgr2=0x5599a48515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff348009f70 tx=0x7ff348017000 comp rx=0 tx=0).stop 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/3551909847 shutdown_connections 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a4852ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a4851090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/3551909847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a48515d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/3551909847 >> 172.21.15.5:0/3551909847 conn(0x5599a47c0260 msgr2=0x5599a48555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/3551909847 shutdown_connections 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/3551909847 wait complete. 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.726+0000 7ff36e2f3b80 1 -- start start 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 Processor -- start 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a485a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a485a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a485aaa0 con 0x5599a4856bd0 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a485abe0 con 0x5599a484b400 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a485ada0 con 0x5599a484bd20 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff36e2f3b80 10 client.0 _ll_register_callbacks cb 0x5599a4598800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff353fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff353fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45366/0 (socket says 172.21.15.5:45366) 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff353fff700 1 -- 172.21.15.5:0/2134383990 learned_addr learned my addr 172.21.15.5:0/2134383990 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.727+0000 7ff3537fe700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a485a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff358f9c700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a485a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 -- 172.21.15.5:0/2134383990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 msgr2=0x5599a485a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a485a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 -- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 msgr2=0x5599a485a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a485a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 -- 172.21.15.5:0/2134383990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff348017070 con 0x5599a484b400 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff358f9c700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a485a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff353fff700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff34409b180 tx=0x7ff344009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94cc68909c9a8f71 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff3337fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34409b2a0 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff34409bbd0 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.728+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a4862cf0 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34409ae30 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 10 client.26234 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5599a45b2c60 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599a4862cf0 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff34409a9c0 con 0x5599a484b400 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 10 client.26234 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 20 client.26234 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a48bac70 0x5599a48bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 4 client.26234 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7ff34409b010 con 0x5599a48bac70 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.729+0000 7ff3537fe700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a48bac70 0x5599a48bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff3537fe700 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a48bac70 0x5599a48bd060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d6d9e05799e7f5 server_cookie=895e92621a40cec7 in_seq=0 out_seq=0 2023-09-23T04:48:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff3337fe700 10 client.26234 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff3337fe700 1 -- 172.21.15.5:0/2134383990 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7ff34409b010 con 0x5599a48bac70 2023-09-23T04:48:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff3337fe700 10 client.26234 handle_command_reply: tid=0 2023-09-23T04:48:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 client.26234 shutdown 2023-09-23T04:48:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 msgr2=0x5599a485bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff34409b180 tx=0x7ff344009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 shutdown_connections 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599a48bac70 0x5599a48bd060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599a4856bd0 0x5599a485a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff3337fe700 0 client.26234 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599a484b400 0x5599a485bcf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.730+0000 7ff36e2f3b80 1 --2- 172.21.15.5:0/2134383990 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599a484bd20 0x5599a485a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.731+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 >> 172.21.15.5:0/2134383990 conn(0x5599a47c0260 msgr2=0x5599a4851090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.731+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 shutdown_connections 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.731+0000 7ff36e2f3b80 1 -- 172.21.15.5:0/2134383990 wait complete. 2023-09-23T04:48:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.731+0000 7ff36e2f3b80 20 client.26234 trim_cache size 0 max 16384 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "86dcab4b-2f53-4435-9462-b11839c883cc": { 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "86dcab4b-2f53-4435-9462-b11839c883cc", 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:48:23.738 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 msgr2=0x7ff33c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 0x7ff33c0a3bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff3480097c0 tx=0x7ff348009750 comp rx=0 tx=0).stop 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 msgr2=0x7ff3541990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff344005910 tx=0x7ff344004710 comp rx=0 tx=0).stop 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 shutdown_connections 2023-09-23T04:48:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff33c0a1710 0x7ff33c0a3bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff35410d580 0x7ff35419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354073250 0x7ff3541990a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 --2- 172.21.15.5:0/1491025620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff354073b70 0x7ff3541995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 >> 172.21.15.5:0/1491025620 conn(0x7ff3540fc850 msgr2=0x7ff35410f740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 shutdown_connections 2023-09-23T04:48:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:23.732+0000 7ff35a9ff700 1 -- 172.21.15.5:0/1491025620 wait complete. 2023-09-23T04:48:23.754 INFO:tasks.cephfs.filesystem:scrub status for tag:86dcab4b-2f53-4435-9462-b11839c883cc - {'path': '/', 'tag': '86dcab4b-2f53-4435-9462-b11839c883cc', 'options': 'recursive,force'} 2023-09-23T04:48:23.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:23 smithi005 ceph-mon[97638]: pgmap v66: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2023-09-23T04:48:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:23 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:23 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:23 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:23 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:23.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78205 7.46 MB/sec warmup 59 sec latency 24.803 ms 2023-09-23T04:48:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:23 smithi069 ceph-mon[133931]: pgmap v66: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2023-09-23T04:48:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:23 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:23 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:23 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:23 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:23 smithi118 ceph-mon[134297]: pgmap v66: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 297 op/s 2023-09-23T04:48:24.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:23 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:23 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:23 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:24.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:23 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 79318 7.40 MB/sec warmup 60 sec latency 20.308 ms 2023-09-23T04:48:24.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:24.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:24 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:24 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:48:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:24 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:25.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81854 7.55 MB/sec warmup 61 sec latency 10.821 ms 2023-09-23T04:48:25.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:25 smithi069 ceph-mon[133931]: pgmap v67: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:25 smithi005 ceph-mon[97638]: pgmap v67: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:25 smithi118 ceph-mon[134297]: pgmap v67: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:26.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 83087 7.51 MB/sec warmup 62 sec latency 10.903 ms 2023-09-23T04:48:26.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:48:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:48:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:48:27.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85296 7.60 MB/sec warmup 63 sec latency 26.444 ms 2023-09-23T04:48:27.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:27 smithi069 ceph-mon[133931]: pgmap v68: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:27 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:27 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:27 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:27.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:27 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:27 smithi005 ceph-mon[97638]: pgmap v68: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:27 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:27 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:27 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:27 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:27 smithi118 ceph-mon[134297]: pgmap v68: 129 pgs: 129 active+clean; 65 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2023-09-23T04:48:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:27 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:27 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:27 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:27 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 86415 7.53 MB/sec warmup 64 sec latency 18.830 ms 2023-09-23T04:48:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:28 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:48:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:28 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:48:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:28 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:28 smithi069 ceph-mon[133931]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-09-23T04:48:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:28 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:48:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:28 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:48:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:28 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:28 smithi005 ceph-mon[97638]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-09-23T04:48:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:28 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:48:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:28 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:48:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:28 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:48:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:28 smithi118 ceph-mon[134297]: pgmap v69: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2023-09-23T04:48:29.926 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88951 7.67 MB/sec warmup 65 sec latency 11.160 ms 2023-09-23T04:48:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:30 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:48:30.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:30 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:48:30.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:30 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:48:30.927 INFO:tasks.workunit.client.0.smithi005.stdout: 1 90111 7.64 MB/sec warmup 66 sec latency 16.938 ms 2023-09-23T04:48:31.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:31 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:48:31.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:31 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:48:31.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:31 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:48:31.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:31 smithi005 ceph-mon[97638]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2023-09-23T04:48:31.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:31 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:48:31.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:31 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:48:31.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:31 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:48:31.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:31 smithi118 ceph-mon[134297]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2023-09-23T04:48:31.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:31 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:48:31.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:31 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:48:31.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:31 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:48:31.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:31 smithi069 ceph-mon[133931]: pgmap v70: 129 pgs: 129 active+clean; 69 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2023-09-23T04:48:31.927 INFO:tasks.workunit.client.0.smithi005.stdout: 1 92503 7.73 MB/sec warmup 67 sec latency 11.557 ms 2023-09-23T04:48:32.927 INFO:tasks.workunit.client.0.smithi005.stdout: 1 93670 7.69 MB/sec warmup 68 sec latency 53.730 ms 2023-09-23T04:48:33.927 INFO:tasks.workunit.client.0.smithi005.stdout: 1 95954 7.77 MB/sec warmup 69 sec latency 72.323 ms 2023-09-23T04:48:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:33 smithi069 ceph-mon[133931]: pgmap v71: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 312 op/s 2023-09-23T04:48:34.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:33 smithi005 ceph-mon[97638]: pgmap v71: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 312 op/s 2023-09-23T04:48:34.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:33 smithi118 ceph-mon[134297]: pgmap v71: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 312 op/s 2023-09-23T04:48:34.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 97219 7.75 MB/sec warmup 70 sec latency 10.515 ms 2023-09-23T04:48:35.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99502 7.82 MB/sec warmup 71 sec latency 24.271 ms 2023-09-23T04:48:35.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:35 smithi069 ceph-mon[133931]: pgmap v72: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:35 smithi005 ceph-mon[97638]: pgmap v72: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:36.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:35 smithi118 ceph-mon[134297]: pgmap v72: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:36.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 100619 7.76 MB/sec warmup 72 sec latency 18.383 ms 2023-09-23T04:48:37.929 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102253 7.78 MB/sec warmup 73 sec latency 325.440 ms 2023-09-23T04:48:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:37 smithi069 ceph-mon[133931]: pgmap v73: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:38.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:37 smithi005 ceph-mon[97638]: pgmap v73: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:37 smithi118 ceph-mon[134297]: pgmap v73: 129 pgs: 129 active+clean; 76 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2023-09-23T04:48:38.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 103280 7.78 MB/sec warmup 74 sec latency 46.912 ms 2023-09-23T04:48:39.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 105352 7.77 MB/sec warmup 75 sec latency 11.106 ms 2023-09-23T04:48:39.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:39 smithi069 ceph-mon[133931]: pgmap v74: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2023-09-23T04:48:40.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:39 smithi005 ceph-mon[97638]: pgmap v74: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2023-09-23T04:48:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:39 smithi118 ceph-mon[134297]: pgmap v74: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 373 op/s 2023-09-23T04:48:40.928 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106913 7.82 MB/sec warmup 76 sec latency 13.326 ms 2023-09-23T04:48:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:41 smithi005 ceph-mon[97638]: pgmap v75: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2023-09-23T04:48:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:41 smithi118 ceph-mon[134297]: pgmap v75: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2023-09-23T04:48:41.929 INFO:tasks.workunit.client.0.smithi005.stdout: 1 109004 7.83 MB/sec warmup 77 sec latency 19.824 ms 2023-09-23T04:48:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:41 smithi069 ceph-mon[133931]: pgmap v75: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 268 op/s 2023-09-23T04:48:42.929 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110525 7.87 MB/sec warmup 78 sec latency 13.277 ms 2023-09-23T04:48:43.929 INFO:tasks.workunit.client.0.smithi005.stdout: 1 112759 7.91 MB/sec warmup 79 sec latency 11.187 ms 2023-09-23T04:48:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:43 smithi069 ceph-mon[133931]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 414 op/s 2023-09-23T04:48:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:43 smithi005 ceph-mon[97638]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 414 op/s 2023-09-23T04:48:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:43 smithi118 ceph-mon[134297]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 414 op/s 2023-09-23T04:48:44.930 INFO:tasks.workunit.client.0.smithi005.stdout: 1 114210 7.91 MB/sec warmup 80 sec latency 10.532 ms 2023-09-23T04:48:45.930 INFO:tasks.workunit.client.0.smithi005.stdout: 1 116442 7.96 MB/sec warmup 81 sec latency 48.732 ms 2023-09-23T04:48:45.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:45 smithi069 ceph-mon[133931]: pgmap v77: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:45 smithi005 ceph-mon[97638]: pgmap v77: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:45 smithi118 ceph-mon[134297]: pgmap v77: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:46.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117768 7.95 MB/sec warmup 82 sec latency 72.138 ms 2023-09-23T04:48:47.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 120101 8.01 MB/sec warmup 83 sec latency 18.029 ms 2023-09-23T04:48:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:47 smithi069 ceph-mon[133931]: pgmap v78: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:48.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:47 smithi005 ceph-mon[97638]: pgmap v78: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:47 smithi118 ceph-mon[134297]: pgmap v78: 129 pgs: 129 active+clean; 71 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 337 op/s 2023-09-23T04:48:48.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 121357 7.99 MB/sec warmup 84 sec latency 63.671 ms 2023-09-23T04:48:49.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 123649 8.04 MB/sec warmup 85 sec latency 24.044 ms 2023-09-23T04:48:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:49 smithi069 ceph-mon[133931]: pgmap v79: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-09-23T04:48:50.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:49 smithi005 ceph-mon[97638]: pgmap v79: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-09-23T04:48:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:49 smithi118 ceph-mon[134297]: pgmap v79: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2023-09-23T04:48:50.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124985 8.03 MB/sec warmup 86 sec latency 11.521 ms 2023-09-23T04:48:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:51 smithi005 ceph-mon[97638]: pgmap v80: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2023-09-23T04:48:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:51 smithi118 ceph-mon[134297]: pgmap v80: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2023-09-23T04:48:51.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 127202 8.08 MB/sec warmup 87 sec latency 11.595 ms 2023-09-23T04:48:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:51 smithi069 ceph-mon[133931]: pgmap v80: 129 pgs: 129 active+clean; 76 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2023-09-23T04:48:52.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 128554 8.07 MB/sec warmup 88 sec latency 11.077 ms 2023-09-23T04:48:53.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:48:53.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130916 8.12 MB/sec warmup 89 sec latency 18.102 ms 2023-09-23T04:48:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 -- 172.21.15.5:0/1831859655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c1020c0 msgr2=0x7efd9c1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 --2- 172.21.15.5:0/1831859655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c1020c0 0x7efd9c1024a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd98009990 tx=0x7efd98009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 -- 172.21.15.5:0/1831859655 shutdown_connections 2023-09-23T04:48:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 --2- 172.21.15.5:0/1831859655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c10b490 0x7efd9c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 --2- 172.21.15.5:0/1831859655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c1020c0 0x7efd9c1024a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 --2- 172.21.15.5:0/1831859655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 0x7efd9c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 -- 172.21.15.5:0/1831859655 >> 172.21.15.5:0/1831859655 conn(0x7efd9c076f30 msgr2=0x7efd9c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 -- 172.21.15.5:0/1831859655 shutdown_connections 2023-09-23T04:48:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.978+0000 7efda45ff700 1 -- 172.21.15.5:0/1831859655 wait complete. 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.979+0000 7efda45ff700 1 Processor -- start 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.979+0000 7efda45ff700 1 -- start start 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 0x7efd9c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 0x7efd9c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c19e240 con 0x7efd9c1029e0 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c19e3b0 con 0x7efd9c10b490 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda45ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c19e570 con 0x7efd9c1020c0 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda239b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 0x7efd9c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 0x7efd9c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda239b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 0x7efd9c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34970/0 (socket says 172.21.15.5:34970) 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52230/0 (socket says 172.21.15.5:52230) 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda239b700 1 -- 172.21.15.5:0/689657434 learned_addr learned my addr 172.21.15.5:0/689657434 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 -- 172.21.15.5:0/689657434 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 msgr2=0x7efd9c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 0x7efd9c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 -- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 msgr2=0x7efd9c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 0x7efd9c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.980+0000 7efda2b9c700 1 -- 172.21.15.5:0/689657434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980095f0 con 0x7efd9c10b490 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efda2b9c700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd9400efc0 tx=0x7efd9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efd937fe700 1 -- 172.21.15.5:0/689657434 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94018070 con 0x7efd9c10b490 2023-09-23T04:48:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efd937fe700 1 -- 172.21.15.5:0/689657434 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9400f040 con 0x7efd9c10b490 2023-09-23T04:48:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c19e7f0 con 0x7efd9c10b490 2023-09-23T04:48:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efd937fe700 1 -- 172.21.15.5:0/689657434 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9400cd40 con 0x7efd9c10b490 2023-09-23T04:48:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.981+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c1a6090 con 0x7efd9c10b490 2023-09-23T04:48:53.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.985+0000 7efd937fe700 1 -- 172.21.15.5:0/689657434 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7efd94003a00 con 0x7efd9c10b490 2023-09-23T04:48:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.987+0000 7efd937fe700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 0x7efd880a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.987+0000 7efdb7f1ab80 1 Processor -- start 2023-09-23T04:48:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.987+0000 7efd937fe700 1 -- 172.21.15.5:0/689657434 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efd940cd200 con 0x7efd9c10b490 2023-09-23T04:48:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.987+0000 7efdb7f1ab80 1 -- start start 2023-09-23T04:48:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.987+0000 7efda239b700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 0x7efd880a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f922f460 con 0x55e9f92d1bd0 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f9220c20 con 0x55e9f92c6d20 2023-09-23T04:48:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f921e540 con 0x55e9f92c6400 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efda2b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52244/0 (socket says 172.21.15.5:52244) 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efda1b9a700 1 -- 172.21.15.5:0/3728934566 learned_addr learned my addr 172.21.15.5:0/3728934566 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.988+0000 7efda239b700 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 -- 172.21.15.5:0/3728934566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 msgr2=0x55e9f92c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 -- 172.21.15.5:0/3728934566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 msgr2=0x55e9f92d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda239b700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 0x7efd880a3c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd98006010 tx=0x7efd9800b610 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 -- 172.21.15.5:0/3728934566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f927c8d0 con 0x55e9f92c6d20 2023-09-23T04:48:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efda1b9a700 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009d20 tx=0x7efd8c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf79f3ed61167076 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efd90bfe700 1 -- 172.21.15.5:0/3728934566 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c019630 con 0x55e9f92c6d20 2023-09-23T04:48:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efd90bfe700 1 -- 172.21.15.5:0/3728934566 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c019c50 con 0x55e9f92c6d20 2023-09-23T04:48:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.989+0000 7efd90bfe700 1 -- 172.21.15.5:0/3728934566 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00ec40 con 0x55e9f92c6d20 2023-09-23T04:48:53.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:53 smithi069 ceph-mon[133931]: pgmap v81: 129 pgs: 129 active+clean; 80 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 347 op/s 2023-09-23T04:48:53.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/3728934566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 msgr2=0x55e9f92d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009d20 tx=0x7efd8c011720 comp rx=0 tx=0).stop 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/3728934566 shutdown_connections 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/3728934566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d1690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/3728934566 >> 172.21.15.5:0/3728934566 conn(0x55e9f923b260 msgr2=0x55e9f92c2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/3728934566 shutdown_connections 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.990+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/3728934566 wait complete. 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 -- start start 2023-09-23T04:48:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 Processor -- start 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f9219e30 con 0x55e9f92d1bd0 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f91b6710 con 0x55e9f92c6d20 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f91b63b0 con 0x55e9f92c6400 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.991+0000 7efdb7f1ab80 10 client.0 _ll_register_callbacks cb 0x55e9f9013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52248/0 (socket says 172.21.15.5:52248) 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 -- 172.21.15.5:0/2403812730 learned_addr learned my addr 172.21.15.5:0/2403812730 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda2b9c700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda239b700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 -- 172.21.15.5:0/2403812730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 msgr2=0x55e9f92ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 -- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 msgr2=0x55e9f92cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 -- 172.21.15.5:0/2403812730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c009980 con 0x55e9f92c6d20 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efda1b9a700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005c50 tx=0x7efd8c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1fe025f0eba214 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.992+0000 7efd85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c019840 con 0x55e9f92c6d20 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f92ba250 con 0x55e9f92c6d20 2023-09-23T04:48:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efd8c00e3b0 con 0x55e9f92c6d20 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8c005ce0 con 0x55e9f92c6d20 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efdb7f1ab80 10 client.26246 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e9f9044de0 con 0x55e9f92c6d20 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.993+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9f92ba250 con 0x55e9f92c6d20 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efd8c004890 con 0x55e9f92c6d20 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 10 client.26246 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 20 client.26246 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f92e16d0 0x55e9f92e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 4 client.26246 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f9212e10 con 0x55e9f92e16d0 2023-09-23T04:48:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.994+0000 7efda239b700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f92e16d0 0x55e9f92e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.995+0000 7efda239b700 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f92e16d0 0x55e9f92e3af0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d4408314611ecb server_cookie=19e0d68b08c78160 in_seq=0 out_seq=0 2023-09-23T04:48:53.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.995+0000 7efd85ffb700 10 client.26246 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.995+0000 7efd85ffb700 1 -- 172.21.15.5:0/2403812730 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9f9212e10 con 0x55e9f92e16d0 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.995+0000 7efd85ffb700 10 client.26246 handle_command_reply: tid=0 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 client.26246 shutdown 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 msgr2=0x55e9f92cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd8c005c50 tx=0x7efd8c00ee70 comp rx=0 tx=0).stop 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 shutdown_connections 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f92e16d0 0x55e9f92e3af0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efd85ffb700 0 client.26246 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/2403812730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92cee60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 >> 172.21.15.5:0/2403812730 conn(0x55e9f923b260 msgr2=0x55e9f92c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 shutdown_connections 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/2403812730 wait complete. 2023-09-23T04:48:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:53.996+0000 7efdb7f1ab80 20 client.26246 trim_cache size 0 max 16384 2023-09-23T04:48:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 Processor -- start 2023-09-23T04:48:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 -- start start 2023-09-23T04:48:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f909a920 con 0x55e9f92d1bd0 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f92c6a30 con 0x55e9f92c6d20 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f92c97e0 con 0x55e9f92c6400 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda2b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda239b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52262/0 (socket says 172.21.15.5:52262) 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda2b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57922/0 (socket says 172.21.15.5:57922) 2023-09-23T04:48:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.017+0000 7efda1b9a700 1 -- 172.21.15.5:0/1749212374 learned_addr learned my addr 172.21.15.5:0/1749212374 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 -- 172.21.15.5:0/1749212374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 msgr2=0x55e9f92c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 -- 172.21.15.5:0/1749212374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 msgr2=0x55e9f92cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 -- 172.21.15.5:0/1749212374 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f92e86e0 con 0x55e9f92c6400 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda1b9a700 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda2b9c700 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efda239b700 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92cdc30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd9800b980 tx=0x7efd98025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14653108ad57dc87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efd913ff700 1 -- 172.21.15.5:0/1749212374 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9803a070 con 0x55e9f92c6400 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efd913ff700 1 -- 172.21.15.5:0/1749212374 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9801e040 con 0x55e9f92c6400 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.018+0000 7efd913ff700 1 -- 172.21.15.5:0/1749212374 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98004a30 con 0x55e9f92c6400 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/1749212374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 msgr2=0x55e9f92cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92cdc30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd9800b980 tx=0x7efd98025000 comp rx=0 tx=0).stop 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/1749212374 shutdown_connections 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92d1bd0 0x55e9f92cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92c6400 0x55e9f92cdc30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/1749212374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/1749212374 >> 172.21.15.5:0/1749212374 conn(0x55e9f923b260 msgr2=0x55e9f92cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/1749212374 shutdown_connections 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.019+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/1749212374 wait complete. 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 -- start start 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 Processor -- start 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92c6400 0x55e9f92d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.020+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f92d7700 con 0x55e9f92c6400 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f92d7870 con 0x55e9f92c6d20 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efdb7f1ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f92d7a30 con 0x55e9f92d1bd0 2023-09-23T04:48:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efdb7f1ab80 10 client.0 _ll_register_callbacks cb 0x7efd94009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92c6400 0x55e9f92d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52264/0 (socket says 172.21.15.5:52264) 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 -- 172.21.15.5:0/283790868 learned_addr learned my addr 172.21.15.5:0/283790868 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda2b9c700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 -- 172.21.15.5:0/283790868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92d1bd0 msgr2=0x55e9f92d70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 -- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92c6400 msgr2=0x55e9f92d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92c6400 0x55e9f92d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 -- 172.21.15.5:0/283790868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9f9044de0 con 0x55e9f92c6d20 2023-09-23T04:48:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efda1b9a700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd8c004c20 tx=0x7efd8c0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa46c7fe70cfc6eb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.021+0000 7efd85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00eee0 con 0x55e9f92c6d20 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd8c019c50 con 0x55e9f92c6d20 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efd8c000e40 con 0x55e9f92c6d20 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8c00e6e0 con 0x55e9f92c6d20 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 10 client.26252 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7efd9c1a6090 con 0x55e9f92c6d20 2023-09-23T04:48:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c019c50 con 0x55e9f92c6d20 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efd8c0198d0 con 0x55e9f92c6d20 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 10 client.26252 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 20 client.26252 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f9336840 0x55e9f9338c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 4 client.26252 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.022+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9f9212b30 con 0x55e9f9336840 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efda239b700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f9336840 0x55e9f9338c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efda239b700 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f9336840 0x55e9f9338c60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d335bff792fc6 server_cookie=8101d988bcdc0ad4 in_seq=0 out_seq=0 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efd85ffb700 10 client.26252 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efd85ffb700 1 -- 172.21.15.5:0/283790868 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e9f9212b30 con 0x55e9f9336840 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efd85ffb700 10 client.26252 handle_command_reply: tid=0 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.023+0000 7efdb7f1ab80 1 client.26252 shutdown 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 msgr2=0x55e9f92d6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd8c004c20 tx=0x7efd8c0092d0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 shutdown_connections 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55e9f9336840 0x55e9f9338c60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e9f92d1bd0 0x55e9f92d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9f92c6400 0x55e9f92d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 --2- 172.21.15.5:0/283790868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f92c6d20 0x55e9f92d6b70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 >> 172.21.15.5:0/283790868 conn(0x55e9f923b260 msgr2=0x55e9f92d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efd85ffb700 0 client.26252 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 shutdown_connections 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 1 -- 172.21.15.5:0/283790868 wait complete. 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.024+0000 7efdb7f1ab80 20 client.26252 trim_cache size 0 max 16384 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:48:54.029 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 msgr2=0x7efd880a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 0x7efd880a3c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd98006010 tx=0x7efd9800b610 comp rx=0 tx=0).stop 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 msgr2=0x7efd9c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd9400efc0 tx=0x7efd9400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 shutdown_connections 2023-09-23T04:48:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efd880a1790 0x7efd880a3c50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9c10b490 0x7efd9c19dc00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efd9c1020c0 0x7efd9c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 --2- 172.21.15.5:0/689657434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c1029e0 0x7efd9c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 >> 172.21.15.5:0/689657434 conn(0x7efd9c076f30 msgr2=0x7efd9c107270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 shutdown_connections 2023-09-23T04:48:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.025+0000 7efda45ff700 1 -- 172.21.15.5:0/689657434 wait complete. 2023-09-23T04:48:54.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:53 smithi005 ceph-mon[97638]: pgmap v81: 129 pgs: 129 active+clean; 80 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 347 op/s 2023-09-23T04:48:54.044 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:48:54.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:48:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:53 smithi118 ceph-mon[134297]: pgmap v81: 129 pgs: 129 active+clean; 80 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 347 op/s 2023-09-23T04:48:54.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 -- 172.21.15.5:0/3251680260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff930101a60 msgr2=0x7ff930105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 --2- 172.21.15.5:0/3251680260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff930101a60 0x7ff930105aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff924009990 tx=0x7ff924009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 -- 172.21.15.5:0/3251680260 shutdown_connections 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 --2- 172.21.15.5:0/3251680260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301061d0 0x7ff93010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 --2- 172.21.15.5:0/3251680260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301010b0 0x7ff930101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 --2- 172.21.15.5:0/3251680260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff930101a60 0x7ff930105aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 -- 172.21.15.5:0/3251680260 >> 172.21.15.5:0/3251680260 conn(0x7ff9300fc930 msgr2=0x7ff9300fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 -- 172.21.15.5:0/3251680260 shutdown_connections 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.270+0000 7ff9355ff700 1 -- 172.21.15.5:0/3251680260 wait complete. 2023-09-23T04:48:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.272+0000 7ff9355ff700 1 Processor -- start 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.272+0000 7ff9355ff700 1 -- start start 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.272+0000 7ff9355ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301010b0 0x7ff930199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.272+0000 7ff9355ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff9355ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 0x7ff93019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff9355ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93019e560 con 0x7ff9301010b0 2023-09-23T04:48:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff9355ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93019e6d0 con 0x7ff93019dae0 2023-09-23T04:48:54.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff9355ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93019e890 con 0x7ff9301061d0 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35012/0 (socket says 172.21.15.5:35012) 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301010b0 0x7ff930199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 -- 172.21.15.5:0/474316584 learned_addr learned my addr 172.21.15.5:0/474316584 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92f7fe700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 0x7ff93019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 -- 172.21.15.5:0/474316584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 msgr2=0x7ff93019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 0x7ff93019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 -- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301010b0 msgr2=0x7ff930199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301010b0 0x7ff930199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92e7fc700 1 -- 172.21.15.5:0/474316584 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240095f0 con 0x7ff9301061d0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.273+0000 7ff92f7fe700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 0x7ff93019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff92e7fc700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff924004740 tx=0x7ff924009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92401d070 con 0x7ff9301061d0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff924004830 con 0x7ff9301061d0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924005280 con 0x7ff9301061d0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9301a5d90 con 0x7ff9301061d0 2023-09-23T04:48:54.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.274+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9301a61b0 con 0x7ff9301061d0 2023-09-23T04:48:54.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.275+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff930067ca0 con 0x7ff9301061d0 2023-09-23T04:48:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.278+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff92400fb00 con 0x7ff9301061d0 2023-09-23T04:48:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.279+0000 7ff917fff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 0x7ff9180a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.279+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff9240ce680 con 0x7ff9301061d0 2023-09-23T04:48:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.279+0000 7ff92effd700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 0x7ff9180a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.280+0000 7ff92effd700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 0x7ff9180a3c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff920009c60 tx=0x7ff920009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.281+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7ff924097580 con 0x7ff9301061d0 2023-09-23T04:48:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.579+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff930101c70 con 0x7ff9301061d0 2023-09-23T04:48:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.580+0000 7ff917fff700 1 -- 172.21.15.5:0/474316584 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11356 (secure 0 0 0) 0x7ff9240053e0 con 0x7ff9301061d0 2023-09-23T04:48:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:48:54.581 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:47:40.065282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:48:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 msgr2=0x7ff9180a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 0x7ff9180a3c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff920009c60 tx=0x7ff920009260 comp rx=0 tx=0).stop 2023-09-23T04:48:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 msgr2=0x7ff930199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff924004740 tx=0x7ff924009580 comp rx=0 tx=0).stop 2023-09-23T04:48:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 shutdown_connections 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9180a1790 0x7ff9180a3c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff93019dae0 0x7ff93019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9301010b0 0x7ff930199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 --2- 172.21.15.5:0/474316584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9301061d0 0x7ff930199680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 >> 172.21.15.5:0/474316584 conn(0x7ff9300fc930 msgr2=0x7ff9300fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 shutdown_connections 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.583+0000 7ff9355ff700 1 -- 172.21.15.5:0/474316584 wait complete. 2023-09-23T04:48:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2023-09-23T04:48:54.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 -- 172.21.15.5:0/35860299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 msgr2=0x7f684010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 --2- 172.21.15.5:0/35860299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f684010ffb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f683800b0a0 tx=0x7f683800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 -- 172.21.15.5:0/35860299 shutdown_connections 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 --2- 172.21.15.5:0/35860299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f684010ffb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 --2- 172.21.15.5:0/35860299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 0x7f68401036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 --2- 172.21.15.5:0/35860299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6840103cc0 0x7f6840107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 -- 172.21.15.5:0/35860299 >> 172.21.15.5:0/35860299 conn(0x7f68400feb50 msgr2=0x7f6840100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 -- 172.21.15.5:0/35860299 shutdown_connections 2023-09-23T04:48:54.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.827+0000 7f6847fff700 1 -- 172.21.15.5:0/35860299 wait complete. 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 Processor -- start 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 -- start start 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 0x7f6840199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684019dcb0 0x7f684019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019e730 con 0x7f6840103310 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019e8a0 con 0x7f684019dcb0 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.828+0000 7f6847fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019ea60 con 0x7f6840108430 2023-09-23T04:48:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35028/0 (socket says 172.21.15.5:35028) 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 -- 172.21.15.5:0/2055219255 learned_addr learned my addr 172.21.15.5:0/2055219255 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f6845d9b700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 0x7f6840199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684659c700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684019dcb0 0x7f684019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 -- 172.21.15.5:0/2055219255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684019dcb0 msgr2=0x7f684019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684019dcb0 0x7f684019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 -- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 msgr2=0x7f6840199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 0x7f6840199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 -- 172.21.15.5:0/2055219255 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009d00 con 0x7f6840108430 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.829+0000 7f684559a700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f683000e970 tx=0x7f683000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.830+0000 7f6836ffd700 1 -- 172.21.15.5:0/2055219255 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830009790 con 0x7f6840108430 2023-09-23T04:48:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.830+0000 7f6836ffd700 1 -- 172.21.15.5:0/2055219255 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6830004cf0 con 0x7f6840108430 2023-09-23T04:48:54.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.830+0000 7f6836ffd700 1 -- 172.21.15.5:0/2055219255 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830005680 con 0x7f6840108430 2023-09-23T04:48:54.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.830+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68401a5fc0 con 0x7f6840108430 2023-09-23T04:48:54.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.830+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68401a6440 con 0x7f6840108430 2023-09-23T04:48:54.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.833+0000 7f6836ffd700 1 -- 172.21.15.5:0/2055219255 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f6830004750 con 0x7f6840108430 2023-09-23T04:48:54.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.834+0000 7f685b8fdb80 1 Processor -- start 2023-09-23T04:48:54.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.834+0000 7f685b8fdb80 1 -- start start 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544850a460 con 0x5654485acbd0 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654484fbc20 con 0x5654485a1400 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654484f9540 con 0x5654485a1d20 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f6836ffd700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 0x7f682c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35038/0 (socket says 172.21.15.5:35038) 2023-09-23T04:48:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f6845d9b700 1 -- 172.21.15.5:0/3099211916 learned_addr learned my addr 172.21.15.5:0/3099211916 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f6836ffd700 1 -- 172.21.15.5:0/2055219255 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f6830014070 con 0x7f6840108430 2023-09-23T04:48:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f684659c700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 0x7f682c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f684559a700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.835+0000 7f684659c700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 -- 172.21.15.5:0/3099211916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 msgr2=0x5654485a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 -- 172.21.15.5:0/3099211916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 msgr2=0x5654485aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 -- 172.21.15.5:0/3099211916 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654485578d0 con 0x5654485a1d20 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f684559a700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f6845d9b700 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d10 tx=0x7f683c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0cc08b3e3711083 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f684659c700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 0x7f682c0a3c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f6838009320 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f682bfff700 1 -- 172.21.15.5:0/3099211916 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0097e0 con 0x5654485a1d20 2023-09-23T04:48:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f682bfff700 1 -- 172.21.15.5:0/3099211916 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c009e60 con 0x5654485a1d20 2023-09-23T04:48:54.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.836+0000 7f682bfff700 1 -- 172.21.15.5:0/3099211916 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c00f7c0 con 0x5654485a1d20 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- 172.21.15.5:0/3099211916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 msgr2=0x5654485ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d10 tx=0x7f683c0116c0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- 172.21.15.5:0/3099211916 shutdown_connections 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/3099211916 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1d20 0x5654485ac690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- 172.21.15.5:0/3099211916 >> 172.21.15.5:0/3099211916 conn(0x565448516260 msgr2=0x56544859d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- 172.21.15.5:0/3099211916 shutdown_connections 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- 172.21.15.5:0/3099211916 wait complete. 2023-09-23T04:48:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.837+0000 7f685b8fdb80 1 -- start start 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 Processor -- start 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 0x5654485a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654484f4e30 con 0x5654485a1d20 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x565448491710 con 0x5654485a1400 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654484913b0 con 0x5654485acbd0 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.838+0000 7f685b8fdb80 10 client.0 _ll_register_callbacks cb 0x5654482ee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 0x5654485a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35042/0 (socket says 172.21.15.5:35042) 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684659c700 1 -- 172.21.15.5:0/585409183 learned_addr learned my addr 172.21.15.5:0/585409183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 -- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 msgr2=0x5654485a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 -- 172.21.15.5:0/585409183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 msgr2=0x5654485a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 0x5654485a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 -- 172.21.15.5:0/585409183 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c009940 con 0x5654485a1400 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684659c700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f684559a700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 0x5654485a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f6845d9b700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485ac590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f683c000c00 tx=0x7f683c012700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85550ab10f11ca1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.839+0000 7f68297fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c027070 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565448595250 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f683c0050c0 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f683c005ce0 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f685b8fdb80 10 client.26270 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56544831fde0 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.840+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565448595250 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f683c02a030 con 0x5654485a1400 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 10 client.26270 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 10 client.26270 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 20 client.26270 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5654485bc3f0 0x5654485be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 4 client.26270 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654484ede10 con 0x5654485bc3f0 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.841+0000 7f684559a700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5654485bc3f0 0x5654485be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.842+0000 7f68297fa700 10 client.26270 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.842+0000 7f684559a700 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5654485bc3f0 0x5654485be810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e13da85aa8fcb2d6 server_cookie=be0fbda97e9d5fa4 in_seq=0 out_seq=0 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.842+0000 7f68297fa700 1 -- 172.21.15.5:0/585409183 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5654484ede10 con 0x5654485bc3f0 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.842+0000 7f68297fa700 10 client.26270 handle_command_reply: tid=0 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.842+0000 7f685b8fdb80 1 client.26270 shutdown 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 msgr2=0x5654485ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485ac590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f683c000c00 tx=0x7f683c012700 comp rx=0 tx=0).stop 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 shutdown_connections 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1400 0x5654485ac590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5654485bc3f0 0x5654485be810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/585409183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1d20 0x5654485a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f68297fa700 0 client.26270 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 >> 172.21.15.5:0/585409183 conn(0x565448516260 msgr2=0x56544859d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 shutdown_connections 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 1 -- 172.21.15.5:0/585409183 wait complete. 2023-09-23T04:48:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.843+0000 7f685b8fdb80 20 client.26270 trim_cache size 0 max 16384 2023-09-23T04:48:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 Processor -- start 2023-09-23T04:48:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 -- start start 2023-09-23T04:48:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485a5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565448375920 con 0x5654485acbd0 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x565448599d60 con 0x5654485a1d20 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.857+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654485a2bf0 con 0x5654485a1400 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f684559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35052/0 (socket says 172.21.15.5:35052) 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 -- 172.21.15.5:0/1628161251 learned_addr learned my addr 172.21.15.5:0/1628161251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f684659c700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 -- 172.21.15.5:0/1628161251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 msgr2=0x5654485a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 -- 172.21.15.5:0/1628161251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 msgr2=0x5654485a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 -- 172.21.15.5:0/1628161251 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565448308c60 con 0x5654485a1400 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f684559a700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f684659c700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.858+0000 7f6845d9b700 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f683c011a10 tx=0x7f683c005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4513f029aba9f5f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.859+0000 7f6834ff9700 1 -- 172.21.15.5:0/1628161251 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c027040 con 0x5654485a1400 2023-09-23T04:48:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.859+0000 7f6834ff9700 1 -- 172.21.15.5:0/1628161251 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c003b40 con 0x5654485a1400 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.859+0000 7f6834ff9700 1 -- 172.21.15.5:0/1628161251 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c005400 con 0x5654485a1400 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 -- 172.21.15.5:0/1628161251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 msgr2=0x5654485ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f683c011a10 tx=0x7f683c005fa0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 -- 172.21.15.5:0/1628161251 shutdown_connections 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485acbd0 0x5654485a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485a1400 0x5654485ac0c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/1628161251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 -- 172.21.15.5:0/1628161251 >> 172.21.15.5:0/1628161251 conn(0x565448516260 msgr2=0x5654485b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 -- 172.21.15.5:0/1628161251 shutdown_connections 2023-09-23T04:48:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.860+0000 7f685b8fdb80 1 -- 172.21.15.5:0/1628161251 wait complete. 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.861+0000 7f685b8fdb80 1 -- start start 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.861+0000 7f685b8fdb80 1 Processor -- start 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544860f250 con 0x5654485a1400 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544860f390 con 0x5654485a1d20 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56544860f550 con 0x5654485acbd0 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f685b8fdb80 10 client.0 _ll_register_callbacks cb 0x7f6830005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f6845d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58012/0 (socket says 172.21.15.5:58012) 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f6845d9b700 1 -- 172.21.15.5:0/672685376 learned_addr learned my addr 172.21.15.5:0/672685376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684659c700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 -- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 msgr2=0x5654485b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 -- 172.21.15.5:0/672685376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 msgr2=0x5654485af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684559a700 1 -- 172.21.15.5:0/672685376 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68401a6440 con 0x5654485a1d20 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f6845d9b700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.862+0000 7f684659c700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f684559a700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485afc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68300cac40 tx=0x7f683000e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12e0c7447b1fa1c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68300b2070 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f68300a27f0 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f683c009e60 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f683000ccd0 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 10 client.26276 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56544831fde0 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c009e60 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f683009a750 con 0x5654485a1d20 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 10 client.26276 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 10 client.26276 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 20 client.26276 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x565448610c90 0x5654486130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 4 client.26276 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.863+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654484edb30 con 0x565448610c90 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f6845d9b700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x565448610c90 0x5654486130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f6845d9b700 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x565448610c90 0x5654486130b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6008531384eca8e3 server_cookie=ce0d07dbbaccfbcb in_seq=0 out_seq=0 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f68297fa700 10 client.26276 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f68297fa700 1 -- 172.21.15.5:0/672685376 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654484edb30 con 0x565448610c90 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f68297fa700 10 client.26276 handle_command_reply: tid=0 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.864+0000 7f685b8fdb80 1 client.26276 shutdown 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 msgr2=0x5654485afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485afc60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68300cac40 tx=0x7f683000e8f0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 shutdown_connections 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5654485acbd0 0x5654485b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x565448610c90 0x5654486130b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654485a1400 0x5654485af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 --2- 172.21.15.5:0/672685376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5654485a1d20 0x5654485afc60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f68297fa700 0 client.26276 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 >> 172.21.15.5:0/672685376 conn(0x565448516260 msgr2=0x5654485a9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 shutdown_connections 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 1 -- 172.21.15.5:0/672685376 wait complete. 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f685b8fdb80 20 client.26276 trim_cache size 0 max 16384 2023-09-23T04:48:54.869 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 msgr2=0x7f682c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.865+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 0x7f682c0a3c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6838009fd0 tx=0x7f6838009320 comp rx=0 tx=0).stop 2023-09-23T04:48:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 msgr2=0x7f6840199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f683000e970 tx=0x7f683000ed30 comp rx=0 tx=0).stop 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 shutdown_connections 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f682c0a1790 0x7f682c0a3c50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f684019dcb0 0x7f684019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6840103310 0x7f6840199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 --2- 172.21.15.5:0/2055219255 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6840108430 0x7f6840199850 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 >> 172.21.15.5:0/2055219255 conn(0x7f68400feb50 msgr2=0x7f68400ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 shutdown_connections 2023-09-23T04:48:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:54.866+0000 7f6847fff700 1 -- 172.21.15.5:0/2055219255 wait complete. 2023-09-23T04:48:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:54 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/474316584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:54.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:48:54.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 131998 8.10 MB/sec warmup 90 sec latency 21.752 ms 2023-09-23T04:48:54.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:54 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/474316584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 -- 172.21.15.5:0/1542809453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550101aa0 msgr2=0x7ff550105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 --2- 172.21.15.5:0/1542809453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550101aa0 0x7ff550105ae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 -- 172.21.15.5:0/1542809453 shutdown_connections 2023-09-23T04:48:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 --2- 172.21.15.5:0/1542809453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff550106210 0x7ff55010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 --2- 172.21.15.5:0/1542809453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff5501014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 --2- 172.21.15.5:0/1542809453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550101aa0 0x7ff550105ae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 -- 172.21.15.5:0/1542809453 >> 172.21.15.5:0/1542809453 conn(0x7ff5500fc9d0 msgr2=0x7ff5500fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 -- 172.21.15.5:0/1542809453 shutdown_connections 2023-09-23T04:48:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.129+0000 7ff556bff700 1 -- 172.21.15.5:0/1542809453 wait complete. 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 Processor -- start 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 -- start start 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550106210 0x7ff550199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55019dba0 0x7ff55019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e620 con 0x7ff55019dba0 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e790 con 0x7ff550106210 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.130+0000 7ff556bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e950 con 0x7ff5501010f0 2023-09-23T04:48:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35092/0 (socket says 172.21.15.5:35092) 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 -- 172.21.15.5:0/574989441 learned_addr learned my addr 172.21.15.5:0/574989441 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55519c700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55019dba0 0x7ff55019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff54ffff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550106210 0x7ff550199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 -- 172.21.15.5:0/574989441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550106210 msgr2=0x7ff550199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550106210 0x7ff550199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 -- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55019dba0 msgr2=0x7ff55019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55019dba0 0x7ff55019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 -- 172.21.15.5:0/574989441 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff5501010f0 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.131+0000 7ff55499b700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9a0 tx=0x7ff54400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.132+0000 7ff54dffb700 1 -- 172.21.15.5:0/574989441 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544009790 con 0x7ff5501010f0 2023-09-23T04:48:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.132+0000 7ff54dffb700 1 -- 172.21.15.5:0/574989441 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff544010430 con 0x7ff5501010f0 2023-09-23T04:48:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.132+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5501a5eb0 con 0x7ff5501010f0 2023-09-23T04:48:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.132+0000 7ff54dffb700 1 -- 172.21.15.5:0/574989441 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544018680 con 0x7ff5501010f0 2023-09-23T04:48:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.132+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5501a6380 con 0x7ff5501010f0 2023-09-23T04:48:55.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.135+0000 7ff54dffb700 1 -- 172.21.15.5:0/574989441 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff5440105a0 con 0x7ff5501010f0 2023-09-23T04:48:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.137+0000 7ff56a5c6b80 1 Processor -- start 2023-09-23T04:48:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.137+0000 7ff56a5c6b80 1 -- start start 2023-09-23T04:48:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da3a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 0x55683da45690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d9a3460 con 0x55683da3ad20 2023-09-23T04:48:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d994c20 con 0x55683da3a400 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d992540 con 0x55683da45bd0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff54dffb700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 0x7ff5400a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff54ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff54ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35106/0 (socket says 172.21.15.5:35106) 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff54ffff700 1 -- 172.21.15.5:0/2398518173 learned_addr learned my addr 172.21.15.5:0/2398518173 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff55519c700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da3a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff55519c700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 0x7ff5400a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.138+0000 7ff55499b700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 0x55683da45690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54dffb700 1 -- 172.21.15.5:0/574989441 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff544014070 con 0x7ff5501010f0 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 -- 172.21.15.5:0/2398518173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 msgr2=0x55683da3a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da3a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 -- 172.21.15.5:0/2398518173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 msgr2=0x55683da45690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 0x55683da45690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 -- 172.21.15.5:0/2398518173 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55683d9f08d0 con 0x55683da45bd0 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff55499b700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 0x55683da45690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c00b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4423ab2072e26563 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff55519c700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 0x7ff5400a3c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff548009790 tx=0x7ff548009130 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff53affd700 1 -- 172.21.15.5:0/2398518173 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c01d070 con 0x55683da45bd0 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff53affd700 1 -- 172.21.15.5:0/2398518173 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53c01e040 con 0x55683da45bd0 2023-09-23T04:48:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.139+0000 7ff53affd700 1 -- 172.21.15.5:0/2398518173 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c004900 con 0x55683da45bd0 2023-09-23T04:48:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2398518173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 msgr2=0x55683da47fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c00b610 comp rx=0 tx=0).stop 2023-09-23T04:48:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2398518173 shutdown_connections 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da47fc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da3a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2398518173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3ad20 0x55683da45690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2398518173 >> 172.21.15.5:0/2398518173 conn(0x55683d9af260 msgr2=0x55683da36790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2398518173 shutdown_connections 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.140+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2398518173 wait complete. 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.141+0000 7ff56a5c6b80 1 -- start start 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.141+0000 7ff56a5c6b80 1 Processor -- start 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 0x55683da427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da42df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d98de30 con 0x55683da3a400 2023-09-23T04:48:55.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d92a710 con 0x55683da3ad20 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d92a3b0 con 0x55683da45bd0 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff56a5c6b80 10 client.0 _ll_register_callbacks cb 0x55683d787800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58030/0 (socket says 172.21.15.5:58030) 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 -- 172.21.15.5:0/3507896246 learned_addr learned my addr 172.21.15.5:0/3507896246 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55519c700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da42df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff54ffff700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 0x55683da427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 -- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 msgr2=0x55683da42df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da42df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 -- 172.21.15.5:0/3507896246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 msgr2=0x55683da427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 0x55683da427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff55499b700 1 -- 172.21.15.5:0/3507896246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55010e600 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.142+0000 7ff54ffff700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 0x55683da427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff55499b700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff544096130 tx=0x7ff5440cb0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d60881543c395 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54409e320 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff5440a2740 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55683da2e250 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5440cd420 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 10 client.16296 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55683d7b8de0 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55683da2e250 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff5440998a0 con 0x55683da3a400 2023-09-23T04:48:55.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 10 client.16296 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.143+0000 7ff56a5c6b80 20 client.16296 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683da555a0 0x55683da579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff56a5c6b80 4 client.16296 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55683d986e10 con 0x55683da555a0 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff54ffff700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683da555a0 0x55683da579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff54ffff700 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683da555a0 0x55683da579c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=295a7b46413d21f3 server_cookie=d0ed9167f617a6dc in_seq=0 out_seq=0 2023-09-23T04:48:55.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.144+0000 7ff52ffff700 10 client.16296 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:55.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff52ffff700 1 -- 172.21.15.5:0/3507896246 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55683d986e10 con 0x55683da555a0 2023-09-23T04:48:55.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff52ffff700 10 client.16296 handle_command_reply: tid=0 2023-09-23T04:48:55.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 client.16296 shutdown 2023-09-23T04:48:55.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 msgr2=0x55683da45540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff544096130 tx=0x7ff5440cb0f0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 shutdown_connections 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da45bd0 0x55683da42df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683da555a0 0x55683da579c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da45540 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/3507896246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3ad20 0x55683da427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff52ffff700 0 client.16296 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.145+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 >> 172.21.15.5:0/3507896246 conn(0x55683d9af260 msgr2=0x55683da36320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.146+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 shutdown_connections 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.146+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/3507896246 wait complete. 2023-09-23T04:48:55.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.146+0000 7ff56a5c6b80 20 client.16296 trim_cache size 0 max 16384 2023-09-23T04:48:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.160+0000 7ff56a5c6b80 1 Processor -- start 2023-09-23T04:48:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.160+0000 7ff56a5c6b80 1 -- start start 2023-09-23T04:48:55.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:54 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/474316584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 0x55683da41f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d80e920 con 0x55683da45bd0 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683da40d20 con 0x55683da3a400 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683da3e610 con 0x55683da3ad20 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55519c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 0x55683da41f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff54ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52368/0 (socket says 172.21.15.5:52368) 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 -- 172.21.15.5:0/817264635 learned_addr learned my addr 172.21.15.5:0/817264635 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 -- 172.21.15.5:0/817264635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 msgr2=0x55683da450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 -- 172.21.15.5:0/817264635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 msgr2=0x55683da41f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 0x55683da41f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55499b700 1 -- 172.21.15.5:0/817264635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55683d7a1c60 con 0x55683da3a400 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff54ffff700 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.161+0000 7ff55519c700 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 0x55683da41f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.162+0000 7ff55499b700 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c2d0 tx=0x7ff544098960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5469b7bc7f95ef2c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.162+0000 7ff53b7fe700 1 -- 172.21.15.5:0/817264635 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54407eb30 con 0x55683da3a400 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.162+0000 7ff53b7fe700 1 -- 172.21.15.5:0/817264635 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5440a2740 con 0x55683da3a400 2023-09-23T04:48:55.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.162+0000 7ff53b7fe700 1 -- 172.21.15.5:0/817264635 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54409a610 con 0x55683da3a400 2023-09-23T04:48:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/817264635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 msgr2=0x55683da42d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c2d0 tx=0x7ff544098960 comp rx=0 tx=0).stop 2023-09-23T04:48:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/817264635 shutdown_connections 2023-09-23T04:48:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da45bd0 0x55683da41f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da3a400 0x55683da42d70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/817264635 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/817264635 >> 172.21.15.5:0/817264635 conn(0x55683d9af260 msgr2=0x55683da3db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/817264635 shutdown_connections 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/817264635 wait complete. 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.163+0000 7ff56a5c6b80 1 -- start start 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 Processor -- start 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da45bd0 0x55683da4bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683da4c3a0 con 0x55683da3a400 2023-09-23T04:48:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683da4c510 con 0x55683da45bd0 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683da4c6d0 con 0x55683da3ad20 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff56a5c6b80 10 client.0 _ll_register_callbacks cb 0x55683d787800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff55499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff54ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff55499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58044/0 (socket says 172.21.15.5:58044) 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff55499b700 1 -- 172.21.15.5:0/2345429915 learned_addr learned my addr 172.21.15.5:0/2345429915 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.164+0000 7ff55519c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da45bd0 0x55683da4bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 -- 172.21.15.5:0/2345429915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da45bd0 msgr2=0x55683da4bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da45bd0 0x55683da4bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 -- 172.21.15.5:0/2345429915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 msgr2=0x55683da488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 -- 172.21.15.5:0/2345429915 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55683d9f08d0 con 0x55683da3ad20 2023-09-23T04:48:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff55499b700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff54ffff700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da48df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b2e0 tx=0x7ff53c004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e68a5e4377f3c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff52ffff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.165+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c031040 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff53c01e040 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff53c0053c0 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53c01d040 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 10 client.26350 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7ff548011040 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53c01e040 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff53c0045e0 con 0x55683da3ad20 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 10 client.26350 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 10 client.26350 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:48:55.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 20 client.26350 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683daaa150 0x55683daac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 4 client.26350 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:55.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.166+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55683d986b30 con 0x55683daaa150 2023-09-23T04:48:55.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff55499b700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683daaa150 0x55683daac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff55499b700 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683daaa150 0x55683daac540 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce1094aa8ce19ca5 server_cookie=64c520130066ae1a in_seq=0 out_seq=0 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff52ffff700 10 client.26350 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff52ffff700 1 -- 172.21.15.5:0/2345429915 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55683d986b30 con 0x55683daaa150 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff52ffff700 10 client.26350 handle_command_reply: tid=0 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff56a5c6b80 1 client.26350 shutdown 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 msgr2=0x55683da48df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.167+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da48df0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b2e0 tx=0x7ff53c004500 comp rx=0 tx=0).stop 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 shutdown_connections 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55683da45bd0 0x55683da4bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55683daaa150 0x55683daac540 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55683da3a400 0x55683da488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 --2- 172.21.15.5:0/2345429915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55683da3ad20 0x55683da48df0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff52ffff700 0 client.26350 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 >> 172.21.15.5:0/2345429915 conn(0x55683d9af260 msgr2=0x55683da3c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 shutdown_connections 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 1 -- 172.21.15.5:0/2345429915 wait complete. 2023-09-23T04:48:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff56a5c6b80 20 client.26350 trim_cache size 0 max 16384 2023-09-23T04:48:55.173 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 msgr2=0x7ff5400a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 0x7ff5400a3c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff548009790 tx=0x7ff548009130 comp rx=0 tx=0).stop 2023-09-23T04:48:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 msgr2=0x7ff550199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.168+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9a0 tx=0x7ff54400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 shutdown_connections 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff5400a1790 0x7ff5400a3c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55019dba0 0x7ff55019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff5501010f0 0x7ff550199200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 --2- 172.21.15.5:0/574989441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff550106210 0x7ff550199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 >> 172.21.15.5:0/574989441 conn(0x7ff5500fc9d0 msgr2=0x7ff5500fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 shutdown_connections 2023-09-23T04:48:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.169+0000 7ff556bff700 1 -- 172.21.15.5:0/574989441 wait complete. 2023-09-23T04:48:55.188 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.415+0000 7f686dfff700 1 -- 172.21.15.5:0/151838219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6868101a80 msgr2=0x7f6868105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.415+0000 7f686dfff700 1 --2- 172.21.15.5:0/151838219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6868101a80 0x7f6868105ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f685c009990 tx=0x7f685c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 -- 172.21.15.5:0/151838219 shutdown_connections 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 --2- 172.21.15.5:0/151838219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 0x7f686810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 --2- 172.21.15.5:0/151838219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68681010d0 0x7f68681014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 --2- 172.21.15.5:0/151838219 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6868101a80 0x7f6868105ac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 -- 172.21.15.5:0/151838219 >> 172.21.15.5:0/151838219 conn(0x7f68680fc990 msgr2=0x7f68680fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 -- 172.21.15.5:0/151838219 shutdown_connections 2023-09-23T04:48:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.416+0000 7f686dfff700 1 -- 172.21.15.5:0/151838219 wait complete. 2023-09-23T04:48:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.417+0000 7f686dfff700 1 Processor -- start 2023-09-23T04:48:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.417+0000 7f686dfff700 1 -- start start 2023-09-23T04:48:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 0x7f6868073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 0x7f68681a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681a57d0 con 0x7f68681061f0 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681a5940 con 0x7f6868073ba0 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f686dfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681a5b00 con 0x7f68681010d0 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f68677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f68677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35150/0 (socket says 172.21.15.5:35150) 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f6866ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 0x7f6868073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f68677fe700 1 -- 172.21.15.5:0/3936255481 learned_addr learned my addr 172.21.15.5:0/3936255481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.418+0000 7f6867fff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 0x7f68681a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 -- 172.21.15.5:0/3936255481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 msgr2=0x7f68681a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 0x7f68681a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 -- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 msgr2=0x7f6868073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 0x7f6868073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 -- 172.21.15.5:0/3936255481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0095f0 con 0x7f68681010d0 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f6867fff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 0x7f68681a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f68677fe700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9a0 tx=0x7f685800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f6864ff9700 1 -- 172.21.15.5:0/3936255481 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858009790 con 0x7f68681010d0 2023-09-23T04:48:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f6864ff9700 1 -- 172.21.15.5:0/3936255481 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f685800cbd0 con 0x7f68681010d0 2023-09-23T04:48:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f6864ff9700 1 -- 172.21.15.5:0/3936255481 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858018680 con 0x7f68681010d0 2023-09-23T04:48:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.419+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681a5de0 con 0x7f68681010d0 2023-09-23T04:48:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.420+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68681a6210 con 0x7f68681010d0 2023-09-23T04:48:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.423+0000 7f6864ff9700 1 -- 172.21.15.5:0/3936255481 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f685800cd40 con 0x7f68681010d0 2023-09-23T04:48:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 Processor -- start 2023-09-23T04:48:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 -- start start 2023-09-23T04:48:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.425+0000 7f68818b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d15330460 con 0x558d153d2bd0 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f68818b3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d15321c20 con 0x558d153c7d20 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f68818b3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1531f540 con 0x558d153c7400 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6864ff9700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 0x7f68500a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6864ff9700 1 -- 172.21.15.5:0/3936255481 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f6858014070 con 0x7f68681010d0 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35154/0 (socket says 172.21.15.5:35154) 2023-09-23T04:48:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 -- 172.21.15.5:0/1404831760 learned_addr learned my addr 172.21.15.5:0/1404831760 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6866ffd700 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f68677fe700 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 0x7f68500a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 -- 172.21.15.5:0/1404831760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 msgr2=0x558d153d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153d2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 -- 172.21.15.5:0/1404831760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 msgr2=0x558d153d4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153d4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.426+0000 7f6867fff700 1 -- 172.21.15.5:0/1404831760 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d1537d8d0 con 0x558d153c7400 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.427+0000 7f6867fff700 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6860012cf0 tx=0x7f6860017350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d0b0156c71b70c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.427+0000 7f684dffb700 1 -- 172.21.15.5:0/1404831760 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000e040 con 0x558d153c7400 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.427+0000 7f684dffb700 1 -- 172.21.15.5:0/1404831760 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6860017ec0 con 0x558d153c7400 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.427+0000 7f684dffb700 1 -- 172.21.15.5:0/1404831760 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860019a10 con 0x558d153c7400 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 -- 172.21.15.5:0/1404831760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 msgr2=0x558d153c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f6867fff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 0x7f68500a3bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6860021000 tx=0x7f686001c000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6860012cf0 tx=0x7f6860017350 comp rx=0 tx=0).stop 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 -- 172.21.15.5:0/1404831760 shutdown_connections 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153c77e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 --2- 172.21.15.5:0/1404831760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 -- 172.21.15.5:0/1404831760 >> 172.21.15.5:0/1404831760 conn(0x558d1533c260 msgr2=0x558d153c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 -- 172.21.15.5:0/1404831760 shutdown_connections 2023-09-23T04:48:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.428+0000 7f68818b3b80 1 -- 172.21.15.5:0/1404831760 wait complete. 2023-09-23T04:48:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.429+0000 7f68818b3b80 1 -- start start 2023-09-23T04:48:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.429+0000 7f68818b3b80 1 Processor -- start 2023-09-23T04:48:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7d20 0x558d153cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 0x558d153cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1531ae30 con 0x558d153c7400 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d152b7710 con 0x558d153d2bd0 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d152b73b0 con 0x558d153c7d20 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68818b3b80 10 client.0 _ll_register_callbacks cb 0x558d15114800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58076/0 (socket says 172.21.15.5:58076) 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 -- 172.21.15.5:0/763169171 learned_addr learned my addr 172.21.15.5:0/763169171 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f6866ffd700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7d20 0x558d153cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f6867fff700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 0x558d153cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 -- 172.21.15.5:0/763169171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7d20 msgr2=0x558d153cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7d20 0x558d153cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 -- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 msgr2=0x558d153cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 0x558d153cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f68677fe700 1 -- 172.21.15.5:0/763169171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68600128a0 con 0x558d153c7400 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.430+0000 7f6867fff700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 0x558d153cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f68677fe700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6858096fc0 tx=0x7f685806ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e1a3f75423d4d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f6846ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685806ac30 con 0x558d153c7400 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f685809a550 con 0x558d153c7400 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d15313e10 con 0x558d153c7400 2023-09-23T04:48:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f685809e6a0 con 0x558d153c7400 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f68818b3b80 10 client.16314 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558d15145de0 con 0x558d153c7400 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.431+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d15313e10 con 0x558d153c7400 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f68580a2530 con 0x558d153c7400 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 20 client.16314 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d153e2640 0x558d153e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 4 client.16314 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d15313b30 con 0x558d153e2640 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.432+0000 7f6866ffd700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d153e2640 0x558d153e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.433+0000 7f6846ffd700 10 client.16314 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.433+0000 7f6866ffd700 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d153e2640 0x558d153e4a30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e221255453c84c8 server_cookie=6c1ee1258e24fc7d in_seq=0 out_seq=0 2023-09-23T04:48:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.433+0000 7f6846ffd700 1 -- 172.21.15.5:0/763169171 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558d15313b30 con 0x558d153e2640 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.433+0000 7f6846ffd700 10 client.16314 handle_command_reply: tid=0 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.433+0000 7f68818b3b80 1 client.16314 shutdown 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 msgr2=0x558d153d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6858096fc0 tx=0x7f685806ab40 comp rx=0 tx=0).stop 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 shutdown_connections 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d153e2640 0x558d153e4a30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153d2bd0 0x558d153cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7400 0x558d153d2530 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 --2- 172.21.15.5:0/763169171 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7d20 0x558d153cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f6846ffd700 0 client.16314 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 >> 172.21.15.5:0/763169171 conn(0x558d1533c260 msgr2=0x558d153c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 shutdown_connections 2023-09-23T04:48:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 1 -- 172.21.15.5:0/763169171 wait complete. 2023-09-23T04:48:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.434+0000 7f68818b3b80 20 client.16314 trim_cache size 0 max 16384 2023-09-23T04:48:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 Processor -- start 2023-09-23T04:48:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 -- start start 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d1519b920 con 0x558d153d2bd0 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d153bfd60 con 0x558d153c7d20 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68818b3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d153cdd10 con 0x558d153c7400 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35168/0 (socket says 172.21.15.5:35168) 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.450+0000 7f68677fe700 1 -- 172.21.15.5:0/3360263866 learned_addr learned my addr 172.21.15.5:0/3360263866 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f6866ffd700 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f6867fff700 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 -- 172.21.15.5:0/3360263866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 msgr2=0x558d153cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 -- 172.21.15.5:0/3360263866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 msgr2=0x558d153cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 -- 172.21.15.5:0/3360263866 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d1512ec60 con 0x558d153c7400 2023-09-23T04:48:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f68677fe700 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68580a23b0 tx=0x7f68580ba0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc4e32aa729be1a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f684e7fc700 1 -- 172.21.15.5:0/3360263866 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685809e6a0 con 0x558d153c7400 2023-09-23T04:48:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f684e7fc700 1 -- 172.21.15.5:0/3360263866 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68580a28e0 con 0x558d153c7400 2023-09-23T04:48:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.451+0000 7f684e7fc700 1 -- 172.21.15.5:0/3360263866 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685809e160 con 0x558d153c7400 2023-09-23T04:48:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 -- 172.21.15.5:0/3360263866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 msgr2=0x558d153d20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68580a23b0 tx=0x7f68580ba0b0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 -- 172.21.15.5:0/3360263866 shutdown_connections 2023-09-23T04:48:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153d2bd0 0x558d153cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153c7400 0x558d153d20d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3360263866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7d20 0x558d153cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 -- 172.21.15.5:0/3360263866 >> 172.21.15.5:0/3360263866 conn(0x558d1533c260 msgr2=0x558d153caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 -- 172.21.15.5:0/3360263866 shutdown_connections 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.452+0000 7f68818b3b80 1 -- 172.21.15.5:0/3360263866 wait complete. 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.453+0000 7f68818b3b80 1 -- start start 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.453+0000 7f68818b3b80 1 Processor -- start 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7d20 0x558d153d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153d2bd0 0x558d15435460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d153d7f70 con 0x558d153c7d20 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d154359a0 con 0x558d153c7400 2023-09-23T04:48:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d15435ae0 con 0x558d153d2bd0 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68818b3b80 10 client.0 _ll_register_callbacks cb 0x7f685809a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f6867fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153d2bd0 0x558d15435460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52424/0 (socket says 172.21.15.5:52424) 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68677fe700 1 -- 172.21.15.5:0/3474730161 learned_addr learned my addr 172.21.15.5:0/3474730161 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f6866ffd700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7d20 0x558d153d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68677fe700 1 -- 172.21.15.5:0/3474730161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153d2bd0 msgr2=0x558d15435460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.454+0000 7f68677fe700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153d2bd0 0x558d15435460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68677fe700 1 -- 172.21.15.5:0/3474730161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7d20 msgr2=0x558d153d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68677fe700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7d20 0x558d153d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68677fe700 1 -- 172.21.15.5:0/3474730161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d15145de0 con 0x558d153c7400 2023-09-23T04:48:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68677fe700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6858096f70 tx=0x7f685809a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dce7802b5184c228 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f6846ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685806ad40 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d153bb250 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f68580af050 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f685806aea0 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68818b3b80 10 client.26312 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.455+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558d1516c250 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d153bb250 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6858062080 con 0x558d153c7400 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 10 client.26312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 20 client.26312 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d15436c80 0x558d15439070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 4 client.26312 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558d15166f70 con 0x558d15436c80 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f6866ffd700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d15436c80 0x558d15439070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f6866ffd700 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d15436c80 0x558d15439070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d277f0fc3716130 server_cookie=6abe09ea1a035160 in_seq=0 out_seq=0 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.456+0000 7f6846ffd700 10 client.26312 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f6846ffd700 1 -- 172.21.15.5:0/3474730161 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558d15166f70 con 0x558d15436c80 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f6846ffd700 10 client.26312 handle_command_reply: tid=0 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 client.26312 shutdown 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 msgr2=0x558d153d7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6858096f70 tx=0x7f685809a160 comp rx=0 tx=0).stop 2023-09-23T04:48:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 shutdown_connections 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x558d15436c80 0x558d15439070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d153d2bd0 0x558d15435460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d153c7400 0x558d153d7120 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 --2- 172.21.15.5:0/3474730161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d153c7d20 0x558d153d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 >> 172.21.15.5:0/3474730161 conn(0x558d1533c260 msgr2=0x558d153c9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f6846ffd700 0 client.26312 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 shutdown_connections 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.457+0000 7f68818b3b80 1 -- 172.21.15.5:0/3474730161 wait complete. 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f68818b3b80 20 client.26312 trim_cache size 0 max 16384 2023-09-23T04:48:55.461 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 msgr2=0x7f68500a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 0x7f68500a3bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6860021000 tx=0x7f686001c000 comp rx=0 tx=0).stop 2023-09-23T04:48:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 msgr2=0x7f6868073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9a0 tx=0x7f685800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 shutdown_connections 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68500a1710 0x7f68500a3bd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6868073ba0 0x7f68681a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68681010d0 0x7f6868073120 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 --2- 172.21.15.5:0/3936255481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68681061f0 0x7f6868073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.458+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 >> 172.21.15.5:0/3936255481 conn(0x7f68680fc990 msgr2=0x7f68680fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.459+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 shutdown_connections 2023-09-23T04:48:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.459+0000 7f686dfff700 1 -- 172.21.15.5:0/3936255481 wait complete. 2023-09-23T04:48:55.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:48:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 -- 172.21.15.5:0/439589940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201020c0 msgr2=0x7f31201024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 --2- 172.21.15.5:0/439589940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201020c0 0x7f31201024a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3110009990 tx=0x7f3110009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 -- 172.21.15.5:0/439589940 shutdown_connections 2023-09-23T04:48:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 --2- 172.21.15.5:0/439589940 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f312010b490 0x7f312010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 --2- 172.21.15.5:0/439589940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201020c0 0x7f31201024a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 --2- 172.21.15.5:0/439589940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31201029e0 0x7f312010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 -- 172.21.15.5:0/439589940 >> 172.21.15.5:0/439589940 conn(0x7f3120076f30 msgr2=0x7f3120077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 -- 172.21.15.5:0/439589940 shutdown_connections 2023-09-23T04:48:55.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.718+0000 7f3124dff700 1 -- 172.21.15.5:0/439589940 wait complete. 2023-09-23T04:48:55.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.719+0000 7f3124dff700 1 Processor -- start 2023-09-23T04:48:55.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.719+0000 7f3124dff700 1 -- start start 2023-09-23T04:48:55.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201029e0 0x7f3120100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f312010b490 0x7f31201a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120100a80 con 0x7f312010b490 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201a5750 con 0x7f31201029e0 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f3124dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201a5890 con 0x7f31201020c0 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35182/0 (socket says 172.21.15.5:35182) 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 -- 172.21.15.5:0/4310302 learned_addr learned my addr 172.21.15.5:0/4310302 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311dd9b700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201029e0 0x7f3120100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311ed9d700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f312010b490 0x7f31201a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 -- 172.21.15.5:0/4310302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201029e0 msgr2=0x7f3120100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201029e0 0x7f3120100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.720+0000 7f311e59c700 1 -- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f312010b490 msgr2=0x7f31201a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f311e59c700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f312010b490 0x7f31201a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f311e59c700 1 -- 172.21.15.5:0/4310302 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100095f0 con 0x7f31201020c0 2023-09-23T04:48:55.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f311e59c700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3110005910 tx=0x7f3110005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f310f7fe700 1 -- 172.21.15.5:0/4310302 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311001d070 con 0x7f31201020c0 2023-09-23T04:48:55.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31201a5a90 con 0x7f31201020c0 2023-09-23T04:48:55.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f310f7fe700 1 -- 172.21.15.5:0/4310302 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3110022470 con 0x7f31201020c0 2023-09-23T04:48:55.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f310f7fe700 1 -- 172.21.15.5:0/4310302 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110017650 con 0x7f31201020c0 2023-09-23T04:48:55.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.721+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31201a5f60 con 0x7f31201020c0 2023-09-23T04:48:55.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.725+0000 7f310f7fe700 1 -- 172.21.15.5:0/4310302 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f31100225e0 con 0x7f31201020c0 2023-09-23T04:48:55.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.726+0000 7f310f7fe700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 0x7f31080a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.726+0000 7f3138801b80 1 Processor -- start 2023-09-23T04:48:55.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f310f7fe700 1 -- 172.21.15.5:0/4310302 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f31100ce9d0 con 0x7f31201020c0 2023-09-23T04:48:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f311dd9b700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 0x7f31080a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 -- start start 2023-09-23T04:48:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfb67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfc3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bf1f460 con 0x55602bfb6400 2023-09-23T04:48:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bf10c20 con 0x55602bfb6d20 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.727+0000 7f3138801b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bf0e540 con 0x55602bfc1bd0 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfb67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfc3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52460/0 (socket says 172.21.15.5:52460) 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 -- 172.21.15.5:0/1598348339 learned_addr learned my addr 172.21.15.5:0/1598348339 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 0x7f31080a3c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3120101500 tx=0x7f3114009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 -- 172.21.15.5:0/1598348339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 msgr2=0x55602bfc3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfc3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 -- 172.21.15.5:0/1598348339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 msgr2=0x55602bfb67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfb67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 -- 172.21.15.5:0/1598348339 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55602bf6c8d0 con 0x55602bfb6d20 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3114011d10 tx=0x7f3114014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc7c5bba1ef6f07 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.728+0000 7f310cff9700 1 -- 172.21.15.5:0/1598348339 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400eb20 con 0x55602bfb6d20 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f310cff9700 1 -- 172.21.15.5:0/1598348339 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3114021070 con 0x55602bfb6d20 2023-09-23T04:48:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f310cff9700 1 -- 172.21.15.5:0/1598348339 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114005500 con 0x55602bfb6d20 2023-09-23T04:48:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 -- 172.21.15.5:0/1598348339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 msgr2=0x55602bfc1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3114011d10 tx=0x7f3114014000 comp rx=0 tx=0).stop 2023-09-23T04:48:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 -- 172.21.15.5:0/1598348339 shutdown_connections 2023-09-23T04:48:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfc3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 --2- 172.21.15.5:0/1598348339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc1690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.729+0000 7f3138801b80 1 -- 172.21.15.5:0/1598348339 >> 172.21.15.5:0/1598348339 conn(0x55602bf2b260 msgr2=0x55602bfb2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.730+0000 7f3138801b80 1 -- 172.21.15.5:0/1598348339 shutdown_connections 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.730+0000 7f3138801b80 1 -- 172.21.15.5:0/1598348339 wait complete. 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.730+0000 7f3138801b80 1 -- start start 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 Processor -- start 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 0x55602bfbeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfc1bd0 0x55602bfbf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bf09e30 con 0x55602bfb6400 2023-09-23T04:48:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bea6710 con 0x55602bfc1bd0 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bea63b0 con 0x55602bfb6d20 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f3138801b80 10 client.0 _ll_register_callbacks cb 0x55602bd03800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f311e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.731+0000 7f311e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58128/0 (socket says 172.21.15.5:58128) 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 -- 172.21.15.5:0/2488462284 learned_addr learned my addr 172.21.15.5:0/2488462284 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311dd9b700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 0x55602bfbeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311ed9d700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfc1bd0 0x55602bfbf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 -- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 msgr2=0x55602bfbeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 0x55602bfbeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 -- 172.21.15.5:0/2488462284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfc1bd0 msgr2=0x55602bfbf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfc1bd0 0x55602bfbf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 -- 172.21.15.5:0/2488462284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3114017070 con 0x55602bfb6400 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311dd9b700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 0x55602bfbeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f311e59c700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3110006010 tx=0x7f311009b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4826c916b6d216 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.732+0000 7f3105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009b9f0 con 0x55602bfb6400 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f311009bc90 con 0x55602bfb6400 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55602bfaa250 con 0x55602bfb6400 2023-09-23T04:48:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311009aef0 con 0x55602bfb6400 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3138801b80 10 client.16332 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55602bd34de0 con 0x55602bfb6400 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55602bfaa250 con 0x55602bfb6400 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.733+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f311009aa80 con 0x55602bfb6400 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 20 client.16332 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602bfd12c0 0x55602bfd36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 4 client.16332 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55602bf02e10 con 0x55602bfd12c0 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f311dd9b700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602bfd12c0 0x55602bfd36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f311dd9b700 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602bfd12c0 0x55602bfd36e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf7e9d2ba509c456 server_cookie=74a16b79d0aa3f25 in_seq=0 out_seq=0 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.734+0000 7f3105ffb700 10 client.16332 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.735+0000 7f3105ffb700 1 -- 172.21.15.5:0/2488462284 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55602bf02e10 con 0x55602bfd12c0 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.735+0000 7f3105ffb700 10 client.16332 handle_command_reply: tid=0 2023-09-23T04:48:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.735+0000 7f3138801b80 1 client.16332 shutdown 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.735+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 msgr2=0x55602bfbe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.735+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3110006010 tx=0x7f311009b360 comp rx=0 tx=0).stop 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 shutdown_connections 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602bfd12c0 0x55602bfd36e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfc1bd0 0x55602bfbf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfbe9a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 --2- 172.21.15.5:0/2488462284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfb6d20 0x55602bfbeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3105ffb700 0 client.16332 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 >> 172.21.15.5:0/2488462284 conn(0x55602bf2b260 msgr2=0x55602bfb2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 shutdown_connections 2023-09-23T04:48:55.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 1 -- 172.21.15.5:0/2488462284 wait complete. 2023-09-23T04:48:55.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.736+0000 7f3138801b80 20 client.16332 trim_cache size 0 max 16384 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.753+0000 7f3138801b80 1 Processor -- start 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 -- start start 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 0x55602bfb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 0x55602bfbf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfbd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bd8a920 con 0x55602bfb6d20 2023-09-23T04:48:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bfb7360 con 0x55602bfb6400 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f3138801b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602bfba560 con 0x55602bfc1bd0 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 0x55602bfbf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f311e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 0x55602bfb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 0x55602bfbf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58142/0 (socket says 172.21.15.5:58142) 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f311dd9b700 1 -- 172.21.15.5:0/726657673 learned_addr learned my addr 172.21.15.5:0/726657673 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.754+0000 7f311ed9d700 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfbd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 -- 172.21.15.5:0/726657673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 msgr2=0x55602bfbd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfbd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 -- 172.21.15.5:0/726657673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 msgr2=0x55602bfbf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 0x55602bfbf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 -- 172.21.15.5:0/726657673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55602bd1dc60 con 0x55602bfb6400 2023-09-23T04:48:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f311e59c700 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 0x55602bfb9440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31100998e0 tx=0x7f311009b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c980d9c92ecabf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f310d7fa700 1 -- 172.21.15.5:0/726657673 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009aef0 con 0x55602bfb6400 2023-09-23T04:48:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f310d7fa700 1 -- 172.21.15.5:0/726657673 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31100b0070 con 0x55602bfb6400 2023-09-23T04:48:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.755+0000 7f310d7fa700 1 -- 172.21.15.5:0/726657673 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009ba50 con 0x55602bfb6400 2023-09-23T04:48:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 -- 172.21.15.5:0/726657673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 msgr2=0x55602bfb9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 0x55602bfb9440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31100998e0 tx=0x7f311009b490 comp rx=0 tx=0).stop 2023-09-23T04:48:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 -- 172.21.15.5:0/726657673 shutdown_connections 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602bfbd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6400 0x55602bfb9440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 --2- 172.21.15.5:0/726657673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6d20 0x55602bfbf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 -- 172.21.15.5:0/726657673 >> 172.21.15.5:0/726657673 conn(0x55602bf2b260 msgr2=0x55602bfbcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 -- 172.21.15.5:0/726657673 shutdown_connections 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.756+0000 7f3138801b80 1 -- 172.21.15.5:0/726657673 wait complete. 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.757+0000 7f3138801b80 1 -- start start 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.757+0000 7f3138801b80 1 Processor -- start 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfc6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602c0240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602c0245f0 con 0x55602bfb6400 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602c024730 con 0x55602bfb6d20 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55602c024870 con 0x55602bfc1bd0 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f3138801b80 10 client.0 _ll_register_callbacks cb 0x7f3110097240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfc6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52476/0 (socket says 172.21.15.5:52476) 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 -- 172.21.15.5:0/1286398898 learned_addr learned my addr 172.21.15.5:0/1286398898 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311ed9d700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602c0240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 -- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 msgr2=0x55602c0240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602c0240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 -- 172.21.15.5:0/1286398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 msgr2=0x55602bfc6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfc6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311dd9b700 1 -- 172.21.15.5:0/1286398898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55602bd34de0 con 0x55602bfb6d20 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.758+0000 7f311e59c700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfc6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f311dd9b700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f311400e970 tx=0x7f31140092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89654e940e80cb42 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3105ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114033430 con 0x55602bfb6d20 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3114021070 con 0x55602bfb6d20 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f311400eb20 con 0x55602bfb6d20 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3114033590 con 0x55602bfb6d20 2023-09-23T04:48:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3138801b80 10 client.26330 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55602bd5b250 con 0x55602bfb6d20 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.759+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3114021070 con 0x55602bfb6d20 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3114039020 con 0x55602bfb6d20 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 10 client.26330 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 10 client.26330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 20 client.26330 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602c025fb0 0x55602c0283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 4 client.26330 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55602bf02b30 con 0x55602c025fb0 2023-09-23T04:48:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.760+0000 7f311e59c700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602c025fb0 0x55602c0283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f311e59c700 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602c025fb0 0x55602c0283d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28ca0fc230798137 server_cookie=6f2baf697d656b46 in_seq=0 out_seq=0 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3105ffb700 10 client.26330 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3105ffb700 1 -- 172.21.15.5:0/1286398898 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55602bf02b30 con 0x55602c025fb0 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3105ffb700 10 client.26330 handle_command_reply: tid=0 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 client.26330 shutdown 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 msgr2=0x55602bfc69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f311400e970 tx=0x7f31140092d0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 shutdown_connections 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55602bfc1bd0 0x55602c0240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55602bfb6400 0x55602bfc6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55602c025fb0 0x55602c0283d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 --2- 172.21.15.5:0/1286398898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55602bfb6d20 0x55602bfc69b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3105ffb700 0 client.26330 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 >> 172.21.15.5:0/1286398898 conn(0x55602bf2b260 msgr2=0x55602bfb2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 shutdown_connections 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 1 -- 172.21.15.5:0/1286398898 wait complete. 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.761+0000 7f3138801b80 20 client.26330 trim_cache size 0 max 16384 2023-09-23T04:48:55.766 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 msgr2=0x7f31080a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 0x7f31080a3c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3120101500 tx=0x7f3114009260 comp rx=0 tx=0).stop 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 msgr2=0x7f31200ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3110005910 tx=0x7f3110005720 comp rx=0 tx=0).stop 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 shutdown_connections 2023-09-23T04:48:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31080a1790 0x7f31080a3c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f312010b490 0x7f31201a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31201020c0 0x7f31200ffd10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 --2- 172.21.15.5:0/4310302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31201029e0 0x7f3120100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 >> 172.21.15.5:0/4310302 conn(0x7f3120076f30 msgr2=0x7f31200fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 shutdown_connections 2023-09-23T04:48:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:55.762+0000 7f3124dff700 1 -- 172.21.15.5:0/4310302 wait complete. 2023-09-23T04:48:55.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:55 smithi005 ceph-mon[97638]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-09-23T04:48:55.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:48:55.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 134196 8.14 MB/sec warmup 91 sec latency 19.164 ms 2023-09-23T04:48:55.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:55 smithi069 ceph-mon[133931]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-09-23T04:48:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/1435733516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f48103310 msgr2=0x7f2f481036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/1435733516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f48103310 0x7f2f481036f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009990 tx=0x7f2f38009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/1435733516 shutdown_connections 2023-09-23T04:48:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/1435733516 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 0x7f2f4810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/1435733516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f48103310 0x7f2f481036f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/1435733516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103cc0 0x7f2f48107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/1435733516 >> 172.21.15.5:0/1435733516 conn(0x7f2f480feb50 msgr2=0x7f2f48100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/1435733516 shutdown_connections 2023-09-23T04:48:56.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.004+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/1435733516 wait complete. 2023-09-23T04:48:56.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.005+0000 7f2f4d7ff700 1 Processor -- start 2023-09-23T04:48:56.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.005+0000 7f2f4d7ff700 1 -- start start 2023-09-23T04:48:56.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 0x7f2f48199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4819dbc0 0x7f2f4819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819e640 con 0x7f2f4819dbc0 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819e7b0 con 0x7f2f48103310 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f4d7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4819e970 con 0x7f2f48108430 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52498/0 (socket says 172.21.15.5:52498) 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 -- 172.21.15.5:0/258811949 learned_addr learned my addr 172.21.15.5:0/258811949 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f467fc700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 0x7f2f48199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f477fe700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4819dbc0 0x7f2f4819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 -- 172.21.15.5:0/258811949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 msgr2=0x7f2f48199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 0x7f2f48199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 -- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4819dbc0 msgr2=0x7f2f4819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4819dbc0 0x7f2f4819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.006+0000 7f2f46ffd700 1 -- 172.21.15.5:0/258811949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009520 con 0x7f2f48103310 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005910 tx=0x7f2f38005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f2ffff700 1 -- 172.21.15.5:0/258811949 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3801d070 con 0x7f2f48103310 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f2ffff700 1 -- 172.21.15.5:0/258811949 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f38022470 con 0x7f2f48103310 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f380095f0 con 0x7f2f48103310 2023-09-23T04:48:56.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f2ffff700 1 -- 172.21.15.5:0/258811949 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38017650 con 0x7f2f48103310 2023-09-23T04:48:56.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.007+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481a6160 con 0x7f2f48103310 2023-09-23T04:48:56.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.011+0000 7f2f2ffff700 1 -- 172.21.15.5:0/258811949 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2f380225e0 con 0x7f2f48103310 2023-09-23T04:48:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.012+0000 7f2f6109db80 1 Processor -- start 2023-09-23T04:48:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.012+0000 7f2f2ffff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 0x7f2f300a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.012+0000 7f2f6109db80 1 -- start start 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.012+0000 7f2f2ffff700 1 -- 172.21.15.5:0/258811949 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2f380ce9e0 con 0x7f2f48103310 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f477fe700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 0x7f2f300a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd1631690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1631bd0 0x55bfd1633fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd158f460 con 0x55bfd1626400 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1580c20 con 0x55bfd1626d20 2023-09-23T04:48:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f6109db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd157e540 con 0x55bfd1631bd0 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58180/0 (socket says 172.21.15.5:58180) 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f46ffd700 1 -- 172.21.15.5:0/3683221183 learned_addr learned my addr 172.21.15.5:0/3683221183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f467fc700 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd1631690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f477fe700 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1631bd0 0x55bfd1633fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f46ffd700 1 -- 172.21.15.5:0/3683221183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1631bd0 msgr2=0x55bfd1633fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.013+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1631bd0 0x55bfd1633fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f46ffd700 1 -- 172.21.15.5:0/3683221183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 msgr2=0x55bfd1631690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd1631690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f46ffd700 1 -- 172.21.15.5:0/3683221183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd15dc8d0 con 0x55bfd1626400 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f477fe700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 0x7f2f300a3c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000f590 tx=0x7f2f4000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2f38098010 tx=0x7f2f38009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abcc08cd1182cd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f2d7fa700 1 -- 172.21.15.5:0/3683221183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3809af00 con 0x55bfd1626400 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f2d7fa700 1 -- 172.21.15.5:0/3683221183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3809bca0 con 0x55bfd1626400 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.014+0000 7f2f2d7fa700 1 -- 172.21.15.5:0/3683221183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3809b800 con 0x55bfd1626400 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- 172.21.15.5:0/3683221183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 msgr2=0x55bfd16267e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2f38098010 tx=0x7f2f38009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- 172.21.15.5:0/3683221183 shutdown_connections 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1631bd0 0x55bfd1633fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16267e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 --2- 172.21.15.5:0/3683221183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd1631690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- 172.21.15.5:0/3683221183 >> 172.21.15.5:0/3683221183 conn(0x55bfd159b260 msgr2=0x55bfd1622790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- 172.21.15.5:0/3683221183 shutdown_connections 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- 172.21.15.5:0/3683221183 wait complete. 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.015+0000 7f2f6109db80 1 -- start start 2023-09-23T04:48:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 Processor -- start 2023-09-23T04:48:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 0x55bfd1631fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 0x55bfd163d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1579e30 con 0x55bfd1631bd0 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1516710 con 0x55bfd1626d20 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd15163b0 con 0x55bfd1626400 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.016+0000 7f2f6109db80 10 client.0 _ll_register_callbacks cb 0x55bfd1373800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 0x55bfd1631fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52516/0 (socket says 172.21.15.5:52516) 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 -- 172.21.15.5:0/1311991481 learned_addr learned my addr 172.21.15.5:0/1311991481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 0x55bfd1631fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35264/0 (socket says 172.21.15.5:35264) 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f477fe700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 0x55bfd163d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 -- 172.21.15.5:0/1311991481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 msgr2=0x55bfd1631fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 0x55bfd1631fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 -- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 msgr2=0x55bfd163d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 0x55bfd163d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 -- 172.21.15.5:0/1311991481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481a6160 con 0x55bfd1626d20 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f477fe700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 0x55bfd163d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:48:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f467fc700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c012980 tx=0x7f2f3c012c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2bad48344d9493 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.017+0000 7f2f2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c017070 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2f3c00f830 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfd161a250 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3c0122e0 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 10 client.26342 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bfd13a4de0 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfd161a250 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2f3c010b60 con 0x55bfd1626d20 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 10 client.26342 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 20 client.26342 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1641490 0x55bfd16438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 4 client.26342 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.018+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfd1572e10 con 0x55bfd1641490 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.019+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1641490 0x55bfd16438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.019+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1641490 0x55bfd16438b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229aa63623c25467 server_cookie=3d49f9ae12273a1c in_seq=0 out_seq=0 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.019+0000 7f2f2aff5700 10 client.26342 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f2aff5700 1 -- 172.21.15.5:0/1311991481 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bfd1572e10 con 0x55bfd1641490 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f2aff5700 10 client.26342 handle_command_reply: tid=0 2023-09-23T04:48:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 client.26342 shutdown 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 msgr2=0x55bfd16324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c012980 tx=0x7f2f3c012c90 comp rx=0 tx=0).stop 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 shutdown_connections 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1641490 0x55bfd16438b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1631bd0 0x55bfd163d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626400 0x55bfd1631fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 --2- 172.21.15.5:0/1311991481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1626d20 0x55bfd16324f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f2aff5700 0 client.26342 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:56.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 >> 172.21.15.5:0/1311991481 conn(0x55bfd159b260 msgr2=0x55bfd1622320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 shutdown_connections 2023-09-23T04:48:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.020+0000 7f2f6109db80 1 -- 172.21.15.5:0/1311991481 wait complete. 2023-09-23T04:48:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.021+0000 7f2f6109db80 20 client.26342 trim_cache size 0 max 16384 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 Processor -- start 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 -- start start 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd163c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd163cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd13fa920 con 0x55bfd1626400 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd163e9c0 con 0x55bfd1631bd0 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.035+0000 7f2f6109db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1631900 con 0x55bfd1626d20 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58200/0 (socket says 172.21.15.5:58200) 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f46ffd700 1 -- 172.21.15.5:0/2667966673 learned_addr learned my addr 172.21.15.5:0/2667966673 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd163c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f477fe700 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd163cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 -- 172.21.15.5:0/2667966673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 msgr2=0x55bfd163cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd163cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 -- 172.21.15.5:0/2667966673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 msgr2=0x55bfd16305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 -- 172.21.15.5:0/2667966673 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd161e480 con 0x55bfd1626d20 2023-09-23T04:48:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.036+0000 7f2f467fc700 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd163c700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c010f10 tx=0x7f2f3c009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56642f5c3d70af7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f2dffb700 1 -- 172.21.15.5:0/2667966673 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c017040 con 0x55bfd1626d20 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f2dffb700 1 -- 172.21.15.5:0/2667966673 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3c005110 con 0x55bfd1626d20 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f2dffb700 1 -- 172.21.15.5:0/2667966673 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0038e0 con 0x55bfd1626d20 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 -- 172.21.15.5:0/2667966673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 msgr2=0x55bfd163c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd163c700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c010f10 tx=0x7f2f3c009480 comp rx=0 tx=0).stop 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 -- 172.21.15.5:0/2667966673 shutdown_connections 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd163cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd16305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 --2- 172.21.15.5:0/2667966673 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd163c700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.037+0000 7f2f6109db80 1 -- 172.21.15.5:0/2667966673 >> 172.21.15.5:0/2667966673 conn(0x55bfd159b260 msgr2=0x55bfd162c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.038+0000 7f2f6109db80 1 -- 172.21.15.5:0/2667966673 shutdown_connections 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.038+0000 7f2f6109db80 1 -- 172.21.15.5:0/2667966673 wait complete. 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.038+0000 7f2f6109db80 1 -- start start 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.038+0000 7f2f6109db80 1 Processor -- start 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd1632ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd1633430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd1633970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd16943f0 con 0x55bfd1626400 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1694560 con 0x55bfd1631bd0 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1694720 con 0x55bfd1626d20 2023-09-23T04:48:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f6109db80 10 client.0 _ll_register_callbacks cb 0x7f2f3c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd1632ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd1632ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58204/0 (socket says 172.21.15.5:58204) 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f467fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd1633430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f46ffd700 1 -- 172.21.15.5:0/528532917 learned_addr learned my addr 172.21.15.5:0/528532917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.039+0000 7f2f477fe700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd1633970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 -- 172.21.15.5:0/528532917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 msgr2=0x55bfd1633970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd1633970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 -- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 msgr2=0x55bfd1632ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd1632ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 -- 172.21.15.5:0/528532917 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd15dc8d0 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f467fc700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd1633430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55bfd1627760 tx=0x7f2f3c012500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968f025b338de6ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c017070 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f3809bca0 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.040+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2f3c003680 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3c005810 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 10 client.26404 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bfd13a4de0 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3809bca0 con 0x55bfd1626d20 2023-09-23T04:48:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2f3c010790 con 0x55bfd1626d20 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 10 client.26404 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 20 client.26404 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1695a90 0x55bfd1697e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 4 client.26404 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfd1572b30 con 0x55bfd1695a90 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.041+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1695a90 0x55bfd1697e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f46ffd700 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1695a90 0x55bfd1697e80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab43f705490f301 server_cookie=e23a7d86d327d691 in_seq=0 out_seq=0 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f2aff5700 10 client.26404 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f2aff5700 1 -- 172.21.15.5:0/528532917 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfd1572b30 con 0x55bfd1695a90 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f2aff5700 10 client.26404 handle_command_reply: tid=0 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 client.26404 shutdown 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 msgr2=0x55bfd1633430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd1633430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55bfd1627760 tx=0x7f2f3c012500 comp rx=0 tx=0).stop 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 shutdown_connections 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bfd1695a90 0x55bfd1697e80 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f2aff5700 0 client.26404 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bfd1631bd0 0x55bfd1633970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfd1626400 0x55bfd1632ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 --2- 172.21.15.5:0/528532917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bfd1626d20 0x55bfd1633430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 >> 172.21.15.5:0/528532917 conn(0x55bfd159b260 msgr2=0x55bfd1628460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 shutdown_connections 2023-09-23T04:48:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.042+0000 7f2f6109db80 1 -- 172.21.15.5:0/528532917 wait complete. 2023-09-23T04:48:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f6109db80 20 client.26404 trim_cache size 0 max 16384 2023-09-23T04:48:56.049 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:48:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 msgr2=0x7f2f300a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 0x7f2f300a3c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000f590 tx=0x7f2f4000b040 comp rx=0 tx=0).stop 2023-09-23T04:48:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 msgr2=0x7f2f48199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005910 tx=0x7f2f38005740 comp rx=0 tx=0).stop 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 shutdown_connections 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2f300a1790 0x7f2f300a3c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4819dbc0 0x7f2f4819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f48103310 0x7f2f48199220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 --2- 172.21.15.5:0/258811949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2f48108430 0x7f2f48199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.043+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 >> 172.21.15.5:0/258811949 conn(0x7f2f480feb50 msgr2=0x7f2f480ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.044+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 shutdown_connections 2023-09-23T04:48:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:56.044+0000 7f2f4d7ff700 1 -- 172.21.15.5:0/258811949 wait complete. 2023-09-23T04:48:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:55 smithi118 ceph-mon[134297]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-09-23T04:48:56.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135504 8.13 MB/sec warmup 92 sec latency 10.906 ms 2023-09-23T04:48:57.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:48:57.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 -- 172.21.15.5:0/2038698490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481010d0 msgr2=0x7f16481014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 --2- 172.21.15.5:0/2038698490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481010d0 0x7f16481014b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1644009990 tx=0x7f1644009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 -- 172.21.15.5:0/2038698490 shutdown_connections 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 --2- 172.21.15.5:0/2038698490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481061f0 0x7f164810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 --2- 172.21.15.5:0/2038698490 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481010d0 0x7f16481014b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 --2- 172.21.15.5:0/2038698490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1648101a80 0x7f1648105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 -- 172.21.15.5:0/2038698490 >> 172.21.15.5:0/2038698490 conn(0x7f16480fc990 msgr2=0x7f16480fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 -- 172.21.15.5:0/2038698490 shutdown_connections 2023-09-23T04:48:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.291+0000 7f16503ff700 1 -- 172.21.15.5:0/2038698490 wait complete. 2023-09-23T04:48:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.292+0000 7f16503ff700 1 Processor -- start 2023-09-23T04:48:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.292+0000 7f16503ff700 1 -- start start 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481061f0 0x7f16481996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 0x7f164819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e580 con 0x7f164819db00 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e6f0 con 0x7f16481010d0 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f16503ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164819e8b0 con 0x7f16481061f0 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52556/0 (socket says 172.21.15.5:52556) 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 -- 172.21.15.5:0/1897875361 learned_addr learned my addr 172.21.15.5:0/1897875361 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e99c700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 0x7f164819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164d99a700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481061f0 0x7f16481996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 -- 172.21.15.5:0/1897875361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481061f0 msgr2=0x7f16481996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481061f0 0x7f16481996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 -- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 msgr2=0x7f164819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 0x7f164819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e19b700 1 -- 172.21.15.5:0/1897875361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440095f0 con 0x7f16481010d0 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.293+0000 7f164e99c700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 0x7f164819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f164e19b700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1644005910 tx=0x7f164400bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f163f7fe700 1 -- 172.21.15.5:0/1897875361 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164401d070 con 0x7f16481010d0 2023-09-23T04:48:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16481a5db0 con 0x7f16481010d0 2023-09-23T04:48:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f163f7fe700 1 -- 172.21.15.5:0/1897875361 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1644022470 con 0x7f16481010d0 2023-09-23T04:48:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f163f7fe700 1 -- 172.21.15.5:0/1897875361 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644017650 con 0x7f16481010d0 2023-09-23T04:48:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.294+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16481a61d0 con 0x7f16481010d0 2023-09-23T04:48:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.298+0000 7f163f7fe700 1 -- 172.21.15.5:0/1897875361 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f16440225e0 con 0x7f16481010d0 2023-09-23T04:48:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.300+0000 7f163f7fe700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 0x7f16340a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.300+0000 7f163f7fe700 1 -- 172.21.15.5:0/1897875361 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f16440cea20 con 0x7f16481010d0 2023-09-23T04:48:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.300+0000 7f1663db1b80 1 Processor -- start 2023-09-23T04:48:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.300+0000 7f164d99a700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 0x7f16340a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.300+0000 7f1663db1b80 1 -- start start 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cabda0 0x5579a2cb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cb9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2c144e0 con 0x5579a2cabda0 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2c05ca0 con 0x5579a2cab480 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f1663db1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2c035c0 con 0x5579a2cb6c50 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e99c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cb9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52570/0 (socket says 172.21.15.5:52570) 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 -- 172.21.15.5:0/1365333016 learned_addr learned my addr 172.21.15.5:0/1365333016 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164d99a700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 0x7f16340a3bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f164819a770 tx=0x7f1638009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164d99a700 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cabda0 0x5579a2cb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 -- 172.21.15.5:0/1365333016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 msgr2=0x5579a2cb9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cb9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 -- 172.21.15.5:0/1365333016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cabda0 msgr2=0x5579a2cb6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cabda0 0x5579a2cb6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.301+0000 7f164e19b700 1 -- 172.21.15.5:0/1365333016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a2c61950 con 0x5579a2cab480 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.302+0000 7f164e19b700 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1644024000 tx=0x7f1644097a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d8adc2198f8d8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.302+0000 7f163cbfe700 1 -- 172.21.15.5:0/1365333016 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164409bce0 con 0x5579a2cab480 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.302+0000 7f163cbfe700 1 -- 172.21.15.5:0/1365333016 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164409b5e0 con 0x5579a2cab480 2023-09-23T04:48:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.302+0000 7f163cbfe700 1 -- 172.21.15.5:0/1365333016 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164409baa0 con 0x5579a2cab480 2023-09-23T04:48:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- 172.21.15.5:0/1365333016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 msgr2=0x5579a2cab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1644024000 tx=0x7f1644097a50 comp rx=0 tx=0).stop 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- 172.21.15.5:0/1365333016 shutdown_connections 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cb9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cab480 0x5579a2cab860 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1365333016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cabda0 0x5579a2cb6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- 172.21.15.5:0/1365333016 >> 172.21.15.5:0/1365333016 conn(0x5579a2c202e0 msgr2=0x5579a2ca7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- 172.21.15.5:0/1365333016 shutdown_connections 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- 172.21.15.5:0/1365333016 wait complete. 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.303+0000 7f1663db1b80 1 -- start start 2023-09-23T04:48:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.304+0000 7f1663db1b80 1 Processor -- start 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.304+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cabda0 0x5579a2cb7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cb6c50 0x5579a2cc20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2bfeeb0 con 0x5579a2cab480 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2b9b790 con 0x5579a2cb6c50 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2b9b430 con 0x5579a2cabda0 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f1663db1b80 10 client.0 _ll_register_callbacks cb 0x5579a29f8880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cb6c50 0x5579a2cc20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58236/0 (socket says 172.21.15.5:58236) 2023-09-23T04:48:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 -- 172.21.15.5:0/1364354920 learned_addr learned my addr 172.21.15.5:0/1364354920 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 -- 172.21.15.5:0/1364354920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cabda0 msgr2=0x5579a2cb7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cabda0 0x5579a2cb7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 -- 172.21.15.5:0/1364354920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cb6c50 msgr2=0x5579a2cc20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cb6c50 0x5579a2cc20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.305+0000 7f164e19b700 1 -- 172.21.15.5:0/1364354920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16481a61d0 con 0x5579a2cab480 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f164e19b700 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1644005f50 tx=0x7f1644022b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69b25e20d3e516a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440a38e0 con 0x5579a2cab480 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579a2c9f2d0 con 0x5579a2cab480 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f16440b0020 con 0x5579a2cab480 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f164409f730 con 0x5579a2cab480 2023-09-23T04:48:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1663db1b80 10 client.16365 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5579a2a29e60 con 0x5579a2cab480 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.306+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579a2c9f2d0 con 0x5579a2cab480 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f16440973d0 con 0x5579a2cab480 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 20 client.16365 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2cc5ed0 0x5579a2cc82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 4 client.16365 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579a2bf7e90 con 0x5579a2cc5ed0 2023-09-23T04:48:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.307+0000 7f164d99a700 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2cc5ed0 0x5579a2cc82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.308+0000 7f164d99a700 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2cc5ed0 0x5579a2cc82c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73c9aab5aadf5c1 server_cookie=39f2faf296a76056 in_seq=0 out_seq=0 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.308+0000 7f1631ffb700 10 client.16365 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.308+0000 7f1631ffb700 1 -- 172.21.15.5:0/1364354920 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5579a2bf7e90 con 0x5579a2cc5ed0 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.308+0000 7f1631ffb700 10 client.16365 handle_command_reply: tid=0 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.308+0000 7f1663db1b80 1 client.16365 shutdown 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 msgr2=0x5579a2cb7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1644005f50 tx=0x7f1644022b30 comp rx=0 tx=0).stop 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 shutdown_connections 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2cc5ed0 0x5579a2cc82c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cb6c50 0x5579a2cc20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cb7030 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 --2- 172.21.15.5:0/1364354920 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cabda0 0x5579a2cb7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 >> 172.21.15.5:0/1364354920 conn(0x5579a2c202e0 msgr2=0x5579a2ca73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1631ffb700 0 client.16365 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 shutdown_connections 2023-09-23T04:48:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 1 -- 172.21.15.5:0/1364354920 wait complete. 2023-09-23T04:48:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.309+0000 7f1663db1b80 20 client.16365 trim_cache size 0 max 16384 2023-09-23T04:48:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 Processor -- start 2023-09-23T04:48:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 -- start start 2023-09-23T04:48:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cc1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cb6c50 0x5579a2cc1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2a7f9a0 con 0x5579a2cb6c50 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2cbc560 con 0x5579a2cabda0 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f1663db1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2cacc70 con 0x5579a2cab480 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.328+0000 7f164e19b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164d99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cc1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:35300/0 (socket says 172.21.15.5:35300) 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 -- 172.21.15.5:0/3014569137 learned_addr learned my addr 172.21.15.5:0/3014569137 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e99c700 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cb6c50 0x5579a2cc1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 -- 172.21.15.5:0/3014569137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 msgr2=0x5579a2cc1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cc1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 -- 172.21.15.5:0/3014569137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cb6c50 msgr2=0x5579a2cc1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cb6c50 0x5579a2cc1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 -- 172.21.15.5:0/3014569137 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579a2a12ce0 con 0x5579a2cab480 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164d99a700 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cc1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f164e19b700 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16440a33b0 tx=0x7f1644022cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1d0c3d91a9a477 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f163d3ff700 1 -- 172.21.15.5:0/3014569137 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440a38e0 con 0x5579a2cab480 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f163d3ff700 1 -- 172.21.15.5:0/3014569137 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164409b120 con 0x5579a2cab480 2023-09-23T04:48:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.329+0000 7f163d3ff700 1 -- 172.21.15.5:0/3014569137 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644084d20 con 0x5579a2cab480 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 -- 172.21.15.5:0/3014569137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 msgr2=0x5579a2cbc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16440a33b0 tx=0x7f1644022cd0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 -- 172.21.15.5:0/3014569137 shutdown_connections 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cb6c50 0x5579a2cc1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cab480 0x5579a2cbc020 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3014569137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cc1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 -- 172.21.15.5:0/3014569137 >> 172.21.15.5:0/3014569137 conn(0x5579a2c202e0 msgr2=0x5579a2cb4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 -- 172.21.15.5:0/3014569137 shutdown_connections 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.330+0000 7f1663db1b80 1 -- 172.21.15.5:0/3014569137 wait complete. 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.331+0000 7f1663db1b80 1 -- start start 2023-09-23T04:48:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.331+0000 7f1663db1b80 1 Processor -- start 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.331+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cbb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2d19e10 con 0x5579a2cab480 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2d19f50 con 0x5579a2cabda0 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579a2d1a110 con 0x5579a2cb6c50 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f1663db1b80 10 client.0 _ll_register_callbacks cb 0x5579a29f8880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164e99c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cbb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52592/0 (socket says 172.21.15.5:52592) 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 -- 172.21.15.5:0/3667920180 learned_addr learned my addr 172.21.15.5:0/3667920180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164e19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 -- 172.21.15.5:0/3667920180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 msgr2=0x5579a2cbb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cbb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 -- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 msgr2=0x5579a2cba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164d99a700 1 -- 172.21.15.5:0/3667920180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1638011970 con 0x5579a2cabda0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.332+0000 7f164e99c700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cbb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f164d99a700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5579a2cc8bd0 tx=0x7f16380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf49d525936ce064 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638025040 con 0x5579a2cabda0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f164409b5e0 con 0x5579a2cabda0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1638018070 con 0x5579a2cabda0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16380095c0 con 0x5579a2cabda0 2023-09-23T04:48:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1663db1b80 10 client.26366 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5579a2a29e60 con 0x5579a2cabda0 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.333+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164409b5e0 con 0x5579a2cabda0 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1638028b50 con 0x5579a2cabda0 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 10 client.26366 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 20 client.26366 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2d1b9d0 0x5579a2d1ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 4 client.26366 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5579a2bf7bb0 con 0x5579a2d1b9d0 2023-09-23T04:48:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f164e19b700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2d1b9d0 0x5579a2d1ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f164e19b700 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2d1b9d0 0x5579a2d1ddf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692dc153d12950c4 server_cookie=bf6c363c350b7488 in_seq=0 out_seq=0 2023-09-23T04:48:57.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.334+0000 7f1631ffb700 10 client.26366 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.335+0000 7f1631ffb700 1 -- 172.21.15.5:0/3667920180 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5579a2bf7bb0 con 0x5579a2d1b9d0 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.335+0000 7f1631ffb700 10 client.26366 handle_command_reply: tid=0 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.335+0000 7f1663db1b80 1 client.26366 shutdown 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 msgr2=0x5579a2cbaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5579a2cc8bd0 tx=0x7f16380047c0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 shutdown_connections 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5579a2d1b9d0 0x5579a2d1ddf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5579a2cb6c50 0x5579a2cbb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579a2cab480 0x5579a2cba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 --2- 172.21.15.5:0/3667920180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5579a2cabda0 0x5579a2cbaba0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1631ffb700 0 client.26366 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 >> 172.21.15.5:0/3667920180 conn(0x5579a2c202e0 msgr2=0x5579a2ca6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 shutdown_connections 2023-09-23T04:48:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 1 -- 172.21.15.5:0/3667920180 wait complete. 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.336+0000 7f1663db1b80 20 client.26366 trim_cache size 0 max 16384 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "051025d1-0029-48f1-a302-265219ea98b8", 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:48:57.340 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 msgr2=0x7f16340a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 0x7f16340a3bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f164819a770 tx=0x7f1638009260 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 msgr2=0x7f1648199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1644005910 tx=0x7f164400bfc0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 shutdown_connections 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16340a1710 0x7f16340a3bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f164819db00 0x7f164819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16481010d0 0x7f1648199160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 --2- 172.21.15.5:0/1897875361 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16481061f0 0x7f16481996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 >> 172.21.15.5:0/1897875361 conn(0x7f16480fc990 msgr2=0x7f164810d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 shutdown_connections 2023-09-23T04:48:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.337+0000 7f16503ff700 1 -- 172.21.15.5:0/1897875361 wait complete. 2023-09-23T04:48:57.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 -- 172.21.15.5:0/1545473413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400101e00 msgr2=0x7fe4001021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1545473413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400101e00 0x7fe4001021e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009990 tx=0x7fe3f0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 -- 172.21.15.5:0/1545473413 shutdown_connections 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1545473413 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 0x7fe40010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1545473413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400101e00 0x7fe4001021e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1545473413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400102720 0x7fe40010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 -- 172.21.15.5:0/1545473413 >> 172.21.15.5:0/1545473413 conn(0x7fe4000fb580 msgr2=0x7fe4000fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 -- 172.21.15.5:0/1545473413 shutdown_connections 2023-09-23T04:48:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.603+0000 7fe3fffff700 1 -- 172.21.15.5:0/1545473413 wait complete. 2023-09-23T04:48:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.604+0000 7fe3fffff700 1 Processor -- start 2023-09-23T04:48:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 -- start start 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 0x7fe4001a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400100830 con 0x7fe400101e00 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001a5420 con 0x7fe400102720 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.605+0000 7fe3fffff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001a5560 con 0x7fe40010b1d0 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52614/0 (socket says 172.21.15.5:52614) 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3feffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58260/0 (socket says 172.21.15.5:58260) 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 -- 172.21.15.5:0/1899417938 learned_addr learned my addr 172.21.15.5:0/1899417938 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 0x7fe4001a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 -- 172.21.15.5:0/1899417938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 msgr2=0x7fe4001a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 0x7fe4001a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 -- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 msgr2=0x7fe4000ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 -- 172.21.15.5:0/1899417938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8009520 con 0x7fe400102720 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3feffd700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.606+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800eb60 tx=0x7fe3e800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.607+0000 7fe3fcff9700 1 -- 172.21.15.5:0/1899417938 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800cbd0 con 0x7fe400102720 2023-09-23T04:48:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.607+0000 7fe3fcff9700 1 -- 172.21.15.5:0/1899417938 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e80044e0 con 0x7fe400102720 2023-09-23T04:48:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.607+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f00095f0 con 0x7fe400102720 2023-09-23T04:48:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.607+0000 7fe3fcff9700 1 -- 172.21.15.5:0/1899417938 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8003df0 con 0x7fe400102720 2023-09-23T04:48:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.607+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001a5be0 con 0x7fe400102720 2023-09-23T04:48:57.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.610+0000 7fe3fcff9700 1 -- 172.21.15.5:0/1899417938 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe3e8004000 con 0x7fe400102720 2023-09-23T04:48:57.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.611+0000 7fe418d18b80 1 Processor -- start 2023-09-23T04:48:57.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 -- start start 2023-09-23T04:48:57.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877e90 0x558d2f882800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 0x558d2f885130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f7e0460 con 0x558d2f882d40 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f7d1c20 con 0x558d2f877570 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe418d18b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f7cf540 con 0x558d2f877e90 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52626/0 (socket says 172.21.15.5:52626) 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/4255019841 learned_addr learned my addr 172.21.15.5:0/4255019841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 0x558d2f885130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.612+0000 7fe3feffd700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877e90 0x558d2f882800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3fcff9700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 0x7fe3ec0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/4255019841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877e90 msgr2=0x558d2f882800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877e90 0x558d2f882800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/4255019841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 msgr2=0x558d2f885130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 0x558d2f885130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/4255019841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2f82d8d0 con 0x558d2f877570 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 0x558d2f885130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:48:57.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3fcff9700 1 -- 172.21.15.5:0/1899417938 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe3e8014070 con 0x7fe400102720 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8011680 tx=0x7fe3f8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602c7e64918ba172 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.613+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 0x7fe3ec0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe3f57fa700 1 -- 172.21.15.5:0/4255019841 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800e040 con 0x558d2f877570 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe3f57fa700 1 -- 172.21.15.5:0/4255019841 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f800f040 con 0x558d2f877570 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe3f57fa700 1 -- 172.21.15.5:0/4255019841 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007900 con 0x558d2f877570 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 -- 172.21.15.5:0/4255019841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 msgr2=0x558d2f877950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 0x7fe3ec0a3c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8011300 tx=0x7fe3f8007b40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8011680 tx=0x7fe3f8009920 comp rx=0 tx=0).stop 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 -- 172.21.15.5:0/4255019841 shutdown_connections 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f882d40 0x558d2f885130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f877950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 --2- 172.21.15.5:0/4255019841 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877e90 0x558d2f882800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.614+0000 7fe418d18b80 1 -- 172.21.15.5:0/4255019841 >> 172.21.15.5:0/4255019841 conn(0x558d2f7ec260 msgr2=0x558d2f8739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.615+0000 7fe418d18b80 1 -- 172.21.15.5:0/4255019841 shutdown_connections 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.615+0000 7fe418d18b80 1 -- 172.21.15.5:0/4255019841 wait complete. 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.615+0000 7fe418d18b80 1 -- start start 2023-09-23T04:48:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.615+0000 7fe418d18b80 1 Processor -- start 2023-09-23T04:48:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f8827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f87fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f880140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f7cae30 con 0x558d2f877e90 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f767710 con 0x558d2f882d40 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f7673b0 con 0x558d2f877570 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.616+0000 7fe418d18b80 10 client.0 _ll_register_callbacks cb 0x558d2f5c4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f880140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f880140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52634/0 (socket says 172.21.15.5:52634) 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/3994795210 learned_addr learned my addr 172.21.15.5:0/3994795210 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f8827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f87fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 -- 172.21.15.5:0/3994795210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 msgr2=0x558d2f880140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f880140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 -- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 msgr2=0x558d2f87fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f87fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 -- 172.21.15.5:0/3994795210 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8011330 con 0x558d2f877570 2023-09-23T04:48:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.617+0000 7fe3feffd700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f8827f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4360836fe0a3f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe3e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f001d070 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d2f86b4a0 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe3f0004910 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f0004ad0 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe418d18b80 10 client.26431 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558d2f5f5de0 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.618+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d2f86b4a0 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe3f000fc90 con 0x558d2f877570 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 10 client.26431 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:57.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 10 client.26431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 20 client.26431 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8928a0 0x558d2f894cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 4 client.26431 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d2f7c3e10 con 0x558d2f8928a0 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.619+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8928a0 0x558d2f894cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.620+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8928a0 0x558d2f894cc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606df4fb2426ecb6 server_cookie=3a0ffeee78147b4c in_seq=0 out_seq=0 2023-09-23T04:48:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.620+0000 7fe3e67fc700 10 client.26431 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.620+0000 7fe3e67fc700 1 -- 172.21.15.5:0/3994795210 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558d2f7c3e10 con 0x558d2f8928a0 2023-09-23T04:48:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.620+0000 7fe3e67fc700 10 client.26431 handle_command_reply: tid=0 2023-09-23T04:48:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.620+0000 7fe418d18b80 1 client.26431 shutdown 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 msgr2=0x558d2f8827f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f8827f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0005270 comp rx=0 tx=0).stop 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 shutdown_connections 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8928a0 0x558d2f894cc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f880140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f8827f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 --2- 172.21.15.5:0/3994795210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f87fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 >> 172.21.15.5:0/3994795210 conn(0x558d2f7ec260 msgr2=0x558d2f8735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe3e67fc700 0 client.26431 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 shutdown_connections 2023-09-23T04:48:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 1 -- 172.21.15.5:0/3994795210 wait complete. 2023-09-23T04:48:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.621+0000 7fe418d18b80 20 client.26431 trim_cache size 0 max 16384 2023-09-23T04:48:57.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.642+0000 7fe418d18b80 1 Processor -- start 2023-09-23T04:48:57.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.642+0000 7fe418d18b80 1 -- start start 2023-09-23T04:48:57.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f87a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f8732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f64b920 con 0x558d2f877e90 2023-09-23T04:48:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f878820 con 0x558d2f882d40 2023-09-23T04:48:57.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe418d18b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f889e30 con 0x558d2f877570 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f8732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f8732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52646/0 (socket says 172.21.15.5:52646) 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe3ff7fe700 1 -- 172.21.15.5:0/952552626 learned_addr learned my addr 172.21.15.5:0/952552626 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.643+0000 7fe3feffd700 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f87a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 -- 172.21.15.5:0/952552626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 msgr2=0x558d2f87a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f87a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 -- 172.21.15.5:0/952552626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 msgr2=0x558d2f8732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f8732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 -- 172.21.15.5:0/952552626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2f5dec60 con 0x558d2f877e90 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3feffd700 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f87a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8898f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80cad70 tx=0x7fe3e80cfcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5befe9dc5806d424 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f5ffb700 1 -- 172.21.15.5:0/952552626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8099ef0 con 0x558d2f877e90 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f5ffb700 1 -- 172.21.15.5:0/952552626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e809ac90 con 0x558d2f877e90 2023-09-23T04:48:57.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.644+0000 7fe3f5ffb700 1 -- 172.21.15.5:0/952552626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80042b0 con 0x558d2f877e90 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 -- 172.21.15.5:0/952552626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 msgr2=0x558d2f8898f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8898f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80cad70 tx=0x7fe3e80cfcb0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 -- 172.21.15.5:0/952552626 shutdown_connections 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f882d40 0x558d2f8732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f877570 0x558d2f87a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 --2- 172.21.15.5:0/952552626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8898f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 -- 172.21.15.5:0/952552626 >> 172.21.15.5:0/952552626 conn(0x558d2f7ec260 msgr2=0x558d2f872bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 -- 172.21.15.5:0/952552626 shutdown_connections 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.645+0000 7fe418d18b80 1 -- 172.21.15.5:0/952552626 wait complete. 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 -- start start 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 Processor -- start 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f882d40 0x558d2f887ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f8885b0 con 0x558d2f877e90 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f888720 con 0x558d2f877570 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2f8e6720 con 0x558d2f882d40 2023-09-23T04:48:57.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.646+0000 7fe418d18b80 10 client.0 _ll_register_callbacks cb 0x7fe3f000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:52654/0 (socket says 172.21.15.5:52654) 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 -- 172.21.15.5:0/688057548 learned_addr learned my addr 172.21.15.5:0/688057548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3ff7fe700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f882d40 0x558d2f887ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 -- 172.21.15.5:0/688057548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f882d40 msgr2=0x558d2f887ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f882d40 0x558d2f887ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 -- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 msgr2=0x558d2f8879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 -- 172.21.15.5:0/688057548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d2f5f5de0 con 0x558d2f877570 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3feffd700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0022ad0 tx=0x7fe3f0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1592958842418b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:48:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.647+0000 7fe3e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f001d040 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3f800f040 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe3f0024970 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f0005990 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 10 client.26384 fetch_fsmap learned FSMap version 29 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe4001a5be0 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f800f040 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe3f002e050 con 0x558d2f877570 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 10 client.26384 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 10 client.26384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 20 client.26384 populate_metadata read hostname 'smithi005' 2023-09-23T04:48:57.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8e79f0 0x558d2f8e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:48:57.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 4 client.26384 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:48:57.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.648+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d2f7c3b30 con 0x558d2f8e79f0 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.649+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8e79f0 0x558d2f8e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.649+0000 7fe3f7fff700 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8e79f0 0x558d2f8e9e10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b510c3cbee936f server_cookie=afebd37887648bf9 in_seq=0 out_seq=0 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.649+0000 7fe3e67fc700 10 client.26384 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe3e67fc700 1 -- 172.21.15.5:0/688057548 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558d2f7c3b30 con 0x558d2f8e79f0 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe3e67fc700 10 client.26384 handle_command_reply: tid=0 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 client.26384 shutdown 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 msgr2=0x558d2f889720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0022ad0 tx=0x7fe3f0005f10 comp rx=0 tx=0).stop 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 shutdown_connections 2023-09-23T04:48:57.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558d2f8e79f0 0x558d2f8e9e10 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558d2f882d40 0x558d2f887ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d2f877570 0x558d2f889720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 --2- 172.21.15.5:0/688057548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d2f877e90 0x558d2f8879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe3e67fc700 0 client.26384 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 >> 172.21.15.5:0/688057548 conn(0x558d2f7ec260 msgr2=0x558d2f873450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 shutdown_connections 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 1 -- 172.21.15.5:0/688057548 wait complete. 2023-09-23T04:48:57.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.650+0000 7fe418d18b80 20 client.26384 trim_cache size 0 max 16384 2023-09-23T04:48:57.657 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:48:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:48:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "051025d1-0029-48f1-a302-265219ea98b8": { 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "051025d1-0029-48f1-a302-265219ea98b8", 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:48:57.658 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 msgr2=0x7fe3ec0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 0x7fe3ec0a3c50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8011300 tx=0x7fe3f8007b40 comp rx=0 tx=0).stop 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 msgr2=0x7fe4000fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800eb60 tx=0x7fe3e800ee70 comp rx=0 tx=0).stop 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 shutdown_connections 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3ec0a1790 0x7fe3ec0a3c50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe40010b1d0 0x7fe4001a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe400101e00 0x7fe4000ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 --2- 172.21.15.5:0/1899417938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe400102720 0x7fe4000fff90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:48:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 >> 172.21.15.5:0/1899417938 conn(0x7fe4000fb580 msgr2=0x7fe4000fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:48:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 shutdown_connections 2023-09-23T04:48:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:48:57.651+0000 7fe3fffff700 1 -- 172.21.15.5:0/1899417938 wait complete. 2023-09-23T04:48:57.670 INFO:tasks.cephfs.filesystem:scrub status for tag:051025d1-0029-48f1-a302-265219ea98b8 - {'path': '/', 'tag': '051025d1-0029-48f1-a302-265219ea98b8', 'options': 'recursive,force'} 2023-09-23T04:48:57.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 137588 8.16 MB/sec warmup 93 sec latency 40.337 ms 2023-09-23T04:48:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:57 smithi069 ceph-mon[133931]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 184 op/s 2023-09-23T04:48:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:57 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:57 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:57 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:57.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:57 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:57 smithi005 ceph-mon[97638]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 184 op/s 2023-09-23T04:48:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:57 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:57 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:57 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:57 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:57 smithi118 ceph-mon[134297]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 184 op/s 2023-09-23T04:48:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:57 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:57 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:57 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:58.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:57 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138944 8.16 MB/sec warmup 94 sec latency 27.366 ms 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:48:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:48:58 smithi069 ceph-mon[133931]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-09-23T04:48:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:48:59.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:48:58 smithi005 ceph-mon[97638]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-09-23T04:48:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:48:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:48:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:48:58 smithi118 ceph-mon[134297]: pgmap v84: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2023-09-23T04:48:59.932 INFO:tasks.workunit.client.0.smithi005.stdout: 1 141297 8.20 MB/sec warmup 95 sec latency 10.229 ms 2023-09-23T04:49:00.932 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142594 8.19 MB/sec warmup 96 sec latency 10.710 ms 2023-09-23T04:49:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:01 smithi005 ceph-mon[97638]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1000 KiB/s wr, 181 op/s 2023-09-23T04:49:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:01 smithi118 ceph-mon[134297]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1000 KiB/s wr, 181 op/s 2023-09-23T04:49:01.931 INFO:tasks.workunit.client.0.smithi005.stdout: 1 143150 8.12 MB/sec warmup 97 sec latency 54.339 ms 2023-09-23T04:49:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:01 smithi069 ceph-mon[133931]: pgmap v85: 129 pgs: 129 active+clean; 86 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1000 KiB/s wr, 181 op/s 2023-09-23T04:49:02.932 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145658 8.21 MB/sec warmup 98 sec latency 13.492 ms 2023-09-23T04:49:03.932 INFO:tasks.workunit.client.0.smithi005.stdout: 1 146739 8.15 MB/sec warmup 99 sec latency 21.740 ms 2023-09-23T04:49:03.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:03 smithi069 ceph-mon[133931]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2023-09-23T04:49:04.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:03 smithi005 ceph-mon[97638]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2023-09-23T04:49:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:03 smithi118 ceph-mon[134297]: pgmap v86: 129 pgs: 129 active+clean; 90 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2023-09-23T04:49:04.932 INFO:tasks.workunit.client.0.smithi005.stdout: 1 149230 8.24 MB/sec warmup 100 sec latency 12.228 ms 2023-09-23T04:49:05.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150458 8.21 MB/sec warmup 101 sec latency 10.564 ms 2023-09-23T04:49:05.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:05 smithi069 ceph-mon[133931]: pgmap v87: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2023-09-23T04:49:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:05 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:05 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:05 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:05 smithi005 ceph-mon[97638]: pgmap v87: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2023-09-23T04:49:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:05 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:05 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:06.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:05 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:05 smithi118 ceph-mon[134297]: pgmap v87: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2023-09-23T04:49:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:05 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:05 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:05 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:06.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 152802 8.26 MB/sec warmup 102 sec latency 12.975 ms 2023-09-23T04:49:06.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:06 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:06 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:06 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:07.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 154218 8.24 MB/sec warmup 103 sec latency 10.095 ms 2023-09-23T04:49:07.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:07 smithi069 ceph-mon[133931]: pgmap v88: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2023-09-23T04:49:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:07 smithi005 ceph-mon[97638]: pgmap v88: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2023-09-23T04:49:08.204 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:07 smithi118 ceph-mon[134297]: pgmap v88: 129 pgs: 129 active+clean; 91 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2023-09-23T04:49:08.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156339 8.29 MB/sec warmup 104 sec latency 32.660 ms 2023-09-23T04:49:08.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:08 smithi069 ceph-mon[133931]: pgmap v89: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2023-09-23T04:49:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:08 smithi005 ceph-mon[97638]: pgmap v89: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2023-09-23T04:49:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:08 smithi118 ceph-mon[134297]: pgmap v89: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2023-09-23T04:49:09.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 157616 8.26 MB/sec warmup 105 sec latency 42.944 ms 2023-09-23T04:49:10.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 159279 8.26 MB/sec warmup 106 sec latency 417.533 ms 2023-09-23T04:49:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:11 smithi005 ceph-mon[97638]: pgmap v90: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.2 MiB/s wr, 206 op/s 2023-09-23T04:49:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:11 smithi118 ceph-mon[134297]: pgmap v90: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.2 MiB/s wr, 206 op/s 2023-09-23T04:49:11.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160445 8.24 MB/sec warmup 107 sec latency 12.366 ms 2023-09-23T04:49:11.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:11 smithi069 ceph-mon[133931]: pgmap v90: 129 pgs: 129 active+clean; 97 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.2 MiB/s wr, 206 op/s 2023-09-23T04:49:12.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 162828 8.28 MB/sec warmup 108 sec latency 14.765 ms 2023-09-23T04:49:13.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 163803 8.26 MB/sec warmup 109 sec latency 67.662 ms 2023-09-23T04:49:13.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:13 smithi069 ceph-mon[133931]: pgmap v91: 129 pgs: 129 active+clean; 103 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2023-09-23T04:49:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:13 smithi005 ceph-mon[97638]: pgmap v91: 129 pgs: 129 active+clean; 103 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2023-09-23T04:49:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:13 smithi118 ceph-mon[134297]: pgmap v91: 129 pgs: 129 active+clean; 103 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 281 op/s 2023-09-23T04:49:14.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 165590 8.25 MB/sec warmup 110 sec latency 142.885 ms 2023-09-23T04:49:15.933 INFO:tasks.workunit.client.0.smithi005.stdout: 1 166995 8.28 MB/sec warmup 111 sec latency 44.740 ms 2023-09-23T04:49:15.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:15 smithi069 ceph-mon[133931]: pgmap v92: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2023-09-23T04:49:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:15 smithi005 ceph-mon[97638]: pgmap v92: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2023-09-23T04:49:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:15 smithi118 ceph-mon[134297]: pgmap v92: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 206 op/s 2023-09-23T04:49:16.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 168303 8.26 MB/sec warmup 112 sec latency 24.424 ms 2023-09-23T04:49:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:16 smithi005 ceph-mon[97638]: pgmap v93: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-09-23T04:49:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:16 smithi118 ceph-mon[134297]: pgmap v93: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-09-23T04:49:17.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:16 smithi069 ceph-mon[133931]: pgmap v93: 129 pgs: 129 active+clean; 104 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-09-23T04:49:17.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 170537 8.30 MB/sec warmup 113 sec latency 20.421 ms 2023-09-23T04:49:18.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171761 8.28 MB/sec warmup 114 sec latency 73.890 ms 2023-09-23T04:49:19.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 174023 8.33 MB/sec warmup 115 sec latency 49.341 ms 2023-09-23T04:49:19.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:19 smithi069 ceph-mon[133931]: pgmap v94: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 382 op/s 2023-09-23T04:49:20.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:19 smithi005 ceph-mon[97638]: pgmap v94: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 382 op/s 2023-09-23T04:49:20.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:19 smithi118 ceph-mon[134297]: pgmap v94: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 382 op/s 2023-09-23T04:49:20.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 175001 8.28 MB/sec warmup 116 sec latency 30.735 ms 2023-09-23T04:49:21.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:21 smithi005 ceph-mon[97638]: pgmap v95: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 290 op/s 2023-09-23T04:49:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:21 smithi118 ceph-mon[134297]: pgmap v95: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 290 op/s 2023-09-23T04:49:21.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 177575 8.35 MB/sec warmup 117 sec latency 11.503 ms 2023-09-23T04:49:21.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:21 smithi069 ceph-mon[133931]: pgmap v95: 129 pgs: 129 active+clean; 104 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 290 op/s 2023-09-23T04:49:22.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178801 8.32 MB/sec warmup 118 sec latency 10.256 ms 2023-09-23T04:49:23.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181155 8.37 MB/sec warmup 119 sec latency 12.943 ms 2023-09-23T04:49:23.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:23 smithi069 ceph-mon[133931]: pgmap v96: 129 pgs: 129 active+clean; 108 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 395 op/s 2023-09-23T04:49:24.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:23 smithi005 ceph-mon[97638]: pgmap v96: 129 pgs: 129 active+clean; 108 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 395 op/s 2023-09-23T04:49:24.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:23 smithi118 ceph-mon[134297]: pgmap v96: 129 pgs: 129 active+clean; 108 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.9 MiB/s wr, 395 op/s 2023-09-23T04:49:25.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 184672 13.72 MB/sec execute 1 sec latency 27.936 ms 2023-09-23T04:49:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:25 smithi069 ceph-mon[133931]: pgmap v97: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 332 op/s 2023-09-23T04:49:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:25 smithi005 ceph-mon[97638]: pgmap v97: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 332 op/s 2023-09-23T04:49:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:25 smithi118 ceph-mon[134297]: pgmap v97: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 332 op/s 2023-09-23T04:49:26.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 185925 9.56 MB/sec execute 2 sec latency 20.470 ms 2023-09-23T04:49:27.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 -- 172.21.15.5:0/1043354350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c0ffeb0 msgr2=0x7f3f2c100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 --2- 172.21.15.5:0/1043354350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c100290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009990 tx=0x7f3f1c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 -- 172.21.15.5:0/1043354350 shutdown_connections 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 --2- 172.21.15.5:0/1043354350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 0x7f3f2c105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 --2- 172.21.15.5:0/1043354350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c100290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 --2- 172.21.15.5:0/1043354350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 -- 172.21.15.5:0/1043354350 >> 172.21.15.5:0/1043354350 conn(0x7f3f2c076f30 msgr2=0x7f3f2c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 -- 172.21.15.5:0/1043354350 shutdown_connections 2023-09-23T04:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.907+0000 7f3f325ff700 1 -- 172.21.15.5:0/1043354350 wait complete. 2023-09-23T04:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 Processor -- start 2023-09-23T04:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 -- start start 2023-09-23T04:49:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 0x7f3f2c1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c1a2590 con 0x7f3f2c1052b0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c1a2700 con 0x7f3f2c1007d0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.909+0000 7f3f325ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2c1a28c0 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42446/0 (socket says 172.21.15.5:42446) 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2920329994 learned_addr learned my addr 172.21.15.5:0/2920329994 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 0x7f3f2c1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2920329994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 msgr2=0x7f3f2c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 msgr2=0x7f3f2c1a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 0x7f3f2c1a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2920329994 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0095f0 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.910+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00b680 tx=0x7f3f1c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.911+0000 7f3f297fa700 1 -- 172.21.15.5:0/2920329994 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c01d070 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.911+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c1a2b40 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.911+0000 7f3f297fa700 1 -- 172.21.15.5:0/2920329994 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f1c021410 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.911+0000 7f3f297fa700 1 -- 172.21.15.5:0/2920329994 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c017650 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.911+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c1aa3e0 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.915+0000 7f3f297fa700 1 -- 172.21.15.5:0/2920329994 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3f1c021580 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.916+0000 7f3f45f0cb80 1 Processor -- start 2023-09-23T04:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.916+0000 7f3f297fa700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 0x7f3f140a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.916+0000 7f3f45f0cb80 1 -- start start 2023-09-23T04:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.916+0000 7f3f297fa700 1 -- 172.21.15.5:0/2920329994 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3f1c0ce570 con 0x7f3f2c0ffeb0 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 0x7f3f140a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80d20 0x55b2fca8b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc9e9460 con 0x55b2fca80d20 2023-09-23T04:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc9dac20 con 0x55b2fca80400 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc9d8540 con 0x55b2fca8bbd0 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45316/0 (socket says 172.21.15.5:45316) 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2465844607 learned_addr learned my addr 172.21.15.5:0/2465844607 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80d20 0x55b2fca8b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2465844607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 msgr2=0x55b2fca8dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca8dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2465844607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80d20 msgr2=0x55b2fca8b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80d20 0x55b2fca8b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.917+0000 7f3f2bfff700 1 -- 172.21.15.5:0/2465844607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2fca368d0 con 0x55b2fca80400 2023-09-23T04:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.918+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c097690 tx=0x7f3f1c09b090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b55747f21524fa4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.918+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 0x7f3f140a3bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400f870 tx=0x7f3f2400fc30 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.918+0000 7f3f127fc700 1 -- 172.21.15.5:0/2465844607 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c09b190 con 0x55b2fca80400 2023-09-23T04:49:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.918+0000 7f3f127fc700 1 -- 172.21.15.5:0/2465844607 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f1c09f2e0 con 0x55b2fca80400 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.918+0000 7f3f127fc700 1 -- 172.21.15.5:0/2465844607 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c09b650 con 0x55b2fca80400 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/2465844607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 msgr2=0x55b2fca807e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c097690 tx=0x7f3f1c09b090 comp rx=0 tx=0).stop 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/2465844607 shutdown_connections 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca8dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80400 0x55b2fca807e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/2465844607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80d20 0x55b2fca8b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/2465844607 >> 172.21.15.5:0/2465844607 conn(0x55b2fc9f5260 msgr2=0x55b2fca7c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/2465844607 shutdown_connections 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/2465844607 wait complete. 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.919+0000 7f3f45f0cb80 1 -- start start 2023-09-23T04:49:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 Processor -- start 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca97030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc9d3e30 con 0x55b2fca8bbd0 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc970710 con 0x55b2fca80d20 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc9703b0 con 0x55b2fca80400 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f45f0cb80 10 client.0 _ll_register_callbacks cb 0x55b2fc7cd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f30b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca97030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42462/0 (socket says 172.21.15.5:42462) 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f30b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca97030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41750/0 (socket says 172.21.15.5:41750) 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.920+0000 7f3f2bfff700 1 -- 172.21.15.5:0/1204275332 learned_addr learned my addr 172.21.15.5:0/1204275332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 -- 172.21.15.5:0/1204275332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 msgr2=0x55b2fca8c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 -- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 msgr2=0x55b2fca97030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca97030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 -- 172.21.15.5:0/1204275332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2c1aa3e0 con 0x55b2fca80400 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c006010 tx=0x7f3f1c09a930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9901e1c13113aeed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0974d0 con 0x55b2fca80400 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3f1c079e40 con 0x55b2fca80400 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.921+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2fca74250 con 0x55b2fca80400 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f1c0a3490 con 0x55b2fca80400 2023-09-23T04:49:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 10 client.26449 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b2fc7fede0 con 0x55b2fca80400 2023-09-23T04:49:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2fca74250 con 0x55b2fca80400 2023-09-23T04:49:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3f1c0a3120 con 0x55b2fca80400 2023-09-23T04:49:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 10 client.26449 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 10 client.26449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 20 client.26449 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fca9ae10 0x55b2fca9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 4 client.26449 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.922+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2fc9cce10 con 0x55b2fca9ae10 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.923+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fca9ae10 0x55b2fca9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.923+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fca9ae10 0x55b2fca9d200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a57e293965b62ff server_cookie=603a61d330f0eaf1 in_seq=0 out_seq=0 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.923+0000 7f3f0b7fe700 10 client.26449 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.924+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/1204275332 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b2fc9cce10 con 0x55b2fca9ae10 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.924+0000 7f3f0b7fe700 10 client.26449 handle_command_reply: tid=0 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.924+0000 7f3f45f0cb80 1 client.26449 shutdown 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.924+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 msgr2=0x55b2fca8bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.924+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c006010 tx=0x7f3f1c09a930 comp rx=0 tx=0).stop 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 shutdown_connections 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fca9ae10 0x55b2fca9d200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca97030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8bfb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f0b7fe700 0 client.26449 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/1204275332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 >> 172.21.15.5:0/1204275332 conn(0x55b2fc9f5260 msgr2=0x55b2fca7c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 shutdown_connections 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/1204275332 wait complete. 2023-09-23T04:49:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.925+0000 7f3f45f0cb80 20 client.26449 trim_cache size 0 max 16384 2023-09-23T04:49:27.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188237 10.95 MB/sec execute 3 sec latency 39.409 ms 2023-09-23T04:49:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.946+0000 7f3f45f0cb80 1 Processor -- start 2023-09-23T04:49:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.946+0000 7f3f45f0cb80 1 -- start start 2023-09-23T04:49:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 0x55b2fca852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca96720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fc854920 con 0x55b2fca80400 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fca8c860 con 0x55b2fca80d20 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fca9d830 con 0x55b2fca8bbd0 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 0x55b2fca852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45330/0 (socket says 172.21.15.5:45330) 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 0x55b2fca852b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41756/0 (socket says 172.21.15.5:41756) 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 -- 172.21.15.5:0/4156952956 learned_addr learned my addr 172.21.15.5:0/4156952956 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 -- 172.21.15.5:0/4156952956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 msgr2=0x55b2fca96720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca96720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 -- 172.21.15.5:0/4156952956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 msgr2=0x55b2fca852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 0x55b2fca852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.947+0000 7f3f2b7fe700 1 -- 172.21.15.5:0/4156952956 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2fca8b690 con 0x55b2fca80d20 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.948+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f31975ec4d218f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.948+0000 7f3f12ffd700 1 -- 172.21.15.5:0/4156952956 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20017030 con 0x55b2fca80d20 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.948+0000 7f3f12ffd700 1 -- 172.21.15.5:0/4156952956 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f20024070 con 0x55b2fca80d20 2023-09-23T04:49:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.948+0000 7f3f12ffd700 1 -- 172.21.15.5:0/4156952956 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200043d0 con 0x55b2fca80d20 2023-09-23T04:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/4156952956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 msgr2=0x55b2fca857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20009260 comp rx=0 tx=0).stop 2023-09-23T04:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/4156952956 shutdown_connections 2023-09-23T04:49:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca8bbd0 0x55b2fca96720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca80400 0x55b2fca852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/4156952956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca857f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/4156952956 >> 172.21.15.5:0/4156952956 conn(0x55b2fc9f5260 msgr2=0x55b2fca860c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/4156952956 shutdown_connections 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/4156952956 wait complete. 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.949+0000 7f3f45f0cb80 1 -- start start 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 Processor -- start 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca90070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fcaeebb0 con 0x55b2fca8bbd0 2023-09-23T04:49:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fcaeecf0 con 0x55b2fca80d20 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2fcaeee30 con 0x55b2fca80400 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f45f0cb80 10 client.0 _ll_register_callbacks cb 0x7f3f1c09ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.950+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42482/0 (socket says 172.21.15.5:42482) 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 -- 172.21.15.5:0/861484150 learned_addr learned my addr 172.21.15.5:0/861484150 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45342/0 (socket says 172.21.15.5:45342) 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca90070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 -- 172.21.15.5:0/861484150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 msgr2=0x55b2fca8fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 -- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 msgr2=0x55b2fca90070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca90070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 -- 172.21.15.5:0/861484150 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f20012170 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f30b9c700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca90070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f2bfff700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c097c00 tx=0x7f3f1c099740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e0374b031eba31 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.951+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0ce3f0 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3f1c079e40 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f1c09f2e0 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f1c0974d0 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 10 client.26455 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b2fc7e7c60 con 0x55b2fca80400 2023-09-23T04:49:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f1c09f2e0 con 0x55b2fca80400 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3f1c0a3120 con 0x55b2fca80400 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 10 client.26455 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 20 client.26455 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fcaf0090 0x55b2fcaf2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 4 client.26455 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.952+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2fc9ccb30 con 0x55b2fcaf0090 2023-09-23T04:49:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fcaf0090 0x55b2fcaf2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f2b7fe700 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fcaf0090 0x55b2fcaf2480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d798bed5fbd2197f server_cookie=8239636082736c84 in_seq=0 out_seq=0 2023-09-23T04:49:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f0b7fe700 10 client.26455 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f0b7fe700 1 -- 172.21.15.5:0/861484150 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b2fc9ccb30 con 0x55b2fcaf0090 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f0b7fe700 10 client.26455 handle_command_reply: tid=0 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.953+0000 7f3f45f0cb80 1 client.26455 shutdown 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 msgr2=0x55b2fca8f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c097c00 tx=0x7f3f1c099740 comp rx=0 tx=0).stop 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 shutdown_connections 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b2fcaf0090 0x55b2fcaf2480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2fca8bbd0 0x55b2fca90070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b2fca80400 0x55b2fca8f5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 --2- 172.21.15.5:0/861484150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b2fca80d20 0x55b2fca8fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 >> 172.21.15.5:0/861484150 conn(0x55b2fc9f5260 msgr2=0x55b2fca7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f0b7fe700 0 client.26455 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 shutdown_connections 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 1 -- 172.21.15.5:0/861484150 wait complete. 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.954+0000 7f3f45f0cb80 20 client.26455 trim_cache size 0 max 16384 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:49:27.959 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 msgr2=0x7f3f140a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 0x7f3f140a3bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400f870 tx=0x7f3f2400fc30 comp rx=0 tx=0).stop 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 msgr2=0x7f3f2c19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00b680 tx=0x7f3f1c009580 comp rx=0 tx=0).stop 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 shutdown_connections 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f140a1710 0x7f3f140a3bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f2c1052b0 0x7f3f2c1a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3f2c0ffeb0 0x7f3f2c19d5b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 --2- 172.21.15.5:0/2920329994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f2c1007d0 0x7f3f2c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 >> 172.21.15.5:0/2920329994 conn(0x7f3f2c076f30 msgr2=0x7f3f2c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 shutdown_connections 2023-09-23T04:49:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:27.955+0000 7f3f325ff700 1 -- 172.21.15.5:0/2920329994 wait complete. 2023-09-23T04:49:27.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:27 smithi005 ceph-mon[97638]: pgmap v98: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2023-09-23T04:49:27.978 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:49:27.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:49:27.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:27 smithi069 ceph-mon[133931]: pgmap v98: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2023-09-23T04:49:28.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:27 smithi118 ceph-mon[134297]: pgmap v98: 129 pgs: 129 active+clean; 108 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 317 op/s 2023-09-23T04:49:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 -- 172.21.15.5:0/859373538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 msgr2=0x7f2b5410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 --2- 172.21.15.5:0/859373538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b5410dd60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00b0a0 tx=0x7f2b4c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 -- 172.21.15.5:0/859373538 shutdown_connections 2023-09-23T04:49:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 --2- 172.21.15.5:0/859373538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b5410dd60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 --2- 172.21.15.5:0/859373538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b541010c0 0x7f2b541014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 --2- 172.21.15.5:0/859373538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b54101a70 0x7f2b54105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 -- 172.21.15.5:0/859373538 >> 172.21.15.5:0/859373538 conn(0x7f2b540fc960 msgr2=0x7f2b540fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 -- 172.21.15.5:0/859373538 shutdown_connections 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.218+0000 7f2b589ff700 1 -- 172.21.15.5:0/859373538 wait complete. 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 Processor -- start 2023-09-23T04:49:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 -- start start 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b541010c0 0x7f2b541970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 0x7f2b5419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419c4e0 con 0x7f2b5419ba60 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419c650 con 0x7f2b541010c0 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.220+0000 7f2b589ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5419c810 con 0x7f2b541061e0 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42508/0 (socket says 172.21.15.5:42508) 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b52d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 0x7f2b5419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 -- 172.21.15.5:0/3720076903 learned_addr learned my addr 172.21.15.5:0/3720076903 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b5259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b541010c0 0x7f2b541970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 -- 172.21.15.5:0/3720076903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b541010c0 msgr2=0x7f2b541970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b541010c0 0x7f2b541970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 -- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 msgr2=0x7f2b5419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 0x7f2b5419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b51d9b700 1 -- 172.21.15.5:0/3720076903 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009d00 con 0x7f2b541061e0 2023-09-23T04:49:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.221+0000 7f2b52d9d700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 0x7f2b5419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b51d9b700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e970 tx=0x7f2b4800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48009790 con 0x7f2b541061e0 2023-09-23T04:49:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b4800c7e0 con 0x7f2b541061e0 2023-09-23T04:49:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480054e0 con 0x7f2b541061e0 2023-09-23T04:49:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b541a3d70 con 0x7f2b541061e0 2023-09-23T04:49:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.222+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b541a41a0 con 0x7f2b541061e0 2023-09-23T04:49:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.223+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b54067ca0 con 0x7f2b541061e0 2023-09-23T04:49:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.225+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2b4800c950 con 0x7f2b541061e0 2023-09-23T04:49:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.228+0000 7f2b437fe700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 0x7f2b3c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.228+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2b48014070 con 0x7f2b541061e0 2023-09-23T04:49:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.228+0000 7f2b5259c700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 0x7f2b3c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.229+0000 7f2b5259c700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 0x7f2b3c0a3bd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009c60 tx=0x7f2b44009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.231+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f2b48096790 con 0x7f2b541061e0 2023-09-23T04:49:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.552+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b54101cc0 con 0x7f2b541061e0 2023-09-23T04:49:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.553+0000 7f2b437fe700 1 -- 172.21.15.5:0/3720076903 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f2b54101cc0 con 0x7f2b541061e0 2023-09-23T04:49:28.553 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:49:28.553 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:47:40.065282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:49:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 msgr2=0x7f2b3c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 0x7f2b3c0a3bd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009c60 tx=0x7f2b44009260 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 msgr2=0x7f2b54197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e970 tx=0x7f2b4800ec80 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 shutdown_connections 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b3c0a1710 0x7f2b3c0a3bd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b5419ba60 0x7f2b5419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b541010c0 0x7f2b541970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 --2- 172.21.15.5:0/3720076903 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b541061e0 0x7f2b54197600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.555+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 >> 172.21.15.5:0/3720076903 conn(0x7f2b540fc960 msgr2=0x7f2b540fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.556+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 shutdown_connections 2023-09-23T04:49:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.556+0000 7f2b589ff700 1 -- 172.21.15.5:0/3720076903 wait complete. 2023-09-23T04:49:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2023-09-23T04:49:28.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:49:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.831+0000 7f00221ff700 1 -- 172.21.15.5:0/568212475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 msgr2=0x7f001c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.831+0000 7f00221ff700 1 --2- 172.21.15.5:0/568212475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c1014d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f000c009990 tx=0x7f000c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 -- 172.21.15.5:0/568212475 shutdown_connections 2023-09-23T04:49:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 --2- 172.21.15.5:0/568212475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 --2- 172.21.15.5:0/568212475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c1014d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 --2- 172.21.15.5:0/568212475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c101aa0 0x7f001c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 -- 172.21.15.5:0/568212475 >> 172.21.15.5:0/568212475 conn(0x7f001c0fc9d0 msgr2=0x7f001c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 -- 172.21.15.5:0/568212475 shutdown_connections 2023-09-23T04:49:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.832+0000 7f00221ff700 1 -- 172.21.15.5:0/568212475 wait complete. 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 Processor -- start 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 -- start start 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a5840 con 0x7f001c106210 2023-09-23T04:49:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a59b0 con 0x7f001c073ba0 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.833+0000 7f00221ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a5b70 con 0x7f001c1010f0 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42518/0 (socket says 172.21.15.5:42518) 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 -- 172.21.15.5:0/1533261097 learned_addr learned my addr 172.21.15.5:0/1533261097 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001affd700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001bfff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 -- 172.21.15.5:0/1533261097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 msgr2=0x7f001c1a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 -- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 msgr2=0x7f001c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 -- 172.21.15.5:0/1533261097 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0095f0 con 0x7f001c1010f0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001affd700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001bfff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.834+0000 7f001b7fe700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f000c005910 tx=0x7f000c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.835+0000 7f0018ff9700 1 -- 172.21.15.5:0/1533261097 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c01d070 con 0x7f001c1010f0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.835+0000 7f0018ff9700 1 -- 172.21.15.5:0/1533261097 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c022470 con 0x7f001c1010f0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.835+0000 7f0018ff9700 1 -- 172.21.15.5:0/1533261097 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c017650 con 0x7f001c1010f0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.835+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1a5df0 con 0x7f001c1010f0 2023-09-23T04:49:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.835+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a61c0 con 0x7f001c1010f0 2023-09-23T04:49:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.838+0000 7f0018ff9700 1 -- 172.21.15.5:0/1533261097 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f000c0225e0 con 0x7f001c1010f0 2023-09-23T04:49:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0018ff9700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 0x7f00040a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 Processor -- start 2023-09-23T04:49:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0018ff9700 1 -- 172.21.15.5:0/1533261097 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f000c0ce820 con 0x7f001c1010f0 2023-09-23T04:49:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 -- start start 2023-09-23T04:49:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f001affd700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 0x7f00040a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3ea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f348460 con 0x564c2f3eabd0 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f339c20 con 0x564c2f3dfd20 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.840+0000 7f0035c1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f337540 con 0x564c2f3df400 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42528/0 (socket says 172.21.15.5:42528) 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 -- 172.21.15.5:0/2322585240 learned_addr learned my addr 172.21.15.5:0/2322585240 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 -- 172.21.15.5:0/2322585240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 msgr2=0x564c2f3ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 -- 172.21.15.5:0/2322585240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 msgr2=0x564c2f3ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 -- 172.21.15.5:0/2322585240 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2f3958d0 con 0x564c2f3df400 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001affd700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 0x7f00040a3c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f001c074910 tx=0x7f0010011cd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.841+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f000c0a3210 tx=0x7f000c09b210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8243dbf8616460d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.842+0000 7f0001ffb700 1 -- 172.21.15.5:0/2322585240 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c09ada0 con 0x564c2f3df400 2023-09-23T04:49:28.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.842+0000 7f0001ffb700 1 -- 172.21.15.5:0/2322585240 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c09bb40 con 0x564c2f3df400 2023-09-23T04:49:28.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.842+0000 7f0001ffb700 1 -- 172.21.15.5:0/2322585240 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0ce680 con 0x564c2f3df400 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.842+0000 7f0035c1db80 1 -- 172.21.15.5:0/2322585240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 msgr2=0x564c2f3df7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.842+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f000c0a3210 tx=0x7f000c09b210 comp rx=0 tx=0).stop 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 -- 172.21.15.5:0/2322585240 shutdown_connections 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3df7e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2322585240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 -- 172.21.15.5:0/2322585240 >> 172.21.15.5:0/2322585240 conn(0x564c2f354260 msgr2=0x564c2f3db790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 -- 172.21.15.5:0/2322585240 shutdown_connections 2023-09-23T04:49:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.843+0000 7f0035c1db80 1 -- 172.21.15.5:0/2322585240 wait complete. 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.844+0000 7f0035c1db80 1 -- start start 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.844+0000 7f0035c1db80 1 Processor -- start 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3dfd20 0x564c2f3eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f332e30 con 0x564c2f3dfd20 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f2cf710 con 0x564c2f3df400 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f2cf3b0 con 0x564c2f3eabd0 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f0035c1db80 10 client.0 _ll_register_callbacks cb 0x564c2f12c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45402/0 (socket says 172.21.15.5:45402) 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 -- 172.21.15.5:0/2805453998 learned_addr learned my addr 172.21.15.5:0/2805453998 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001bfff700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001affd700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3dfd20 0x564c2f3eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 -- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 msgr2=0x564c2f3f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 -- 172.21.15.5:0/2805453998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3dfd20 msgr2=0x564c2f3eb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3dfd20 0x564c2f3eb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.845+0000 7f001b7fe700 1 -- 172.21.15.5:0/2805453998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a61c0 con 0x564c2f3df400 2023-09-23T04:49:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.846+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b690 tx=0x7f000c09bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7464d3493d78b92 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.846+0000 7efffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.848+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0a3740 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.848+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f000c07c020 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.848+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2f3d3250 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.848+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f000c09ada0 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.848+0000 7f0035c1db80 10 client.26420 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564c2f15dde0 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c0a7020 con 0x564c2f3df400 2023-09-23T04:49:28.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f000c09b6a0 con 0x564c2f3df400 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 10 client.26420 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 10 client.26420 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 20 client.26420 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f3f9f80 0x564c2f3fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 4 client.26420 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2f32be10 con 0x564c2f3f9f80 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.849+0000 7f001affd700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f3f9f80 0x564c2f3fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.850+0000 7efffaffd700 10 client.26420 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.850+0000 7f001affd700 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f3f9f80 0x564c2f3fc3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d103f66572478a7 server_cookie=d783760f916d7c88 in_seq=0 out_seq=0 2023-09-23T04:49:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7efffaffd700 1 -- 172.21.15.5:0/2805453998 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c2f32be10 con 0x564c2f3f9f80 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7efffaffd700 10 client.26420 handle_command_reply: tid=0 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 client.26420 shutdown 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 msgr2=0x564c2f3eafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b690 tx=0x7f000c09bcb0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 shutdown_connections 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3df400 0x564c2f3eafb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f3f9f80 0x564c2f3fc3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2805453998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3dfd20 0x564c2f3eb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 >> 172.21.15.5:0/2805453998 conn(0x564c2f354260 msgr2=0x564c2f3db320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7efffaffd700 0 client.26420 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:49:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 shutdown_connections 2023-09-23T04:49:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.851+0000 7f0035c1db80 1 -- 172.21.15.5:0/2805453998 wait complete. 2023-09-23T04:49:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.852+0000 7f0035c1db80 20 client.26420 trim_cache size 0 max 16384 2023-09-23T04:49:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.869+0000 7f0035c1db80 1 Processor -- start 2023-09-23T04:49:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.869+0000 7f0035c1db80 1 -- start start 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3df400 0x564c2f3e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f1b3920 con 0x564c2f3df400 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f3e0ec0 con 0x564c2f3dfd20 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f0035c1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f3e9ca0 con 0x564c2f3eabd0 2023-09-23T04:49:28.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f001affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f001affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45404/0 (socket says 172.21.15.5:45404) 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f001bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f001affd700 1 -- 172.21.15.5:0/3855416343 learned_addr learned my addr 172.21.15.5:0/3855416343 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.870+0000 7f001b7fe700 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3df400 0x564c2f3e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 -- 172.21.15.5:0/3855416343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 msgr2=0x564c2f3f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 -- 172.21.15.5:0/3855416343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3df400 msgr2=0x564c2f3e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3df400 0x564c2f3e50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 -- 172.21.15.5:0/3855416343 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2f146c60 con 0x564c2f3dfd20 2023-09-23T04:49:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f001affd700 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0010005e90 tx=0x7f0010020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cd8963a6ba66e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f00027fc700 1 -- 172.21.15.5:0/3855416343 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010016040 con 0x564c2f3dfd20 2023-09-23T04:49:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f00027fc700 1 -- 172.21.15.5:0/3855416343 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0010004550 con 0x564c2f3dfd20 2023-09-23T04:49:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.871+0000 7f00027fc700 1 -- 172.21.15.5:0/3855416343 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000fd40 con 0x564c2f3dfd20 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 -- 172.21.15.5:0/3855416343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 msgr2=0x564c2f3e55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0010005e90 tx=0x7f0010020000 comp rx=0 tx=0).stop 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 -- 172.21.15.5:0/3855416343 shutdown_connections 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3eabd0 0x564c2f3f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3df400 0x564c2f3e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 --2- 172.21.15.5:0/3855416343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3e55f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 -- 172.21.15.5:0/3855416343 >> 172.21.15.5:0/3855416343 conn(0x564c2f354260 msgr2=0x564c2f3eafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 -- 172.21.15.5:0/3855416343 shutdown_connections 2023-09-23T04:49:28.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.872+0000 7f0035c1db80 1 -- 172.21.15.5:0/3855416343 wait complete. 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 -- start start 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 Processor -- start 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f3ee560 con 0x564c2f3eabd0 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f3ee6d0 con 0x564c2f3dfd20 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2f3ee890 con 0x564c2f3df400 2023-09-23T04:49:28.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.873+0000 7f0035c1db80 10 client.0 _ll_register_callbacks cb 0x7f0010009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42568/0 (socket says 172.21.15.5:42568) 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 -- 172.21.15.5:0/2180370520 learned_addr learned my addr 172.21.15.5:0/2180370520 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001affd700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001bfff700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 -- 172.21.15.5:0/2180370520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 msgr2=0x564c2f3efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 -- 172.21.15.5:0/2180370520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 msgr2=0x564c2f3edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 -- 172.21.15.5:0/2180370520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0010025040 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001affd700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f001b7fe700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f000c005f50 tx=0x7f000c09b210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1aa58b0c6a39db8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7efffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0a3740 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f000c07c070 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.874+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f000c09bb40 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f000c09ada0 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 10 client.26479 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564c2f184250 con 0x564c2f3df400 2023-09-23T04:49:28.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c09bb40 con 0x564c2f3df400 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f000c0a3400 con 0x564c2f3df400 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 10 client.26479 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 10 client.26479 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 20 client.26479 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f44ea70 0x564c2f450e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 4 client.26479 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c2f32bb30 con 0x564c2f44ea70 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.875+0000 7f001affd700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f44ea70 0x564c2f450e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7efffaffd700 10 client.26479 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:49:28.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7f001affd700 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f44ea70 0x564c2f450e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1263ac70d20e002e server_cookie=3d4436e81dd19ada in_seq=0 out_seq=0 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7efffaffd700 1 -- 172.21.15.5:0/2180370520 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c2f32bb30 con 0x564c2f44ea70 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7efffaffd700 10 client.26479 handle_command_reply: tid=0 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7f0035c1db80 1 client.26479 shutdown 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 msgr2=0x564c2f3ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.876+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f000c005f50 tx=0x7f000c09b210 comp rx=0 tx=0).stop 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 shutdown_connections 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564c2f44ea70 0x564c2f450e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c2f3eabd0 0x564c2f3edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7efffaffd700 0 client.26479 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c2f3df400 0x564c2f3ef9b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 --2- 172.21.15.5:0/2180370520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c2f3dfd20 0x564c2f3efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 >> 172.21.15.5:0/2180370520 conn(0x564c2f354260 msgr2=0x564c2f3e8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 shutdown_connections 2023-09-23T04:49:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 1 -- 172.21.15.5:0/2180370520 wait complete. 2023-09-23T04:49:28.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.877+0000 7f0035c1db80 20 client.26479 trim_cache size 0 max 16384 2023-09-23T04:49:28.883 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 msgr2=0x7f00040a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 0x7f00040a3c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f001c074910 tx=0x7f0010011cd0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 msgr2=0x7f001c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f000c005910 tx=0x7f000c004730 comp rx=0 tx=0).stop 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 shutdown_connections 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00040a1790 0x7f00040a3c50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1010f0 0x7f001c073120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 --2- 172.21.15.5:0/1533261097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c106210 0x7f001c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 >> 172.21.15.5:0/1533261097 conn(0x7f001c0fc9d0 msgr2=0x7f001c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 shutdown_connections 2023-09-23T04:49:28.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:28.878+0000 7f00221ff700 1 -- 172.21.15.5:0/1533261097 wait complete. 2023-09-23T04:49:28.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:28 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:49:28.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3720076903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:49:28.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:49:28.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 189064 8.77 MB/sec execute 4 sec latency 17.850 ms 2023-09-23T04:49:28.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:28 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:49:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3720076903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:49:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 -- 172.21.15.5:0/3710699107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 msgr2=0x7f664c069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 --2- 172.21.15.5:0/3710699107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c069480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f664800faf0 tx=0x7f664800fe00 comp rx=0 tx=0).stop 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 -- 172.21.15.5:0/3710699107 shutdown_connections 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 --2- 172.21.15.5:0/3710699107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c106210 0x7f664c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 --2- 172.21.15.5:0/3710699107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c069480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 --2- 172.21.15.5:0/3710699107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c069a50 0x7f664c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 -- 172.21.15.5:0/3710699107 >> 172.21.15.5:0/3710699107 conn(0x7f664c077ca0 msgr2=0x7f664c0780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 -- 172.21.15.5:0/3710699107 shutdown_connections 2023-09-23T04:49:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.147+0000 7f66539ff700 1 -- 172.21.15.5:0/3710699107 wait complete. 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.148+0000 7f66539ff700 1 Processor -- start 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 -- start start 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c106210 0x7f664c1952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 0x7f664c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c19a1c0 con 0x7f664c199740 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c19a330 con 0x7f664c106210 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f66539ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c19a4f0 con 0x7f664c0690a0 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f665179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f665179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42598/0 (socket says 172.21.15.5:42598) 2023-09-23T04:49:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f665179b700 1 -- 172.21.15.5:0/2724543648 learned_addr learned my addr 172.21.15.5:0/2724543648 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.149+0000 7f6650f9a700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c106210 0x7f664c1952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f6651f9c700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 0x7f664c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 -- 172.21.15.5:0/2724543648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c106210 msgr2=0x7f664c1952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c106210 0x7f664c1952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 -- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 msgr2=0x7f664c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 0x7f664c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 -- 172.21.15.5:0/2724543648 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664800f750 con 0x7f664c0690a0 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f6651f9c700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 0x7f664c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f665179b700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f664800f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f66427fc700 1 -- 172.21.15.5:0/2724543648 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648003f90 con 0x7f664c0690a0 2023-09-23T04:49:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.150+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c1a19f0 con 0x7f664c0690a0 2023-09-23T04:49:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.151+0000 7f66427fc700 1 -- 172.21.15.5:0/2724543648 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6648004cf0 con 0x7f664c0690a0 2023-09-23T04:49:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.151+0000 7f66427fc700 1 -- 172.21.15.5:0/2724543648 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648005690 con 0x7f664c0690a0 2023-09-23T04:49:29.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.151+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c1a1e10 con 0x7f664c0690a0 2023-09-23T04:49:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.154+0000 7f66427fc700 1 -- 172.21.15.5:0/2724543648 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f6648004180 con 0x7f664c0690a0 2023-09-23T04:49:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.156+0000 7f6667410b80 1 Processor -- start 2023-09-23T04:49:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 -- start start 2023-09-23T04:49:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f66427fc700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 0x7f66380a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f66427fc700 1 -- 172.21.15.5:0/2724543648 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f66480ce710 con 0x7f664c0690a0 2023-09-23T04:49:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014b1bd0 0x55b4014b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40140f460 con 0x55b4014a6400 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b401400c20 con 0x55b4014b1bd0 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6667410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4013fe540 con 0x55b4014a6d20 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6651f9c700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 0x7f66380a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42610/0 (socket says 172.21.15.5:42610) 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.157+0000 7f6650f9a700 1 -- 172.21.15.5:0/3614119263 learned_addr learned my addr 172.21.15.5:0/3614119263 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6651f9c700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014b1bd0 0x55b4014b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f665179b700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 -- 172.21.15.5:0/3614119263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014b1bd0 msgr2=0x55b4014b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014b1bd0 0x55b4014b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 -- 172.21.15.5:0/3614119263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 msgr2=0x55b4014a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 -- 172.21.15.5:0/3614119263 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40145c8d0 con 0x55b4014a6d20 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f665179b700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f663c004380 tx=0x7f663c004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=418d4172bb53cc6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f6651f9c700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 0x7f66380a3bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6644009550 tx=0x7f66440092f0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.158+0000 7f66377fe700 1 -- 172.21.15.5:0/3614119263 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c01f480 con 0x55b4014a6d20 2023-09-23T04:49:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.159+0000 7f66377fe700 1 -- 172.21.15.5:0/3614119263 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663c019040 con 0x55b4014a6d20 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.159+0000 7f66377fe700 1 -- 172.21.15.5:0/3614119263 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c009940 con 0x55b4014a6d20 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.159+0000 7f6667410b80 1 -- 172.21.15.5:0/3614119263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 msgr2=0x55b4014b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.159+0000 7f6667410b80 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f663c004380 tx=0x7f663c004690 comp rx=0 tx=0).stop 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 -- 172.21.15.5:0/3614119263 shutdown_connections 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014b1bd0 0x55b4014b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 --2- 172.21.15.5:0/3614119263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014b1690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 -- 172.21.15.5:0/3614119263 >> 172.21.15.5:0/3614119263 conn(0x55b40141b260 msgr2=0x55b4014a2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 -- 172.21.15.5:0/3614119263 shutdown_connections 2023-09-23T04:49:29.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 -- 172.21.15.5:0/3614119263 wait complete. 2023-09-23T04:49:29.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:28 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:49:29.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3720076903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:49:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.160+0000 7f6667410b80 1 -- start start 2023-09-23T04:49:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 Processor -- start 2023-09-23T04:49:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6400 0x55b4014b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4013f9e30 con 0x55b4014b1bd0 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b401396710 con 0x55b4014a6400 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4013963b0 con 0x55b4014a6d20 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6667410b80 10 client.0 _ll_register_callbacks cb 0x55b4011f3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.161+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42622/0 (socket says 172.21.15.5:42622) 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 -- 172.21.15.5:0/3418614694 learned_addr learned my addr 172.21.15.5:0/3418614694 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f665179b700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6400 0x55b4014b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6651f9c700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 -- 172.21.15.5:0/3418614694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6400 msgr2=0x55b4014b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6400 0x55b4014b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 -- 172.21.15.5:0/3418614694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 msgr2=0x55b4014aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6650f9a700 1 -- 172.21.15.5:0/3418614694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c004060 con 0x55b4014a6d20 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.162+0000 7f6651f9c700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6650f9a700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f663c005710 tx=0x7f663c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de2abddf9787f721 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6634ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c012c60 con 0x55b4014a6d20 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b40149a250 con 0x55b4014a6d20 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f663c012dc0 con 0x55b4014a6d20 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.163+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f663c004750 con 0x55b4014a6d20 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 10 client.26491 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b401224de0 con 0x55b4014a6d20 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b40149a250 con 0x55b4014a6d20 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f663c01fa30 con 0x55b4014a6d20 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 10 client.26491 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 10 client.26491 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 20 client.26491 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b4014c1620 0x55b4014c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 4 client.26491 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.164+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4013f2e10 con 0x55b4014c1620 2023-09-23T04:49:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.165+0000 7f665179b700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b4014c1620 0x55b4014c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.165+0000 7f665179b700 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b4014c1620 0x55b4014c3a40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a63cd765afd1eb8c server_cookie=25d70663f86daf05 in_seq=0 out_seq=0 2023-09-23T04:49:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.165+0000 7f6634ff9700 10 client.26491 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6634ff9700 1 -- 172.21.15.5:0/3418614694 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b4013f2e10 con 0x55b4014c1620 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6634ff9700 10 client.26491 handle_command_reply: tid=0 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 client.26491 shutdown 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 msgr2=0x55b4014ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f663c005710 tx=0x7f663c005740 comp rx=0 tx=0).stop 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 shutdown_connections 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b4014c1620 0x55b4014c3a40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6400 0x55b4014b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6634ff9700 0 client.26491 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:49:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.166+0000 7f6667410b80 1 --2- 172.21.15.5:0/3418614694 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6d20 0x55b4014ae8d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.167+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 >> 172.21.15.5:0/3418614694 conn(0x55b40141b260 msgr2=0x55b4014a2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.167+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 shutdown_connections 2023-09-23T04:49:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.167+0000 7f6667410b80 1 -- 172.21.15.5:0/3418614694 wait complete. 2023-09-23T04:49:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.167+0000 7f6667410b80 20 client.26491 trim_cache size 0 max 16384 2023-09-23T04:49:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 Processor -- start 2023-09-23T04:49:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 -- start start 2023-09-23T04:49:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 0x55b4014ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40127a920 con 0x55b4014a6400 2023-09-23T04:49:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4014a7100 con 0x55b4014a6d20 2023-09-23T04:49:29.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.183+0000 7f6667410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b40149ed60 con 0x55b4014b1bd0 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45452/0 (socket says 172.21.15.5:45452) 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 -- 172.21.15.5:0/115382064 learned_addr learned my addr 172.21.15.5:0/115382064 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f665179b700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6651f9c700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 0x55b4014ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 -- 172.21.15.5:0/115382064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 msgr2=0x55b4014ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 0x55b4014ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 -- 172.21.15.5:0/115382064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 msgr2=0x55b4014a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6650f9a700 1 -- 172.21.15.5:0/115382064 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b40120dc60 con 0x55b4014a6d20 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f6651f9c700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 0x55b4014ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.184+0000 7f665179b700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.185+0000 7f6650f9a700 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663c01fe20 tx=0x7f663c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272f1ca66f48271f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.185+0000 7f6637fff700 1 -- 172.21.15.5:0/115382064 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c013780 con 0x55b4014a6d20 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.185+0000 7f6637fff700 1 -- 172.21.15.5:0/115382064 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663c0138e0 con 0x55b4014a6d20 2023-09-23T04:49:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.185+0000 7f6637fff700 1 -- 172.21.15.5:0/115382064 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c01faf0 con 0x55b4014a6d20 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- 172.21.15.5:0/115382064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 msgr2=0x55b4014b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f663c01fe20 tx=0x7f663c009260 comp rx=0 tx=0).stop 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- 172.21.15.5:0/115382064 shutdown_connections 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014b1bd0 0x55b4014ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014a6400 0x55b4014a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 --2- 172.21.15.5:0/115382064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014b8710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- 172.21.15.5:0/115382064 >> 172.21.15.5:0/115382064 conn(0x55b40141b260 msgr2=0x55b4014a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- 172.21.15.5:0/115382064 shutdown_connections 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- 172.21.15.5:0/115382064 wait complete. 2023-09-23T04:49:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.186+0000 7f6667410b80 1 -- start start 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 Processor -- start 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4014ab2d0 con 0x55b4014b1bd0 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4014ab440 con 0x55b4014a6d20 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4014ab600 con 0x55b4014a6400 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.187+0000 7f6667410b80 10 client.0 _ll_register_callbacks cb 0x7f663c005370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42636/0 (socket says 172.21.15.5:42636) 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 -- 172.21.15.5:0/2385029736 learned_addr learned my addr 172.21.15.5:0/2385029736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f6650f9a700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f6651f9c700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 -- 172.21.15.5:0/2385029736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 msgr2=0x55b4014ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 -- 172.21.15.5:0/2385029736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 msgr2=0x55b4014aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 -- 172.21.15.5:0/2385029736 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c1a1e10 con 0x55b4014a6400 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f6650f9a700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f665179b700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f664809b370 tx=0x7f6648005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c889f4b4101c3639 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.188+0000 7f6634ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664809f530 con 0x55b4014a6400 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f66480a36e0 con 0x55b4014a6400 2023-09-23T04:49:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f663c019040 con 0x55b4014a6400 2023-09-23T04:49:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f664809f160 con 0x55b4014a6400 2023-09-23T04:49:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6667410b80 10 client.26497 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b401224de0 con 0x55b4014a6400 2023-09-23T04:49:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.189+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663c019040 con 0x55b4014a6400 2023-09-23T04:49:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f664809a8d0 con 0x55b4014a6400 2023-09-23T04:49:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 10 client.26497 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 10 client.26497 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:49:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 20 client.26497 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b401516400 0x55b4015187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 4 client.26497 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4013f2b30 con 0x55b401516400 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.190+0000 7f6650f9a700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b401516400 0x55b4015187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6650f9a700 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b401516400 0x55b4015187f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=787e3bdb6d216a7 server_cookie=1a5ad8ec1b44168d in_seq=0 out_seq=0 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6634ff9700 10 client.26497 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6634ff9700 1 -- 172.21.15.5:0/2385029736 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4013f2b30 con 0x55b401516400 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6634ff9700 10 client.26497 handle_command_reply: tid=0 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 client.26497 shutdown 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 msgr2=0x55b4014ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f664809b370 tx=0x7f6648005a90 comp rx=0 tx=0).stop 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 shutdown_connections 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55b401516400 0x55b4015187f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4014b1bd0 0x55b4014aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6634ff9700 0 client.26497 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b4014a6400 0x55b4014ac010 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 --2- 172.21.15.5:0/2385029736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b4014a6d20 0x55b4014ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 >> 172.21.15.5:0/2385029736 conn(0x55b40141b260 msgr2=0x55b4014a2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.191+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 shutdown_connections 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f6667410b80 1 -- 172.21.15.5:0/2385029736 wait complete. 2023-09-23T04:49:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f6667410b80 20 client.26497 trim_cache size 0 max 16384 2023-09-23T04:49:29.196 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:49:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 msgr2=0x7f66380a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 0x7f66380a3bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6644009550 tx=0x7f66440092f0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 msgr2=0x7f664c194da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6648000c00 tx=0x7f664800f6e0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 shutdown_connections 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f66380a1710 0x7f66380a3bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f664c199740 0x7f664c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.192+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f664c0690a0 0x7f664c194da0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.193+0000 7f66539ff700 1 --2- 172.21.15.5:0/2724543648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f664c106210 0x7f664c1952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.193+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 >> 172.21.15.5:0/2724543648 conn(0x7f664c077ca0 msgr2=0x7f664c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.193+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 shutdown_connections 2023-09-23T04:49:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.193+0000 7f66539ff700 1 -- 172.21.15.5:0/2724543648 wait complete. 2023-09-23T04:49:29.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:49:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 -- 172.21.15.5:0/2842830979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec1039e0 msgr2=0x7f62ec107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 --2- 172.21.15.5:0/2842830979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec1039e0 0x7f62ec107a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009990 tx=0x7f62dc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 -- 172.21.15.5:0/2842830979 shutdown_connections 2023-09-23T04:49:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 --2- 172.21.15.5:0/2842830979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec108150 0x7f62ec10fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 --2- 172.21.15.5:0/2842830979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 --2- 172.21.15.5:0/2842830979 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec1039e0 0x7f62ec107a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 -- 172.21.15.5:0/2842830979 >> 172.21.15.5:0/2842830979 conn(0x7f62ec0fe850 msgr2=0x7f62ec100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 -- 172.21.15.5:0/2842830979 shutdown_connections 2023-09-23T04:49:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.486+0000 7f62f219d700 1 -- 172.21.15.5:0/2842830979 wait complete. 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.487+0000 7f62f219d700 1 Processor -- start 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.487+0000 7f62f219d700 1 -- start start 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 0x7f62ec199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 0x7f62ec19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19e440 con 0x7f62ec103030 2023-09-23T04:49:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19e5b0 con 0x7f62ec19d9c0 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f219d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19e770 con 0x7f62ec108150 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f199c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 0x7f62ec19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41914/0 (socket says 172.21.15.5:41914) 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f099a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 0x7f62ec199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.488+0000 7f62f119b700 1 -- 172.21.15.5:0/3744420323 learned_addr learned my addr 172.21.15.5:0/3744420323 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 -- 172.21.15.5:0/3744420323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 msgr2=0x7f62ec199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 0x7f62ec199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 -- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 msgr2=0x7f62ec199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 -- 172.21.15.5:0/3744420323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8009520 con 0x7f62ec19d9c0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f119b700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f099a700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 0x7f62ec199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f199c700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 0x7f62ec19de40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62e400efc0 tx=0x7f62e400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62e27fc700 1 -- 172.21.15.5:0/3744420323 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018070 con 0x7f62ec19d9c0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62e27fc700 1 -- 172.21.15.5:0/3744420323 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e400f040 con 0x7f62ec19d9c0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc0095f0 con 0x7f62ec19d9c0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62e27fc700 1 -- 172.21.15.5:0/3744420323 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400cd40 con 0x7f62ec19d9c0 2023-09-23T04:49:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.489+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1a5f70 con 0x7f62ec19d9c0 2023-09-23T04:49:29.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.493+0000 7f62e27fc700 1 -- 172.21.15.5:0/3744420323 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f62e4003a00 con 0x7f62ec19d9c0 2023-09-23T04:49:29.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.494+0000 7f6306d55b80 1 Processor -- start 2023-09-23T04:49:29.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f62e27fc700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 0x7f62d80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 -- start start 2023-09-23T04:49:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f62e27fc700 1 -- 172.21.15.5:0/3744420323 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f62e40cd260 con 0x7f62ec19d9c0 2023-09-23T04:49:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f62f099a700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 0x7f62d80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ffe90 0x55e8d060a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 0x55e8d060d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0568460 con 0x55e8d05ff570 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0559c20 con 0x55e8d060ad40 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.495+0000 7f6306d55b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0557540 con 0x55e8d05ffe90 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f099a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ffe90 0x55e8d060a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41924/0 (socket says 172.21.15.5:41924) 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 0x55e8d060d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f119b700 1 -- 172.21.15.5:0/1934349984 learned_addr learned my addr 172.21.15.5:0/1934349984 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 -- 172.21.15.5:0/1934349984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ffe90 msgr2=0x55e8d060a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ffe90 0x55e8d060a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 -- 172.21.15.5:0/1934349984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 msgr2=0x55e8d05ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 -- 172.21.15.5:0/1934349984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d05b58d0 con 0x55e8d060ad40 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f119b700 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f099a700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 0x7f62d80a3bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62ec19a630 tx=0x7f62dc00bf00 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.496+0000 7f62f199c700 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 0x55e8d060d130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62e40ca570 tx=0x7f62e4096290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39ebe46274397ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f62d77fe700 1 -- 172.21.15.5:0/1934349984 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e401b320 con 0x55e8d060ad40 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f62d77fe700 1 -- 172.21.15.5:0/1934349984 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e4004ad0 con 0x55e8d060ad40 2023-09-23T04:49:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f62d77fe700 1 -- 172.21.15.5:0/1934349984 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40a21b0 con 0x55e8d060ad40 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 -- 172.21.15.5:0/1934349984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 msgr2=0x55e8d060d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 0x55e8d060d130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62e40ca570 tx=0x7f62e4096290 comp rx=0 tx=0).stop 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 -- 172.21.15.5:0/1934349984 shutdown_connections 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d060ad40 0x55e8d060d130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d05ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 --2- 172.21.15.5:0/1934349984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ffe90 0x55e8d060a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 -- 172.21.15.5:0/1934349984 >> 172.21.15.5:0/1934349984 conn(0x55e8d0574260 msgr2=0x55e8d05fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.497+0000 7f6306d55b80 1 -- 172.21.15.5:0/1934349984 shutdown_connections 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.498+0000 7f6306d55b80 1 -- 172.21.15.5:0/1934349984 wait complete. 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.498+0000 7f6306d55b80 1 -- start start 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.498+0000 7f6306d55b80 1 Processor -- start 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d0607b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d060ad40 0x55e8d0608610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0552e30 con 0x55e8d05ff570 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d04ef710 con 0x55e8d05ffe90 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d04ef3b0 con 0x55e8d060ad40 2023-09-23T04:49:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f6306d55b80 10 client.0 _ll_register_callbacks cb 0x55e8d034c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f62f119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d0607b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45502/0 (socket says 172.21.15.5:45502) 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f62f099a700 1 -- 172.21.15.5:0/734920771 learned_addr learned my addr 172.21.15.5:0/734920771 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.499+0000 7f62f199c700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d060ad40 0x55e8d0608610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 -- 172.21.15.5:0/734920771 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d060ad40 msgr2=0x55e8d0608610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d060ad40 0x55e8d0608610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 -- 172.21.15.5:0/734920771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 msgr2=0x55e8d0607b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d0607b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 -- 172.21.15.5:0/734920771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e80114a0 con 0x55e8d05ffe90 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62f099a700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f62dc015000 tx=0x7f62dc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c433127edc3f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004d70 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8d05f34a0 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f62dc00f460 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62dc00f620 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f6306d55b80 10 client.26456 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.500+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f62ec1a5f70 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8d05f34a0 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f62dc018840 con 0x55e8d05ffe90 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 10 client.26456 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:49:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 20 client.26456 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d061a770 0x55e8d061cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 4 client.26456 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8d054be10 con 0x55e8d061a770 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.501+0000 7f62f119b700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d061a770 0x55e8d061cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.502+0000 7f62f119b700 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d061a770 0x55e8d061cb90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38e1d74d027560e7 server_cookie=c27882bae1be9160 in_seq=0 out_seq=0 2023-09-23T04:49:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.502+0000 7f62d4ff9700 10 client.26456 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:49:29.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.502+0000 7f62d4ff9700 1 -- 172.21.15.5:0/734920771 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8d054be10 con 0x55e8d061a770 2023-09-23T04:49:29.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f62d4ff9700 10 client.26456 handle_command_reply: tid=0 2023-09-23T04:49:29.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 client.26456 shutdown 2023-09-23T04:49:29.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 msgr2=0x55e8d06080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f62dc015000 tx=0x7f62dc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 shutdown_connections 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d061a770 0x55e8d061cb90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d060ad40 0x55e8d0608610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d05ff570 0x55e8d0607b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 --2- 172.21.15.5:0/734920771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d06080d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f62d4ff9700 0 client.26456 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 >> 172.21.15.5:0/734920771 conn(0x55e8d0574260 msgr2=0x55e8d05fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 shutdown_connections 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 1 -- 172.21.15.5:0/734920771 wait complete. 2023-09-23T04:49:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.503+0000 7f6306d55b80 20 client.26456 trim_cache size 0 max 16384 2023-09-23T04:49:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.520+0000 7f6306d55b80 1 Processor -- start 2023-09-23T04:49:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.520+0000 7f6306d55b80 1 -- start start 2023-09-23T04:49:29.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d06118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0602ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d03d3920 con 0x55e8d060ad40 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0600880 con 0x55e8d05ffe90 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f6306d55b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0617310 con 0x55e8d05ff570 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0602ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d06118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0602ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45508/0 (socket says 172.21.15.5:45508) 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f099a700 1 -- 172.21.15.5:0/817767619 learned_addr learned my addr 172.21.15.5:0/817767619 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f199c700 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 -- 172.21.15.5:0/817767619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 msgr2=0x55e8d0602ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0602ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 -- 172.21.15.5:0/817767619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 msgr2=0x55e8d06033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f119b700 1 -- 172.21.15.5:0/817767619 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d037dde0 con 0x55e8d05ff570 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.521+0000 7f62f199c700 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.522+0000 7f62f119b700 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d06118f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55e8d061d440 tx=0x7f62e8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21113d39617a911 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.522+0000 7f62d7fff700 1 -- 172.21.15.5:0/817767619 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8016040 con 0x55e8d05ff570 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.522+0000 7f62d7fff700 1 -- 172.21.15.5:0/817767619 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e8017040 con 0x55e8d05ff570 2023-09-23T04:49:29.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.522+0000 7f62d7fff700 1 -- 172.21.15.5:0/817767619 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8005310 con 0x55e8d05ff570 2023-09-23T04:49:29.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- 172.21.15.5:0/817767619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 msgr2=0x55e8d06118f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d06118f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55e8d061d440 tx=0x7f62e8009260 comp rx=0 tx=0).stop 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- 172.21.15.5:0/817767619 shutdown_connections 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d06118f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 --2- 172.21.15.5:0/817767619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0602ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- 172.21.15.5:0/817767619 >> 172.21.15.5:0/817767619 conn(0x55e8d0574260 msgr2=0x55e8d0608350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- 172.21.15.5:0/817767619 shutdown_connections 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- 172.21.15.5:0/817767619 wait complete. 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.523+0000 7f6306d55b80 1 -- start start 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 Processor -- start 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d0662720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0666e60 con 0x55e8d060ad40 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0666fd0 con 0x55e8d05ffe90 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.524+0000 7f6306d55b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d0667190 con 0x55e8d05ff570 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f6306d55b80 10 client.0 _ll_register_callbacks cb 0x7f62dc00bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f199c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f119b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d0662720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45516/0 (socket says 172.21.15.5:45516) 2023-09-23T04:49:29.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 -- 172.21.15.5:0/4226987017 learned_addr learned my addr 172.21.15.5:0/4226987017 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 -- 172.21.15.5:0/4226987017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 msgr2=0x55e8d0662720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d0662720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 -- 172.21.15.5:0/4226987017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 msgr2=0x55e8d06668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 -- 172.21.15.5:0/4226987017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e8020a00 con 0x55e8d05ffe90 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62f099a700 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f62dc018c00 tx=0x7f62dc0098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79a33f9fb2fc461 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.525+0000 7f62d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004d70 con 0x55e8d05ffe90 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62e4004ad0 con 0x55e8d05ffe90 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f62dc018400 con 0x55e8d05ffe90 2023-09-23T04:49:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62dc0185c0 con 0x55e8d05ffe90 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 10 client.26462 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e8d0366c60 con 0x55e8d05ffe90 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e4004ad0 con 0x55e8d05ffe90 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f62dc038050 con 0x55e8d05ffe90 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 10 client.26462 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 10 client.26462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.526+0000 7f6306d55b80 20 client.26462 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d066f690 0x55e8d0671ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f6306d55b80 4 client.26462 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8d054bb30 con 0x55e8d066f690 2023-09-23T04:49:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f62f119b700 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d066f690 0x55e8d0671ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f62f119b700 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d066f690 0x55e8d0671ab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94c66d7e0b658492 server_cookie=b3b8cc4903f72da4 in_seq=0 out_seq=0 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.527+0000 7f62d4ff9700 10 client.26462 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f62d4ff9700 1 -- 172.21.15.5:0/4226987017 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8d054bb30 con 0x55e8d066f690 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f62d4ff9700 10 client.26462 handle_command_reply: tid=0 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 client.26462 shutdown 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 msgr2=0x55e8d0662cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f62dc018c00 tx=0x7f62dc0098e0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 shutdown_connections 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e8d066f690 0x55e8d0671ab0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8d060ad40 0x55e8d06668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e8d05ff570 0x55e8d0662720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f62d4ff9700 0 client.26462 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 --2- 172.21.15.5:0/4226987017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8d05ffe90 0x55e8d0662cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 >> 172.21.15.5:0/4226987017 conn(0x55e8d0574260 msgr2=0x55e8d0602ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 shutdown_connections 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.528+0000 7f6306d55b80 1 -- 172.21.15.5:0/4226987017 wait complete. 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f6306d55b80 20 client.26462 trim_cache size 0 max 16384 2023-09-23T04:49:29.532 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 msgr2=0x7f62d80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 0x7f62d80a3bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62ec19a630 tx=0x7f62dc00bf00 comp rx=0 tx=0).stop 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 msgr2=0x7f62ec19de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 0x7f62ec19de40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62e400efc0 tx=0x7f62e400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 shutdown_connections 2023-09-23T04:49:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f62d80a1710 0x7f62d80a3bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62ec19d9c0 0x7f62ec19de40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62ec103030 0x7f62ec199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 --2- 172.21.15.5:0/3744420323 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f62ec108150 0x7f62ec199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.529+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 >> 172.21.15.5:0/3744420323 conn(0x7f62ec0fe850 msgr2=0x7f62ec0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.530+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 shutdown_connections 2023-09-23T04:49:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.530+0000 7f62f219d700 1 -- 172.21.15.5:0/3744420323 wait complete. 2023-09-23T04:49:29.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:49:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 -- 172.21.15.5:0/3041016414 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2788101a50 msgr2=0x7f2788105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 --2- 172.21.15.5:0/3041016414 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2788101a50 0x7f2788105a90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f277c009990 tx=0x7f277c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 -- 172.21.15.5:0/3041016414 shutdown_connections 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 --2- 172.21.15.5:0/3041016414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881061c0 0x7f278810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 --2- 172.21.15.5:0/3041016414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27881010a0 0x7f2788101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 --2- 172.21.15.5:0/3041016414 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2788101a50 0x7f2788105a90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 -- 172.21.15.5:0/3041016414 >> 172.21.15.5:0/3041016414 conn(0x7f27880fc920 msgr2=0x7f27880fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 -- 172.21.15.5:0/3041016414 shutdown_connections 2023-09-23T04:49:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.779+0000 7f278e7ff700 1 -- 172.21.15.5:0/3041016414 wait complete. 2023-09-23T04:49:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 Processor -- start 2023-09-23T04:49:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 -- start start 2023-09-23T04:49:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 0x7f2788073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 0x7f27881a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27881a5830 con 0x7f27881010a0 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27881a59a0 con 0x7f2788073ba0 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f278e7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27881a5b60 con 0x7f27881061c0 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f2787fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f2787fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41980/0 (socket says 172.21.15.5:41980) 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.781+0000 7f2787fff700 1 -- 172.21.15.5:0/1551299342 learned_addr learned my addr 172.21.15.5:0/1551299342 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f27877fe700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 0x7f2788073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f278cd9c700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 0x7f27881a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 -- 172.21.15.5:0/1551299342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 msgr2=0x7f2788073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 0x7f2788073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 -- 172.21.15.5:0/1551299342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 msgr2=0x7f27881a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 0x7f27881a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 -- 172.21.15.5:0/1551299342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0095f0 con 0x7f27881010a0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f278cd9c700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 0x7f27881a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f27877fe700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 0x7f2788073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.782+0000 7f2787fff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f277800e9a0 tx=0x7f277800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.783+0000 7f27857fa700 1 -- 172.21.15.5:0/1551299342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778009790 con 0x7f27881010a0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.783+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27881a5e40 con 0x7f27881010a0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.783+0000 7f27857fa700 1 -- 172.21.15.5:0/1551299342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2778010430 con 0x7f27881010a0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.783+0000 7f27857fa700 1 -- 172.21.15.5:0/1551299342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778018680 con 0x7f27881010a0 2023-09-23T04:49:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.783+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27881a62c0 con 0x7f27881010a0 2023-09-23T04:49:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.786+0000 7f27857fa700 1 -- 172.21.15.5:0/1551299342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f27780105a0 con 0x7f27881010a0 2023-09-23T04:49:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 Processor -- start 2023-09-23T04:49:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 -- start start 2023-09-23T04:49:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27857fa700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 0x7f27700a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845bd20 0x55bcb8466690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb8466bd0 0x55bcb8468fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb83c4460 con 0x55bcb845bd20 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb83b5c20 con 0x55bcb8466bd0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27a21eab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb83b3540 con 0x55bcb845b400 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f27857fa700 1 -- 172.21.15.5:0/1551299342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2778014070 con 0x7f27881010a0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.788+0000 7f278cd9c700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 0x7f27700a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f27877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845bd20 0x55bcb8466690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42708/0 (socket says 172.21.15.5:42708) 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 -- 172.21.15.5:0/3980318164 learned_addr learned my addr 172.21.15.5:0/3980318164 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f278cd9c700 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb8466bd0 0x55bcb8468fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 -- 172.21.15.5:0/3980318164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb8466bd0 msgr2=0x55bcb8468fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb8466bd0 0x55bcb8468fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 -- 172.21.15.5:0/3980318164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845bd20 msgr2=0x55bcb8466690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845bd20 0x55bcb8466690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 -- 172.21.15.5:0/3980318164 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcb84118d0 con 0x55bcb845b400 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f2787fff700 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f27780a2270 tx=0x7f2778099f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f40561994c30d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f278cd9c700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 0x7f27700a3bd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2780009ec0 tx=0x7f2780009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.789+0000 7f276e7fc700 1 -- 172.21.15.5:0/3980318164 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778017a90 con 0x55bcb845b400 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f276e7fc700 1 -- 172.21.15.5:0/3980318164 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277809ab10 con 0x55bcb845b400 2023-09-23T04:49:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f276e7fc700 1 -- 172.21.15.5:0/3980318164 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780987b0 con 0x55bcb845b400 2023-09-23T04:49:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 -- 172.21.15.5:0/3980318164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 msgr2=0x55bcb845b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f27780a2270 tx=0x7f2778099f50 comp rx=0 tx=0).stop 2023-09-23T04:49:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 -- 172.21.15.5:0/3980318164 shutdown_connections 2023-09-23T04:49:29.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb8466bd0 0x55bcb8468fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845b400 0x55bcb845b7e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3980318164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845bd20 0x55bcb8466690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.790+0000 7f27a21eab80 1 -- 172.21.15.5:0/3980318164 >> 172.21.15.5:0/3980318164 conn(0x55bcb83d0260 msgr2=0x55bcb8457790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.791+0000 7f27a21eab80 1 -- 172.21.15.5:0/3980318164 shutdown_connections 2023-09-23T04:49:29.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.791+0000 7f27a21eab80 1 -- 172.21.15.5:0/3980318164 wait complete. 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.791+0000 7f27a21eab80 1 -- start start 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.791+0000 7f27a21eab80 1 Processor -- start 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb8463960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb8463ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb83aee30 con 0x55bcb8466bd0 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb834b710 con 0x55bcb845b400 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb834b3b0 con 0x55bcb845bd20 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27a21eab80 10 client.0 _ll_register_callbacks cb 0x55bcb81a8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27877fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb8463ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27877fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb8463ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42710/0 (socket says 172.21.15.5:42710) 2023-09-23T04:49:29.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f27877fe700 1 -- 172.21.15.5:0/888807573 learned_addr learned my addr 172.21.15.5:0/888807573 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f2787fff700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb8463960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f278cd9c700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.792+0000 7f2787fff700 1 -- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 msgr2=0x55bcb8463ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f2787fff700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb8463ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f2787fff700 1 -- 172.21.15.5:0/888807573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 msgr2=0x55bcb84643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f2787fff700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f2787fff700 1 -- 172.21.15.5:0/888807573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27881a62c0 con 0x55bcb845b400 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f278cd9c700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f2787fff700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb8463960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2778098760 tx=0x7f27780205f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84bee7350f814807 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27677fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780a27a0 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcb844f250 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2778010880 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2778099900 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27a21eab80 10 client.26474 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.793+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bcb81d9de0 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcb844f250 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f27780a2310 con 0x55bcb845b400 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 10 client.26474 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 10 client.26474 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 20 client.26474 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb8476320 0x55bcb8478710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 4 client.26474 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:29.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcb83a7e10 con 0x55bcb8476320 2023-09-23T04:49:29.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.794+0000 7f27877fe700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb8476320 0x55bcb8478710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.795+0000 7f27877fe700 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb8476320 0x55bcb8478710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4671d3269c20e56 server_cookie=10188cb137179053 in_seq=0 out_seq=0 2023-09-23T04:49:29.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.795+0000 7f27677fe700 10 client.26474 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.795+0000 7f27677fe700 1 -- 172.21.15.5:0/888807573 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bcb83a7e10 con 0x55bcb8476320 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.795+0000 7f27677fe700 10 client.26474 handle_command_reply: tid=0 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 client.26474 shutdown 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 msgr2=0x55bcb8463960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb8463960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2778098760 tx=0x7f27780205f0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 shutdown_connections 2023-09-23T04:49:29.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb8476320 0x55bcb8478710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb8463960 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27677fe700 0 client.26474 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 --2- 172.21.15.5:0/888807573 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb8463ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 >> 172.21.15.5:0/888807573 conn(0x55bcb83d0260 msgr2=0x55bcb8457320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 shutdown_connections 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 1 -- 172.21.15.5:0/888807573 wait complete. 2023-09-23T04:49:29.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.796+0000 7f27a21eab80 20 client.26474 trim_cache size 0 max 16384 2023-09-23T04:49:29.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 Processor -- start 2023-09-23T04:49:29.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 -- start start 2023-09-23T04:49:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 0x55bcb8464210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb8466bd0 0x55bcb8462610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb822f920 con 0x55bcb845b400 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb8472f90 con 0x55bcb845bd20 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f27a21eab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb845f560 con 0x55bcb8466bd0 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f2787fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.811+0000 7f2787fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42006/0 (socket says 172.21.15.5:42006) 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 -- 172.21.15.5:0/2645636905 learned_addr learned my addr 172.21.15.5:0/2645636905 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f27877fe700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 0x55bcb8464210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f278cd9c700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb8466bd0 0x55bcb8462610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 -- 172.21.15.5:0/2645636905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb8466bd0 msgr2=0x55bcb8462610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb8466bd0 0x55bcb8462610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 -- 172.21.15.5:0/2645636905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 msgr2=0x55bcb8464210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 0x55bcb8464210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 -- 172.21.15.5:0/2645636905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcb81c2c60 con 0x55bcb845b400 2023-09-23T04:49:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f27877fe700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 0x55bcb8464210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f2787fff700 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f277801e000 tx=0x7f27780989c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6e9f8813a0cc07 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.812+0000 7f276effd700 1 -- 172.21.15.5:0/2645636905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778099f50 con 0x55bcb845b400 2023-09-23T04:49:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.813+0000 7f276effd700 1 -- 172.21.15.5:0/2645636905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27780a27a0 con 0x55bcb845b400 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.813+0000 7f276effd700 1 -- 172.21.15.5:0/2645636905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780cd450 con 0x55bcb845b400 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.813+0000 7f27a21eab80 1 -- 172.21.15.5:0/2645636905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 msgr2=0x55bcb845e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.813+0000 7f27a21eab80 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f277801e000 tx=0x7f27780989c0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 -- 172.21.15.5:0/2645636905 shutdown_connections 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb8466bd0 0x55bcb8462610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb845b400 0x55bcb845e370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 --2- 172.21.15.5:0/2645636905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845bd20 0x55bcb8464210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 -- 172.21.15.5:0/2645636905 >> 172.21.15.5:0/2645636905 conn(0x55bcb83d0260 msgr2=0x55bcb8461f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 -- 172.21.15.5:0/2645636905 shutdown_connections 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 -- 172.21.15.5:0/2645636905 wait complete. 2023-09-23T04:49:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.814+0000 7f27a21eab80 1 -- start start 2023-09-23T04:49:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 Processor -- start 2023-09-23T04:49:29.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb846b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb84c9620 con 0x55bcb8466bd0 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb84c9760 con 0x55bcb845b400 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcb84c98f0 con 0x55bcb845bd20 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f27a21eab80 10 client.0 _ll_register_callbacks cb 0x7f27800097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f2787fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.815+0000 7f2787fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45574/0 (socket says 172.21.15.5:45574) 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 -- 172.21.15.5:0/3654881813 learned_addr learned my addr 172.21.15.5:0/3654881813 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f278cd9c700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27877fe700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb846b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 -- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 msgr2=0x55bcb846b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb846b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 -- 172.21.15.5:0/3654881813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 msgr2=0x55bcb84c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 -- 172.21.15.5:0/3654881813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278810e5b0 con 0x55bcb845b400 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27877fe700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb846b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f278cd9c700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f2787fff700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f277800c2d0 tx=0x7f2778010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e54a7a9365e8b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27677fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778099f50 con 0x55bcb845b400 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f277809ab10 con 0x55bcb845b400 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.816+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f27780cd650 con 0x55bcb845b400 2023-09-23T04:49:29.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f277809e560 con 0x55bcb845b400 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 10 client.26480 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bcb81d9de0 con 0x55bcb845b400 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277809ab10 con 0x55bcb845b400 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f27780995a0 con 0x55bcb845b400 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 10 client.26480 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 10 client.26480 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 20 client.26480 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb84cab20 0x55bcb84ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 4 client.26480 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcb83a7b30 con 0x55bcb84cab20 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.817+0000 7f27877fe700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb84cab20 0x55bcb84ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27877fe700 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb84cab20 0x55bcb84ccf10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c36732664aa4acf server_cookie=d26286c93b777b0a in_seq=0 out_seq=0 2023-09-23T04:49:29.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27677fe700 10 client.26480 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27677fe700 1 -- 172.21.15.5:0/3654881813 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcb83a7b30 con 0x55bcb84cab20 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27677fe700 10 client.26480 handle_command_reply: tid=0 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 client.26480 shutdown 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 msgr2=0x55bcb846b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f277800c2d0 tx=0x7f2778010ab0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 shutdown_connections 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bcb8466bd0 0x55bcb84c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55bcb84cab20 0x55bcb84ccf10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcb845b400 0x55bcb846b4b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 --2- 172.21.15.5:0/3654881813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bcb845bd20 0x55bcb846b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27677fe700 0 client.26480 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:49:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.818+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 >> 172.21.15.5:0/3654881813 conn(0x55bcb83d0260 msgr2=0x55bcb8463f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 shutdown_connections 2023-09-23T04:49:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f27a21eab80 1 -- 172.21.15.5:0/3654881813 wait complete. 2023-09-23T04:49:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f27a21eab80 20 client.26480 trim_cache size 0 max 16384 2023-09-23T04:49:29.825 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 msgr2=0x7f27700a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 0x7f27700a3bd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2780009ec0 tx=0x7f2780009920 comp rx=0 tx=0).stop 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 msgr2=0x7f2788073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.819+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f277800e9a0 tx=0x7f277800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 shutdown_connections 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f27700a1710 0x7f27700a3bd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2788073ba0 0x7f27881a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27881010a0 0x7f2788073120 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 --2- 172.21.15.5:0/1551299342 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f27881061c0 0x7f2788073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 >> 172.21.15.5:0/1551299342 conn(0x7f27880fc920 msgr2=0x7f27880fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 shutdown_connections 2023-09-23T04:49:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:29.820+0000 7f278e7ff700 1 -- 172.21.15.5:0/1551299342 wait complete. 2023-09-23T04:49:29.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:29 smithi005 ceph-mon[97638]: pgmap v99: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 412 op/s 2023-09-23T04:49:29.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:49:29.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:49:29.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:49:29.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:49:29.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191331 9.51 MB/sec execute 5 sec latency 19.407 ms 2023-09-23T04:49:29.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:29 smithi069 ceph-mon[133931]: pgmap v99: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 412 op/s 2023-09-23T04:49:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:49:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:49:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 -- 172.21.15.5:0/2886868360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c0103cc0 msgr2=0x7f26c0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 --2- 172.21.15.5:0/2886868360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c0103cc0 0x7f26c0107d00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009990 tx=0x7f26b4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 -- 172.21.15.5:0/2886868360 shutdown_connections 2023-09-23T04:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 --2- 172.21.15.5:0/2886868360 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 --2- 172.21.15.5:0/2886868360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 0x7f26c01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 --2- 172.21.15.5:0/2886868360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c0103cc0 0x7f26c0107d00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 -- 172.21.15.5:0/2886868360 >> 172.21.15.5:0/2886868360 conn(0x7f26c00feb50 msgr2=0x7f26c0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 -- 172.21.15.5:0/2886868360 shutdown_connections 2023-09-23T04:49:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.073+0000 7f26c63ff700 1 -- 172.21.15.5:0/2886868360 wait complete. 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 Processor -- start 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 -- start start 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 0x7f26c01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c019db50 0x7f26c019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c019e5d0 con 0x7f26c0103310 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c019e740 con 0x7f26c019db50 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.074+0000 7f26c63ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c019e900 con 0x7f26c0108430 2023-09-23T04:49:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42748/0 (socket says 172.21.15.5:42748) 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 -- 172.21.15.5:0/1855995237 learned_addr learned my addr 172.21.15.5:0/1855995237 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bffff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 0x7f26c01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26c499c700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c019db50 0x7f26c019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 -- 172.21.15.5:0/1855995237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c019db50 msgr2=0x7f26c019dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c019db50 0x7f26c019dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 -- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 msgr2=0x7f26c01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 0x7f26c01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.075+0000 7f26bf7fe700 1 -- 172.21.15.5:0/1855995237 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40095f0 con 0x7f26c0108430 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26bd7fa700 1 -- 172.21.15.5:0/1855995237 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b401d070 con 0x7f26c0108430 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c01a5e00 con 0x7f26c0108430 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26bd7fa700 1 -- 172.21.15.5:0/1855995237 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b4022470 con 0x7f26c0108430 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26bd7fa700 1 -- 172.21.15.5:0/1855995237 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4017630 con 0x7f26c0108430 2023-09-23T04:49:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.076+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c01a6220 con 0x7f26c0108430 2023-09-23T04:49:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.080+0000 7f26bd7fa700 1 -- 172.21.15.5:0/1855995237 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f26b400fad0 con 0x7f26c0108430 2023-09-23T04:49:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.081+0000 7f26d9c49b80 1 Processor -- start 2023-09-23T04:49:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.081+0000 7f26d9c49b80 1 -- start start 2023-09-23T04:49:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.081+0000 7f26bd7fa700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 0x7f26a80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26bd7fa700 1 -- 172.21.15.5:0/1855995237 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f26b40cea60 con 0x7f26c0108430 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4c400 0x5596d3d4c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 0x5596d3d59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3cb5460 con 0x5596d3d4c400 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3ca6c20 con 0x5596d3d57bd0 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3ca4540 con 0x5596d3d4cd20 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26c499c700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 0x7f26a80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42754/0 (socket says 172.21.15.5:42754) 2023-09-23T04:49:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4c400 0x5596d3d4c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26bf7fe700 1 -- 172.21.15.5:0/2467385451 learned_addr learned my addr 172.21.15.5:0/2467385451 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.082+0000 7f26c499c700 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 0x5596d3d59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 -- 172.21.15.5:0/2467385451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 msgr2=0x5596d3d59fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 0x5596d3d59fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 -- 172.21.15.5:0/2467385451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4c400 msgr2=0x5596d3d4c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4c400 0x5596d3d4c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 -- 172.21.15.5:0/2467385451 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596d3d028d0 con 0x5596d3d4cd20 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26c499c700 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 0x5596d3d59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26c499c700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 0x7f26a80a3bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26b800f540 tx=0x7f26b8009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26b4098090 tx=0x7f26b4022ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84d2a3c22eea9c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26a67fc700 1 -- 172.21.15.5:0/2467385451 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b409af80 con 0x5596d3d4cd20 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26a67fc700 1 -- 172.21.15.5:0/2467385451 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b40860c0 con 0x5596d3d4cd20 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.083+0000 7f26a67fc700 1 -- 172.21.15.5:0/2467385451 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40999c0 con 0x5596d3d4cd20 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 -- 172.21.15.5:0/2467385451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 msgr2=0x5596d3d57690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f26b4098090 tx=0x7f26b4022ac0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 -- 172.21.15.5:0/2467385451 shutdown_connections 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d57bd0 0x5596d3d59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4c400 0x5596d3d4c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/2467385451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4cd20 0x5596d3d57690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 -- 172.21.15.5:0/2467385451 >> 172.21.15.5:0/2467385451 conn(0x5596d3cc1260 msgr2=0x5596d3d48790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 -- 172.21.15.5:0/2467385451 shutdown_connections 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.084+0000 7f26d9c49b80 1 -- 172.21.15.5:0/2467385451 wait complete. 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.085+0000 7f26d9c49b80 1 -- start start 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.085+0000 7f26d9c49b80 1 Processor -- start 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 0x5596d3d57fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 0x5596d3d63030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3c9fe30 con 0x5596d3d57bd0 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3c3c710 con 0x5596d3d4cd20 2023-09-23T04:49:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3c3c3b0 con 0x5596d3d4c400 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26d9c49b80 10 client.0 _ll_register_callbacks cb 0x5596d3a99800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 0x5596d3d57fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 0x5596d3d57fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42768/0 (socket says 172.21.15.5:42768) 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45594/0 (socket says 172.21.15.5:45594) 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bffff700 1 -- 172.21.15.5:0/3564331269 learned_addr learned my addr 172.21.15.5:0/3564331269 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26c499c700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 0x5596d3d63030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 -- 172.21.15.5:0/3564331269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 msgr2=0x5596d3d57fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 0x5596d3d57fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 -- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 msgr2=0x5596d3d63030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 0x5596d3d63030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26bf7fe700 1 -- 172.21.15.5:0/3564331269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c01a6220 con 0x5596d3d4cd20 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.086+0000 7f26c499c700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 0x5596d3d63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26b4098090 tx=0x7f26b4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17623a2c0667fef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b409b620 con 0x5596d3d4cd20 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f26b409bd20 con 0x5596d3d4cd20 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596d3d40250 con 0x5596d3d4cd20 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26b409f770 con 0x5596d3d4cd20 2023-09-23T04:49:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f26d9c49b80 10 client.26492 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5596d3acade0 con 0x5596d3d4cd20 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596d3d40250 con 0x5596d3d4cd20 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.087+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f26b409ab10 con 0x5596d3d4cd20 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 10 client.26492 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 20 client.26492 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3d66ea0 0x5596d3d69290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 4 client.26492 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596d3c98e10 con 0x5596d3d66ea0 2023-09-23T04:49:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26bffff700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3d66ea0 0x5596d3d69290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f26bffff700 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3d66ea0 0x5596d3d69290 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4309a12fce453197 server_cookie=66a656f8ca97fede in_seq=0 out_seq=0 2023-09-23T04:49:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.088+0000 7f269f7fe700 10 client.26492 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.089+0000 7f269f7fe700 1 -- 172.21.15.5:0/3564331269 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596d3c98e10 con 0x5596d3d66ea0 2023-09-23T04:49:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.089+0000 7f269f7fe700 10 client.26492 handle_command_reply: tid=0 2023-09-23T04:49:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.089+0000 7f26d9c49b80 1 client.26492 shutdown 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 msgr2=0x5596d3d584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26b4098090 tx=0x7f26b4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 shutdown_connections 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3d66ea0 0x5596d3d69290 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d57bd0 0x5596d3d63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d4c400 0x5596d3d57fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/3564331269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4cd20 0x5596d3d584f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f269f7fe700 0 client.26492 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 >> 172.21.15.5:0/3564331269 conn(0x5596d3cc1260 msgr2=0x5596d3d48380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 shutdown_connections 2023-09-23T04:49:30.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 1 -- 172.21.15.5:0/3564331269 wait complete. 2023-09-23T04:49:30.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.090+0000 7f26d9c49b80 20 client.26492 trim_cache size 0 max 16384 2023-09-23T04:49:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 Processor -- start 2023-09-23T04:49:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 -- start start 2023-09-23T04:49:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d51fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3d524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3b20920 con 0x5596d3d4cd20 2023-09-23T04:49:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3d4df50 con 0x5596d3d4c400 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.106+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3d573f0 con 0x5596d3d57bd0 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26c499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3d524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45602/0 (socket says 172.21.15.5:45602) 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 -- 172.21.15.5:0/1788620863 learned_addr learned my addr 172.21.15.5:0/1788620863 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26c499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3d524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42776/0 (socket says 172.21.15.5:42776) 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d51fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 -- 172.21.15.5:0/1788620863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 msgr2=0x5596d3d524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3d524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 -- 172.21.15.5:0/1788620863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 msgr2=0x5596d3d51fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d51fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 -- 172.21.15.5:0/1788620863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596d3d44480 con 0x5596d3d4c400 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26bffff700 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097c0 tx=0x7f26b0004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a0c7f528cd872c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.107+0000 7f26a6ffd700 1 -- 172.21.15.5:0/1788620863 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0016070 con 0x5596d3d4c400 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26a6ffd700 1 -- 172.21.15.5:0/1788620863 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b0018040 con 0x5596d3d4c400 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26a6ffd700 1 -- 172.21.15.5:0/1788620863 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003d20 con 0x5596d3d4c400 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 -- 172.21.15.5:0/1788620863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 msgr2=0x5596d3d5cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26b00097c0 tx=0x7f26b0004320 comp rx=0 tx=0).stop 2023-09-23T04:49:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 -- 172.21.15.5:0/1788620863 shutdown_connections 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3d524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d5cf90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/1788620863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d51fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 -- 172.21.15.5:0/1788620863 >> 172.21.15.5:0/1788620863 conn(0x5596d3cc1260 msgr2=0x5596d3d56590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.108+0000 7f26d9c49b80 1 -- 172.21.15.5:0/1788620863 shutdown_connections 2023-09-23T04:49:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.109+0000 7f26d9c49b80 1 -- 172.21.15.5:0/1788620863 wait complete. 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.109+0000 7f26d9c49b80 1 -- start start 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.109+0000 7f26d9c49b80 1 Processor -- start 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3db4130 con 0x5596d3d4cd20 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3db42a0 con 0x5596d3d4c400 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d3db4460 con 0x5596d3d57bd0 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26d9c49b80 10 client.0 _ll_register_callbacks cb 0x7f26b0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26c499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26c499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42784/0 (socket says 172.21.15.5:42784) 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26c499c700 1 -- 172.21.15.5:0/4210491180 learned_addr learned my addr 172.21.15.5:0/4210491180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.110+0000 7f26bffff700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 -- 172.21.15.5:0/4210491180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 msgr2=0x5596d3d590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 -- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 msgr2=0x5596d3d595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 -- 172.21.15.5:0/4210491180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b0009c70 con 0x5596d3d57bd0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26bf7fe700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26c499c700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26b8017470 tx=0x7f26b8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea96ca86f9706ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f269f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b800e040 con 0x5596d3d57bd0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26b40860c0 con 0x5596d3d57bd0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.111+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f26b8017e00 con 0x5596d3d57bd0 2023-09-23T04:49:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26b8004bb0 con 0x5596d3d57bd0 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 10 client.26551 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5596d3ab3c60 con 0x5596d3d57bd0 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b40860c0 con 0x5596d3d57bd0 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f26b80233f0 con 0x5596d3d57bd0 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 10 client.26551 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 20 client.26551 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3dbbcc0 0x5596d3dbe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 4 client.26551 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:49:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596d3c98b30 con 0x5596d3dbbcc0 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.112+0000 7f26bffff700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3dbbcc0 0x5596d3dbe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f26bffff700 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3dbbcc0 0x5596d3dbe0e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc62772c17b16b0 server_cookie=206d6c6cde1d816c in_seq=0 out_seq=0 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f269f7fe700 10 client.26551 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f269f7fe700 1 -- 172.21.15.5:0/4210491180 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596d3c98b30 con 0x5596d3dbbcc0 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f269f7fe700 10 client.26551 handle_command_reply: tid=0 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f26d9c49b80 1 client.26551 shutdown 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 msgr2=0x5596d3db3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.113+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26b8017470 tx=0x7f26b8004b40 comp rx=0 tx=0).stop 2023-09-23T04:49:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 shutdown_connections 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5596d3dbbcc0 0x5596d3dbe0e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5596d3d57bd0 0x5596d3db3bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5596d3d4c400 0x5596d3d590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f269f7fe700 0 client.26551 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 --2- 172.21.15.5:0/4210491180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596d3d4cd20 0x5596d3d595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 >> 172.21.15.5:0/4210491180 conn(0x5596d3cc1260 msgr2=0x5596d3d4e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 shutdown_connections 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 1 -- 172.21.15.5:0/4210491180 wait complete. 2023-09-23T04:49:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.114+0000 7f26d9c49b80 20 client.26551 trim_cache size 0 max 16384 2023-09-23T04:49:30.121 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:49:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 msgr2=0x7f26a80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 0x7f26a80a3bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26b800f540 tx=0x7f26b8009920 comp rx=0 tx=0).stop 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 msgr2=0x7f26c01996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26b4000c00 tx=0x7f26b4005720 comp rx=0 tx=0).stop 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 shutdown_connections 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26a80a1710 0x7f26a80a3bd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26c019db50 0x7f26c019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c0103310 0x7f26c01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 --2- 172.21.15.5:0/1855995237 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26c0108430 0x7f26c01996f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 >> 172.21.15.5:0/1855995237 conn(0x7f26c00feb50 msgr2=0x7f26c00ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 shutdown_connections 2023-09-23T04:49:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:30.115+0000 7f26c63ff700 1 -- 172.21.15.5:0/1855995237 wait complete. 2023-09-23T04:49:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:29 smithi118 ceph-mon[134297]: pgmap v99: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 412 op/s 2023-09-23T04:49:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:49:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:49:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:49:30.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 192460 8.83 MB/sec execute 6 sec latency 24.472 ms 2023-09-23T04:49:31.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:49:31.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:49:31.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 -- 172.21.15.5:0/1591158873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 msgr2=0x7f68d80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 --2- 172.21.15.5:0/1591158873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d80735d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009990 tx=0x7f68c8009ca0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 -- 172.21.15.5:0/1591158873 shutdown_connections 2023-09-23T04:49:31.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 --2- 172.21.15.5:0/1591158873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 0x7f68d810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 --2- 172.21.15.5:0/1591158873 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d80735d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 --2- 172.21.15.5:0/1591158873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 -- 172.21.15.5:0/1591158873 >> 172.21.15.5:0/1591158873 conn(0x7f68d80fc9d0 msgr2=0x7f68d80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 -- 172.21.15.5:0/1591158873 shutdown_connections 2023-09-23T04:49:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.384+0000 7f68dcdff700 1 -- 172.21.15.5:0/1591158873 wait complete. 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.385+0000 7f68dcdff700 1 Processor -- start 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.385+0000 7f68dcdff700 1 -- start start 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.385+0000 7f68dcdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68dcdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68dcdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 0x7f68d819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68dcdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d819e1f0 con 0x7f68d810d720 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68dcdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d819e360 con 0x7f68d8073b10 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68dcdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d819e520 con 0x7f68d80731f0 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48674/0 (socket says 172.21.15.5:48674) 2023-09-23T04:49:31.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 0x7f68d819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 -- 172.21.15.5:0/2699742313 learned_addr learned my addr 172.21.15.5:0/2699742313 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 -- 172.21.15.5:0/2699742313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 msgr2=0x7f68d8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 -- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 msgr2=0x7f68d819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 0x7f68d819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d659c700 1 -- 172.21.15.5:0/2699742313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0009c40 con 0x7f68d80731f0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.386+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68d659c700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f68c8000c00 tx=0x7f68c800f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68c77fe700 1 -- 172.21.15.5:0/2699742313 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c801d070 con 0x7f68d80731f0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68c77fe700 1 -- 172.21.15.5:0/2699742313 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68c800fc20 con 0x7f68d80731f0 2023-09-23T04:49:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68c77fe700 1 -- 172.21.15.5:0/2699742313 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80173f0 con 0x7f68d80731f0 2023-09-23T04:49:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80095f0 con 0x7f68d80731f0 2023-09-23T04:49:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.387+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d81a5e60 con 0x7f68d80731f0 2023-09-23T04:49:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.391+0000 7f68c77fe700 1 -- 172.21.15.5:0/2699742313 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f68c8021410 con 0x7f68d80731f0 2023-09-23T04:49:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.392+0000 7f68c77fe700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 0x7f68c00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.392+0000 7f68c77fe700 1 -- 172.21.15.5:0/2699742313 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f68c80ce890 con 0x7f68d80731f0 2023-09-23T04:49:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.392+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 0x7f68c00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.393+0000 7f68f06f6b80 1 Processor -- start 2023-09-23T04:49:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.393+0000 7f68f06f6b80 1 -- start start 2023-09-23T04:49:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.393+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 0x7f68c00a3c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f68cc005950 tx=0x7f68cc0058e0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385320c4e0 con 0x5638532a3da0 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638531fdca0 con 0x5638532a3480 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638531fb5c0 con 0x5638532aec50 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38674/0 (socket says 172.21.15.5:38674) 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 -- 172.21.15.5:0/4203732807 learned_addr learned my addr 172.21.15.5:0/4203732807 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 -- 172.21.15.5:0/4203732807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 msgr2=0x5638532b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 -- 172.21.15.5:0/4203732807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 msgr2=0x5638532ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.394+0000 7f68d659c700 1 -- 172.21.15.5:0/4203732807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563853259950 con 0x5638532a3480 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.395+0000 7f68d659c700 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68c8097ad0 tx=0x7f68c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a107095b264f8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.395+0000 7f68c4ff9700 1 -- 172.21.15.5:0/4203732807 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c809b5d0 con 0x5638532a3480 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.395+0000 7f68c4ff9700 1 -- 172.21.15.5:0/4203732807 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68c809f720 con 0x5638532a3480 2023-09-23T04:49:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.395+0000 7f68c4ff9700 1 -- 172.21.15.5:0/4203732807 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c809ad20 con 0x5638532a3480 2023-09-23T04:49:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 -- 172.21.15.5:0/4203732807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 msgr2=0x5638532a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f68c8097ad0 tx=0x7f68c8009310 comp rx=0 tx=0).stop 2023-09-23T04:49:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 -- 172.21.15.5:0/4203732807 shutdown_connections 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532a3860 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/4203732807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 -- 172.21.15.5:0/4203732807 >> 172.21.15.5:0/4203732807 conn(0x5638532182e0 msgr2=0x56385329f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 -- 172.21.15.5:0/4203732807 shutdown_connections 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.396+0000 7f68f06f6b80 1 -- 172.21.15.5:0/4203732807 wait complete. 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 -- start start 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 Processor -- start 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532aec50 0x5638532c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.397+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638531f6eb0 con 0x5638532a3da0 2023-09-23T04:49:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563853193790 con 0x5638532aec50 2023-09-23T04:49:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563853193430 con 0x5638532a3480 2023-09-23T04:49:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68f06f6b80 10 client.0 _ll_register_callbacks cb 0x563852ff0880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48700/0 (socket says 172.21.15.5:48700) 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 -- 172.21.15.5:0/3792089286 learned_addr learned my addr 172.21.15.5:0/3792089286 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532aec50 0x5638532c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 -- 172.21.15.5:0/3792089286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532aec50 msgr2=0x5638532c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532aec50 0x5638532c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 -- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 msgr2=0x5638532b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.398+0000 7f68d659c700 1 -- 172.21.15.5:0/3792089286 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d81a5e60 con 0x5638532a3480 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68d659c700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68c8097350 tx=0x7f68c8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b55c9fc2d4100c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c809ba30 con 0x5638532a3480 2023-09-23T04:49:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f68c809b5d0 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638532972d0 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68c80a3440 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68f06f6b80 10 client.26563 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563853021e60 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.399+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638532972d0 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f68c80216c0 con 0x5638532a3480 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 10 client.26563 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 10 client.26563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 20 client.26563 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638532c67b0 0x5638532c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 4 client.26563 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.400+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638531efe90 con 0x5638532c67b0 2023-09-23T04:49:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.401+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638532c67b0 0x5638532c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.401+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638532c67b0 0x5638532c8bd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2c53dabfb9aace server_cookie=6c61945bc3216c5d in_seq=0 out_seq=0 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.401+0000 7f68bdffb700 10 client.26563 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68bdffb700 1 -- 172.21.15.5:0/3792089286 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638531efe90 con 0x5638532c67b0 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68bdffb700 10 client.26563 handle_command_reply: tid=0 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 client.26563 shutdown 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 msgr2=0x5638532b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f68c8097350 tx=0x7f68c8005c00 comp rx=0 tx=0).stop 2023-09-23T04:49:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 shutdown_connections 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638532c67b0 0x5638532c8bd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532aec50 0x5638532c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68bdffb700 0 client.26563 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x5638532b7840 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/3792089286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 >> 172.21.15.5:0/3792089286 conn(0x5638532182e0 msgr2=0x56385329f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 shutdown_connections 2023-09-23T04:49:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.402+0000 7f68f06f6b80 1 -- 172.21.15.5:0/3792089286 wait complete. 2023-09-23T04:49:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.403+0000 7f68f06f6b80 20 client.26563 trim_cache size 0 max 16384 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.423+0000 7f68f06f6b80 1 Processor -- start 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.423+0000 7f68f06f6b80 1 -- start start 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638530779a0 con 0x5638532a3da0 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385329be50 con 0x5638532a3480 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638532b84d0 con 0x5638532aec50 2023-09-23T04:49:31.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d6d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48710/0 (socket says 172.21.15.5:48710) 2023-09-23T04:49:31.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d6d9d700 1 -- 172.21.15.5:0/920636891 learned_addr learned my addr 172.21.15.5:0/920636891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d659c700 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.424+0000 7f68d6d9d700 1 -- 172.21.15.5:0/920636891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 msgr2=0x5638532bc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532bc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68d6d9d700 1 -- 172.21.15.5:0/920636891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 msgr2=0x5638532c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68d6d9d700 1 -- 172.21.15.5:0/920636891 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385300ace0 con 0x5638532aec50 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f68d0014960 tx=0x7f68d0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f16c6d7647c78d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68c57fa700 1 -- 172.21.15.5:0/920636891 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000e040 con 0x5638532aec50 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68c57fa700 1 -- 172.21.15.5:0/920636891 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68d000f040 con 0x5638532aec50 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.425+0000 7f68c57fa700 1 -- 172.21.15.5:0/920636891 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004290 con 0x5638532aec50 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 -- 172.21.15.5:0/920636891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 msgr2=0x5638532c24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f68d0014960 tx=0x7f68d0009920 comp rx=0 tx=0).stop 2023-09-23T04:49:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 -- 172.21.15.5:0/920636891 shutdown_connections 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532aec50 0x5638532c24d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3480 0x5638532bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/920636891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532a3da0 0x5638532c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 -- 172.21.15.5:0/920636891 >> 172.21.15.5:0/920636891 conn(0x5638532182e0 msgr2=0x5638532acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 -- 172.21.15.5:0/920636891 shutdown_connections 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.426+0000 7f68f06f6b80 1 -- 172.21.15.5:0/920636891 wait complete. 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.427+0000 7f68f06f6b80 1 -- start start 2023-09-23T04:49:31.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.427+0000 7f68f06f6b80 1 Processor -- start 2023-09-23T04:49:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3da0 0x563853315080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 0x5638532bb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638532bbc10 con 0x5638532aec50 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638532bbd80 con 0x5638532a3da0 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638532bbf40 con 0x5638532a3480 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68f06f6b80 10 client.0 _ll_register_callbacks cb 0x563852ff0880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 0x5638532bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48722/0 (socket says 172.21.15.5:48722) 2023-09-23T04:49:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 -- 172.21.15.5:0/2396384442 learned_addr learned my addr 172.21.15.5:0/2396384442 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3da0 0x563853315080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 -- 172.21.15.5:0/2396384442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3da0 msgr2=0x563853315080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3da0 0x563853315080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 -- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 msgr2=0x5638532bb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 0x5638532bb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d659c700 1 -- 172.21.15.5:0/2396384442 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d0014610 con 0x5638532a3480 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.428+0000 7f68d6d9d700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 0x5638532bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68d659c700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f68c809b2d0 tx=0x7f68c8005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa347e993009e48 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80ce650 con 0x5638532a3480 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f68c809b5d0 con 0x5638532a3480 2023-09-23T04:49:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68c809f720 con 0x5638532a3480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68c8097410 con 0x5638532a3480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68f06f6b80 10 client.26569 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5638530482d0 con 0x5638532a3480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68c809f720 con 0x5638532a3480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.429+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f68c809a530 con 0x5638532a3480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 20 client.26569 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385331c480 0x56385331e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 4 client.26569 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5638531efbb0 con 0x56385331c480 2023-09-23T04:49:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385331c480 0x56385331e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68bdffb700 10 client.26569 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.430+0000 7f68d5d9b700 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385331c480 0x56385331e870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652141183fedc8ea server_cookie=4f7ca0cfc052029c in_seq=0 out_seq=0 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68bdffb700 1 -- 172.21.15.5:0/2396384442 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5638531efbb0 con 0x56385331c480 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68bdffb700 10 client.26569 handle_command_reply: tid=0 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 client.26569 shutdown 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 msgr2=0x563853314b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f68c809b2d0 tx=0x7f68c8005ad0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 shutdown_connections 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385331c480 0x56385331e870 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638532aec50 0x5638532bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638532a3480 0x563853314b40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 --2- 172.21.15.5:0/2396384442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638532a3da0 0x563853315080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 >> 172.21.15.5:0/2396384442 conn(0x5638532182e0 msgr2=0x56385329ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.432+0000 7f68bdffb700 0 client.26569 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.433+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 shutdown_connections 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.433+0000 7f68f06f6b80 1 -- 172.21.15.5:0/2396384442 wait complete. 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.433+0000 7f68f06f6b80 20 client.26569 trim_cache size 0 max 16384 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "52de4d7d-3a5c-4c03-8f9a-15df28303842", 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:49:31.436 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 msgr2=0x7f68c00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 0x7f68c00a3c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f68cc005950 tx=0x7f68cc0058e0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 msgr2=0x7f68d8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f68c8000c00 tx=0x7f68c800f510 comp rx=0 tx=0).stop 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 shutdown_connections 2023-09-23T04:49:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f68c00a1790 0x7f68c00a3c50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d810d720 0x7f68d819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f68d80731f0 0x7f68d8199210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 --2- 172.21.15.5:0/2699742313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d8073b10 0x7f68d8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 >> 172.21.15.5:0/2699742313 conn(0x7f68d80fc9d0 msgr2=0x7f68d810f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 shutdown_connections 2023-09-23T04:49:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.434+0000 7f68dcdff700 1 -- 172.21.15.5:0/2699742313 wait complete. 2023-09-23T04:49:31.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:49:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 -- 172.21.15.5:0/3924356651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310106210 msgr2=0x7fe31010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 --2- 172.21.15.5:0/3924356651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310106210 0x7fe31010dd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe308009f90 tx=0x7fe30800b330 comp rx=0 tx=0).stop 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 -- 172.21.15.5:0/3924356651 shutdown_connections 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 --2- 172.21.15.5:0/3924356651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe310106210 0x7fe31010dd90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 --2- 172.21.15.5:0/3924356651 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3100690a0 0x7fe310069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 --2- 172.21.15.5:0/3924356651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe310069a50 0x7fe310105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 -- 172.21.15.5:0/3924356651 >> 172.21.15.5:0/3924356651 conn(0x7fe310077ca0 msgr2=0x7fe3100780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 -- 172.21.15.5:0/3924356651 shutdown_connections 2023-09-23T04:49:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.691+0000 7fe317dff700 1 -- 172.21.15.5:0/3924356651 wait complete. 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.692+0000 7fe317dff700 1 Processor -- start 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.692+0000 7fe317dff700 1 -- start start 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 0x7fe3101990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 0x7fe3101995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 0x7fe31019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31019e4d0 con 0x7fe3100690a0 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31019e640 con 0x7fe31019da50 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe317dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31019e800 con 0x7fe310106210 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 0x7fe3101990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe31539a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 0x7fe3101995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe31639c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 0x7fe31019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 0x7fe3101990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58014/0 (socket says 172.21.15.5:58014) 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.693+0000 7fe315b9b700 1 -- 172.21.15.5:0/4231671677 learned_addr learned my addr 172.21.15.5:0/4231671677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 -- 172.21.15.5:0/4231671677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 msgr2=0x7fe31019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 0x7fe31019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 -- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 msgr2=0x7fe3101990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 0x7fe3101990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 -- 172.21.15.5:0/4231671677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009c40 con 0x7fe310106210 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31639c700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 0x7fe31019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe31539a700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 0x7fe3101995f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe300005fd0 tx=0x7fe30000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe306ffd700 1 -- 172.21.15.5:0/4231671677 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300010900 con 0x7fe310106210 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3101a5d60 con 0x7fe310106210 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe306ffd700 1 -- 172.21.15.5:0/4231671677 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe300018420 con 0x7fe310106210 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe306ffd700 1 -- 172.21.15.5:0/4231671677 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000f680 con 0x7fe310106210 2023-09-23T04:49:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.694+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3101a6230 con 0x7fe310106210 2023-09-23T04:49:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.698+0000 7fe306ffd700 1 -- 172.21.15.5:0/4231671677 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe300018590 con 0x7fe310106210 2023-09-23T04:49:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.699+0000 7fe32b7afb80 1 Processor -- start 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.699+0000 7fe32b7afb80 1 -- start start 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe306ffd700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 0x7fe2fc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe306ffd700 1 -- 172.21.15.5:0/4231671677 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe300014070 con 0x7fe310106210 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 0x55bef97a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef97a5bd0 0x55bef97a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe31639c700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 0x7fe2fc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9703460 con 0x55bef97a5bd0 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef96f4c20 con 0x55bef979a400 2023-09-23T04:49:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef96f2540 con 0x55bef979ad20 2023-09-23T04:49:31.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38726/0 (socket says 172.21.15.5:38726) 2023-09-23T04:49:31.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe315b9b700 1 -- 172.21.15.5:0/4257520965 learned_addr learned my addr 172.21.15.5:0/4257520965 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe31539a700 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 0x55bef97a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.700+0000 7fe31639c700 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef97a5bd0 0x55bef97a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 -- 172.21.15.5:0/4257520965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 msgr2=0x55bef979a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 -- 172.21.15.5:0/4257520965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef97a5bd0 msgr2=0x55bef97a7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef97a5bd0 0x55bef97a7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 -- 172.21.15.5:0/4257520965 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef97508d0 con 0x55bef979ad20 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31639c700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 0x7fe2fc0a3bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe308009790 tx=0x7fe308003f80 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe31539a700 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 0x55bef97a5690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3000caa20 tx=0x7fe300099e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0eb9251100d6ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe2fbfff700 1 -- 172.21.15.5:0/4257520965 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30009a240 con 0x55bef979ad20 2023-09-23T04:49:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe2fbfff700 1 -- 172.21.15.5:0/4257520965 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe30009e390 con 0x55bef979ad20 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.701+0000 7fe2fbfff700 1 -- 172.21.15.5:0/4257520965 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000cd4f0 con 0x55bef979ad20 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 -- 172.21.15.5:0/4257520965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 msgr2=0x55bef97a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 0x55bef97a5690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3000caa20 tx=0x7fe300099e00 comp rx=0 tx=0).stop 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 -- 172.21.15.5:0/4257520965 shutdown_connections 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef97a5bd0 0x55bef97a7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/4257520965 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979ad20 0x55bef97a5690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 -- 172.21.15.5:0/4257520965 >> 172.21.15.5:0/4257520965 conn(0x55bef970f260 msgr2=0x55bef9796790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 -- 172.21.15.5:0/4257520965 shutdown_connections 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.702+0000 7fe32b7afb80 1 -- 172.21.15.5:0/4257520965 wait complete. 2023-09-23T04:49:31.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.703+0000 7fe32b7afb80 1 -- start start 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.703+0000 7fe32b7afb80 1 Processor -- start 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.703+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 0x55bef97a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 0x55bef97a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef96ede30 con 0x55bef979ad20 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef968a710 con 0x55bef97a5bd0 2023-09-23T04:49:31.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef968a3b0 con 0x55bef979a400 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe32b7afb80 10 client.0 _ll_register_callbacks cb 0x55bef94e7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 0x55bef97a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 0x55bef97a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 0x55bef97a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48756/0 (socket says 172.21.15.5:48756) 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe315b9b700 1 -- 172.21.15.5:0/2575423971 learned_addr learned my addr 172.21.15.5:0/2575423971 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 -- 172.21.15.5:0/2575423971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 msgr2=0x55bef97a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 0x55bef97a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 -- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 msgr2=0x55bef97a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.704+0000 7fe31639c700 1 -- 172.21.15.5:0/2575423971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3101a6230 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe31639c700 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 0x55bef97a33e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe308009550 tx=0x7fe308023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf839388308a5098 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe2f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800e040 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bef978e250 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe308005600 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3080057c0 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe32b7afb80 10 client.26528 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bef9518de0 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.705+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bef978e250 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe30801f440 con 0x55bef97a5bd0 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:31.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 20 client.26528 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef97b56c0 0x55bef97b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 4 client.26528 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bef96e6e10 con 0x55bef97b56c0 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.706+0000 7fe315b9b700 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef97b56c0 0x55bef97b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe315b9b700 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef97b56c0 0x55bef97b7ae0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbeb7fa6547122f server_cookie=9e831bb26914c73c in_seq=0 out_seq=0 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe2f97fa700 10 client.26528 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe2f97fa700 1 -- 172.21.15.5:0/2575423971 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bef96e6e10 con 0x55bef97b56c0 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe2f97fa700 10 client.26528 handle_command_reply: tid=0 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe32b7afb80 1 client.26528 shutdown 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 msgr2=0x55bef97a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.707+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 0x55bef97a33e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe308009550 tx=0x7fe308023040 comp rx=0 tx=0).stop 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 shutdown_connections 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef97b56c0 0x55bef97b7ae0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef97a5bd0 0x55bef97a33e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef979a400 0x55bef97a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe2f97fa700 0 client.26528 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/2575423971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 >> 172.21.15.5:0/2575423971 conn(0x55bef970f260 msgr2=0x55bef9796320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 shutdown_connections 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 1 -- 172.21.15.5:0/2575423971 wait complete. 2023-09-23T04:49:31.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.708+0000 7fe32b7afb80 20 client.26528 trim_cache size 0 max 16384 2023-09-23T04:49:31.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.727+0000 7fe32b7afb80 1 Processor -- start 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 -- start start 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef979fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef956e920 con 0x55bef979ad20 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef97acc60 con 0x55bef979a400 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef97b2090 con 0x55bef97a5bd0 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38752/0 (socket says 172.21.15.5:38752) 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe31639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef979fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe315b9b700 1 -- 172.21.15.5:0/1852494521 learned_addr learned my addr 172.21.15.5:0/1852494521 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.728+0000 7fe31539a700 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 -- 172.21.15.5:0/1852494521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 msgr2=0x55bef979d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 -- 172.21.15.5:0/1852494521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 msgr2=0x55bef97ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 -- 172.21.15.5:0/1852494521 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9501c60 con 0x55bef97a5bd0 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe315b9b700 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:49:31.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe31639c700 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef979fbd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe30801f850 tx=0x7fe308009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27aaf48bf28f0075 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe304bff700 1 -- 172.21.15.5:0/1852494521 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30803f070 con 0x55bef97a5bd0 2023-09-23T04:49:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe304bff700 1 -- 172.21.15.5:0/1852494521 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe308003b90 con 0x55bef97a5bd0 2023-09-23T04:49:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.729+0000 7fe304bff700 1 -- 172.21.15.5:0/1852494521 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800e040 con 0x55bef97a5bd0 2023-09-23T04:49:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1852494521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 msgr2=0x55bef979fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef979fbd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe30801f850 tx=0x7fe308009920 comp rx=0 tx=0).stop 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1852494521 shutdown_connections 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef979fbd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef979d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1852494521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1852494521 >> 172.21.15.5:0/1852494521 conn(0x55bef970f260 msgr2=0x55bef97a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1852494521 shutdown_connections 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.730+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1852494521 wait complete. 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 -- start start 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 Processor -- start 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef98024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9802a10 con 0x55bef979ad20 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9802b50 con 0x55bef979a400 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9802d10 con 0x55bef97a5bd0 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.731+0000 7fe32b7afb80 10 client.0 _ll_register_callbacks cb 0x7fe30800be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38754/0 (socket says 172.21.15.5:38754) 2023-09-23T04:49:31.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe31539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 -- 172.21.15.5:0/1774388949 learned_addr learned my addr 172.21.15.5:0/1774388949 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe31639c700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef98024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 -- 172.21.15.5:0/1774388949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 msgr2=0x55bef98024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef98024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 -- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 msgr2=0x55bef97fdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97fdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 -- 172.21.15.5:0/1774388949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c009a50 con 0x55bef979a400 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe315b9b700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe30c01f580 tx=0x7fe30c01fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3daaca1838cd0ca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.732+0000 7fe2f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c010b80 con 0x55bef979a400 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bef96e6b30 con 0x55bef979a400 2023-09-23T04:49:31.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe30c010ce0 con 0x55bef979a400 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c010ea0 con 0x55bef979a400 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe32b7afb80 10 client.26534 fetch_fsmap learned FSMap version 29 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bef9518de0 con 0x55bef979a400 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bef96e6b30 con 0x55bef979a400 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 29) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe30c028440 con 0x55bef979a400 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe32b7afb80 10 client.26534 fetch_fsmap finished waiting for FSMap version 29 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.733+0000 7fe32b7afb80 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe32b7afb80 20 client.26534 populate_metadata read hostname 'smithi005' 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef980a5d0 0x55bef980c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe32b7afb80 4 client.26534 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:49:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bef9539f70 con 0x55bef980a5d0 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe31539a700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef980a5d0 0x55bef980c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe31539a700 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef980a5d0 0x55bef980c9f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ccbd231ff7badd3 server_cookie=3aa08f8c1bdf238d in_seq=0 out_seq=0 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.734+0000 7fe2f97fa700 10 client.26534 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe2f97fa700 1 -- 172.21.15.5:0/1774388949 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bef9539f70 con 0x55bef980a5d0 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe2f97fa700 10 client.26534 handle_command_reply: tid=0 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 client.26534 shutdown 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 msgr2=0x55bef97fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe30c01f580 tx=0x7fe30c01fcd0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 shutdown_connections 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55bef980a5d0 0x55bef980c9f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55bef97a5bd0 0x55bef98024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bef979a400 0x55bef97fd670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 --2- 172.21.15.5:0/1774388949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bef979ad20 0x55bef97fdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe2f97fa700 0 client.26534 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 >> 172.21.15.5:0/1774388949 conn(0x55bef970f260 msgr2=0x55bef97a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 shutdown_connections 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 1 -- 172.21.15.5:0/1774388949 wait complete. 2023-09-23T04:49:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.735+0000 7fe32b7afb80 20 client.26534 trim_cache size 0 max 16384 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "52de4d7d-3a5c-4c03-8f9a-15df28303842": { 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "52de4d7d-3a5c-4c03-8f9a-15df28303842", 2023-09-23T04:49:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 msgr2=0x7fe2fc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 0x7fe2fc0a3bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe308009790 tx=0x7fe308003f80 comp rx=0 tx=0).stop 2023-09-23T04:49:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 msgr2=0x7fe3101995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 0x7fe3101995f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe300005fd0 tx=0x7fe30000eca0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 shutdown_connections 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe2fc0a1710 0x7fe2fc0a3bd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe31019da50 0x7fe31019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3100690a0 0x7fe3101990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 --2- 172.21.15.5:0/4231671677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe310106210 0x7fe3101995f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 >> 172.21.15.5:0/4231671677 conn(0x7fe310077ca0 msgr2=0x7fe31010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 shutdown_connections 2023-09-23T04:49:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:49:31.736+0000 7fe317dff700 1 -- 172.21.15.5:0/4231671677 wait complete. 2023-09-23T04:49:31.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:31 smithi005 ceph-mon[97638]: pgmap v100: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2023-09-23T04:49:31.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:31 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:31.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:31 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:31.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:31 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:31.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:31 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:31.756 INFO:tasks.cephfs.filesystem:scrub status for tag:52de4d7d-3a5c-4c03-8f9a-15df28303842 - {'path': '/', 'tag': '52de4d7d-3a5c-4c03-8f9a-15df28303842', 'options': 'recursive,force'} 2023-09-23T04:49:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:31 smithi118 ceph-mon[134297]: pgmap v100: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2023-09-23T04:49:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:31 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:31 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:31 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:31 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:31.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 194722 9.35 MB/sec execute 7 sec latency 21.116 ms 2023-09-23T04:49:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:31 smithi069 ceph-mon[133931]: pgmap v100: 129 pgs: 129 active+clean; 113 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2023-09-23T04:49:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:31 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:31 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:31 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:31 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:32 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:49:32.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:32 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 196009 9.02 MB/sec execute 8 sec latency 13.374 ms 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:49:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:32.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:32 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:49:33.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:33 smithi005 ceph-mon[97638]: pgmap v101: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-23T04:49:33.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:33 smithi118 ceph-mon[134297]: pgmap v101: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-23T04:49:33.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 198455 9.54 MB/sec execute 9 sec latency 11.520 ms 2023-09-23T04:49:33.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:33 smithi069 ceph-mon[133931]: pgmap v101: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 292 op/s 2023-09-23T04:49:34.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199618 9.14 MB/sec execute 10 sec latency 12.067 ms 2023-09-23T04:49:35.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202126 9.81 MB/sec execute 11 sec latency 11.788 ms 2023-09-23T04:49:35.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:35 smithi069 ceph-mon[133931]: pgmap v102: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-09-23T04:49:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:35 smithi005 ceph-mon[97638]: pgmap v102: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-09-23T04:49:36.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:35 smithi118 ceph-mon[134297]: pgmap v102: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-09-23T04:49:36.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 203166 9.21 MB/sec execute 12 sec latency 37.338 ms 2023-09-23T04:49:37.934 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205198 9.40 MB/sec execute 13 sec latency 16.926 ms 2023-09-23T04:49:37.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:37 smithi069 ceph-mon[133931]: pgmap v103: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2023-09-23T04:49:38.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:37 smithi005 ceph-mon[97638]: pgmap v103: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2023-09-23T04:49:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:37 smithi118 ceph-mon[134297]: pgmap v103: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2023-09-23T04:49:38.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205675 9.07 MB/sec execute 14 sec latency 829.572 ms 2023-09-23T04:49:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:38 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:38.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:38 smithi069 ceph-mon[133931]: pgmap v104: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2023-09-23T04:49:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:38 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:38 smithi005 ceph-mon[97638]: pgmap v104: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2023-09-23T04:49:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:38 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:39.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:38 smithi118 ceph-mon[134297]: pgmap v104: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2023-09-23T04:49:39.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205675 8.47 MB/sec execute 15 sec latency 1829.736 ms 2023-09-23T04:49:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:40 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:40.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:40 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:40.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205675 7.94 MB/sec execute 16 sec latency 2829.850 ms 2023-09-23T04:49:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:40 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:41 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:41 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:41 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:49:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:41 smithi005 ceph-mon[97638]: pgmap v105: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2023-09-23T04:49:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:41 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:41 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:41 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:49:41.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:41 smithi118 ceph-mon[134297]: pgmap v105: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2023-09-23T04:49:41.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205675 7.47 MB/sec execute 17 sec latency 3829.956 ms 2023-09-23T04:49:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:41 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:41 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:41 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:49:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:41 smithi069 ceph-mon[133931]: pgmap v105: 129 pgs: 129 active+clean; 124 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2023-09-23T04:49:42.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206305 7.15 MB/sec execute 18 sec latency 4128.058 ms 2023-09-23T04:49:43.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 208502 7.36 MB/sec execute 19 sec latency 17.356 ms 2023-09-23T04:49:43.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:43 smithi069 ceph-mon[133931]: pgmap v106: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2023-09-23T04:49:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:43 smithi005 ceph-mon[97638]: pgmap v106: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2023-09-23T04:49:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:43 smithi118 ceph-mon[134297]: pgmap v106: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 196 op/s 2023-09-23T04:49:44.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209238 7.31 MB/sec execute 20 sec latency 306.507 ms 2023-09-23T04:49:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:44 smithi005 ceph-mon[97638]: pgmap v107: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 755 KiB/s wr, 119 op/s 2023-09-23T04:49:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:44 smithi118 ceph-mon[134297]: pgmap v107: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 755 KiB/s wr, 119 op/s 2023-09-23T04:49:45.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:44 smithi069 ceph-mon[133931]: pgmap v107: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 755 KiB/s wr, 119 op/s 2023-09-23T04:49:45.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 210000 7.05 MB/sec execute 21 sec latency 47.560 ms 2023-09-23T04:49:46.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 212257 7.29 MB/sec execute 22 sec latency 24.433 ms 2023-09-23T04:49:47.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 213507 7.27 MB/sec execute 23 sec latency 19.220 ms 2023-09-23T04:49:47.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:47 smithi069 ceph-mon[133931]: pgmap v108: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 721 KiB/s wr, 116 op/s 2023-09-23T04:49:48.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:47 smithi005 ceph-mon[97638]: pgmap v108: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 721 KiB/s wr, 116 op/s 2023-09-23T04:49:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:47 smithi118 ceph-mon[134297]: pgmap v108: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 721 KiB/s wr, 116 op/s 2023-09-23T04:49:48.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 215896 7.48 MB/sec execute 24 sec latency 12.798 ms 2023-09-23T04:49:49.935 INFO:tasks.workunit.client.0.smithi005.stdout: 1 217213 7.46 MB/sec execute 25 sec latency 31.634 ms 2023-09-23T04:49:49.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:49 smithi069 ceph-mon[133931]: pgmap v109: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2023-09-23T04:49:50.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:49 smithi005 ceph-mon[97638]: pgmap v109: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2023-09-23T04:49:50.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:49 smithi118 ceph-mon[134297]: pgmap v109: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2023-09-23T04:49:50.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 219432 7.64 MB/sec execute 26 sec latency 26.184 ms 2023-09-23T04:49:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:51 smithi005 ceph-mon[97638]: pgmap v110: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-09-23T04:49:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:51 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:51 smithi118 ceph-mon[134297]: pgmap v110: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-09-23T04:49:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:51 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:51.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220698 7.62 MB/sec execute 27 sec latency 36.932 ms 2023-09-23T04:49:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:51 smithi069 ceph-mon[133931]: pgmap v110: 129 pgs: 129 active+clean; 125 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 3.5 MiB/s wr, 201 op/s 2023-09-23T04:49:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:51 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:49:52.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223131 7.80 MB/sec execute 28 sec latency 11.988 ms 2023-09-23T04:49:53.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 224303 7.75 MB/sec execute 29 sec latency 16.570 ms 2023-09-23T04:49:53.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:53 smithi069 ceph-mon[133931]: pgmap v111: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-23T04:49:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:53 smithi005 ceph-mon[97638]: pgmap v111: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-23T04:49:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:53 smithi118 ceph-mon[134297]: pgmap v111: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2023-09-23T04:49:54.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 226708 7.92 MB/sec execute 30 sec latency 9.602 ms 2023-09-23T04:49:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:54 smithi005 ceph-mon[97638]: pgmap v112: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-09-23T04:49:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:54 smithi118 ceph-mon[134297]: pgmap v112: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-09-23T04:49:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:54 smithi069 ceph-mon[133931]: pgmap v112: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2023-09-23T04:49:55.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227845 7.87 MB/sec execute 31 sec latency 10.453 ms 2023-09-23T04:49:56.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230046 8.00 MB/sec execute 32 sec latency 19.706 ms 2023-09-23T04:49:57.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 231337 7.97 MB/sec execute 33 sec latency 11.247 ms 2023-09-23T04:49:57.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:57 smithi069 ceph-mon[133931]: pgmap v113: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2023-09-23T04:49:58.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:57 smithi005 ceph-mon[97638]: pgmap v113: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2023-09-23T04:49:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:57 smithi118 ceph-mon[134297]: pgmap v113: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 278 op/s 2023-09-23T04:49:58.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 233774 8.11 MB/sec execute 34 sec latency 12.049 ms 2023-09-23T04:49:59.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234948 8.06 MB/sec execute 35 sec latency 12.730 ms 2023-09-23T04:49:59.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:49:59 smithi069 ceph-mon[133931]: pgmap v114: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2023-09-23T04:50:00.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:49:59 smithi005 ceph-mon[97638]: pgmap v114: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2023-09-23T04:50:00.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:49:59 smithi118 ceph-mon[134297]: pgmap v114: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 368 op/s 2023-09-23T04:50:00.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237354 8.20 MB/sec execute 36 sec latency 15.093 ms 2023-09-23T04:50:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:00 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T04:50:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:00 smithi069 ceph-mon[133931]: pgmap v115: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:00 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T04:50:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:00 smithi005 ceph-mon[97638]: pgmap v115: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:00 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T04:50:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:00 smithi118 ceph-mon[134297]: pgmap v115: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:01.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:50:01.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237949 8.12 MB/sec execute 37 sec latency 57.553 ms 2023-09-23T04:50:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:01 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.995+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/2944188182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 msgr2=0x7f3cd410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.995+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/2944188182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd410dd40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b0d0 tx=0x7f3ccc00b3e0 comp rx=0 tx=0).stop 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/2944188182 shutdown_connections 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/2944188182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd410dd40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/2944188182 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4106f30 0x7f3cd4107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/2944188182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4068e10 0x7f3cd4069290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/2944188182 >> 172.21.15.5:0/2944188182 conn(0x7f3cd4077ca0 msgr2=0x7f3cd40780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/2944188182 shutdown_connections 2023-09-23T04:50:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.996+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/2944188182 wait complete. 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.997+0000 7f3cdb9ff700 1 Processor -- start 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.997+0000 7f3cdb9ff700 1 -- start start 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.997+0000 7f3cdb9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4068e10 0x7f3cd4100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.997+0000 7f3cdb9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.997+0000 7f3cdb9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 0x7f3cd41a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cdb9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4100d80 con 0x7f3cd4106f30 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cdb9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41a57d0 con 0x7f3cd40697d0 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cdb9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41a5910 con 0x7f3cd4068e10 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39762/0 (socket says 172.21.15.5:39762) 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/3331306747 learned_addr learned my addr 172.21.15.5:0/3331306747 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 0x7f3cd41a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4068e10 0x7f3cd4100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/3331306747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4068e10 msgr2=0x7f3cd4100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4068e10 0x7f3cd4100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 msgr2=0x7f3cd41a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 0x7f3cd41a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/3331306747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009520 con 0x7f3cd40697d0 2023-09-23T04:50:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.998+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 0x7f3cd41a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400eb30 tx=0x7f3cc400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cca7fc700 1 -- 172.21.15.5:0/3331306747 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4009880 con 0x7f3cd40697d0 2023-09-23T04:50:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cca7fc700 1 -- 172.21.15.5:0/3331306747 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cc40044e0 con 0x7f3cd40697d0 2023-09-23T04:50:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cca7fc700 1 -- 172.21.15.5:0/3331306747 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4010430 con 0x7f3cd40697d0 2023-09-23T04:50:02.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc009d00 con 0x7f3cd40697d0 2023-09-23T04:50:02.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:01.999+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd41a5ec0 con 0x7f3cd40697d0 2023-09-23T04:50:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.002+0000 7f3cca7fc700 1 -- 172.21.15.5:0/3331306747 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3cc4003ee0 con 0x7f3cd40697d0 2023-09-23T04:50:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.004+0000 7f3cca7fc700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 0x7f3cc00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.004+0000 7f3cef2edb80 1 Processor -- start 2023-09-23T04:50:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.004+0000 7f3cca7fc700 1 -- 172.21.15.5:0/3331306747 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3cc4014070 con 0x7f3cd40697d0 2023-09-23T04:50:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.004+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 0x7f3cc00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.004+0000 7f3cef2edb80 1 -- start start 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8400 0x5655148a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565514811460 con 0x5655148a8d20 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x565514802c20 con 0x5655148a8400 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x565514800540 con 0x5655148b3bd0 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40256/0 (socket says 172.21.15.5:40256) 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/913977951 learned_addr learned my addr 172.21.15.5:0/913977951 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 0x7f3cc00a3c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009bd0 tx=0x7f3cd0009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.005+0000 7f3cd979b700 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8400 0x5655148a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 -- 172.21.15.5:0/913977951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8400 msgr2=0x5655148a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8400 0x5655148a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 -- 172.21.15.5:0/913977951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 msgr2=0x5655148b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 -- 172.21.15.5:0/913977951 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551485e8d0 con 0x5655148b3bd0 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b5fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b770 tx=0x7f3ccc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3c2d6d52a44e4be server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cbf7fe700 1 -- 172.21.15.5:0/913977951 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00e040 con 0x5655148b3bd0 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cbf7fe700 1 -- 172.21.15.5:0/913977951 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ccc009b80 con 0x5655148b3bd0 2023-09-23T04:50:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.006+0000 7f3cbf7fe700 1 -- 172.21.15.5:0/913977951 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc01c640 con 0x5655148b3bd0 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 -- 172.21.15.5:0/913977951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 msgr2=0x5655148b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b5fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b770 tx=0x7f3ccc009360 comp rx=0 tx=0).stop 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 -- 172.21.15.5:0/913977951 shutdown_connections 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b5fc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8400 0x5655148a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/913977951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 -- 172.21.15.5:0/913977951 >> 172.21.15.5:0/913977951 conn(0x56551481d260 msgr2=0x5655148a4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 -- 172.21.15.5:0/913977951 shutdown_connections 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.007+0000 7f3cef2edb80 1 -- 172.21.15.5:0/913977951 wait complete. 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.008+0000 7f3cef2edb80 1 -- start start 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.008+0000 7f3cef2edb80 1 Processor -- start 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.008+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.008+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655147fbe30 con 0x5655148a8400 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x565514798710 con 0x5655148a8d20 2023-09-23T04:50:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655147983b0 con 0x5655148b3bd0 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cef2edb80 10 client.0 _ll_register_callbacks cb 0x5655145f5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39794/0 (socket says 172.21.15.5:39794) 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/2886354168 learned_addr learned my addr 172.21.15.5:0/2886354168 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd9f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd979b700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/2886354168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 msgr2=0x5655148b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 msgr2=0x5655148b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.009+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/2886354168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd410e640 con 0x5655148a8d20 2023-09-23T04:50:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4000c00 tx=0x7f3cc409a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d74ef1d1a2afff6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400cd40 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3cc409a730 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5655147f4e10 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc409e880 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cef2edb80 10 client.26546 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565514626de0 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5655147f4e10 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.010+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3cc409a990 con 0x5655148a8d20 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 10 client.26546 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 20 client.26546 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5655148c35c0 0x5655148c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 4 client.26546 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5655147f4b30 con 0x5655148c35c0 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.011+0000 7f3cd979b700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5655148c35c0 0x5655148c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.012+0000 7f3cd979b700 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5655148c35c0 0x5655148c59e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42ad1e17dcadc46 server_cookie=ecffcd4ce803847f in_seq=0 out_seq=0 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.012+0000 7f3cbcff9700 10 client.26546 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.012+0000 7f3cbcff9700 1 -- 172.21.15.5:0/2886354168 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5655147f4b30 con 0x5655148c35c0 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.012+0000 7f3cbcff9700 10 client.26546 handle_command_reply: tid=0 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.012+0000 7f3cef2edb80 1 client.26546 shutdown 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 msgr2=0x5655148b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4000c00 tx=0x7f3cc409a180 comp rx=0 tx=0).stop 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 shutdown_connections 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5655148c35c0 0x5655148c59e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148b3bd0 0x5655148b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/2886354168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148a8d20 0x5655148b07f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cbcff9700 0 client.26546 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 >> 172.21.15.5:0/2886354168 conn(0x56551481d260 msgr2=0x5655148a4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 shutdown_connections 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 1 -- 172.21.15.5:0/2886354168 wait complete. 2023-09-23T04:50:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.013+0000 7f3cef2edb80 20 client.26546 trim_cache size 0 max 16384 2023-09-23T04:50:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 Processor -- start 2023-09-23T04:50:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 -- start start 2023-09-23T04:50:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551467c920 con 0x5655148a8d20 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655148b1400 con 0x5655148b3bd0 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.034+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655148a55b0 con 0x5655148a8400 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58824/0 (socket says 172.21.15.5:58824) 2023-09-23T04:50:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 -- 172.21.15.5:0/3225826572 learned_addr learned my addr 172.21.15.5:0/3225826572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 -- 172.21.15.5:0/3225826572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 msgr2=0x5655148b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 -- 172.21.15.5:0/3225826572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 msgr2=0x5655148b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 -- 172.21.15.5:0/3225826572 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551460fc60 con 0x5655148a8400 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.035+0000 7f3cd979b700 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5655148c62f0 tx=0x7f3cd0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d1323b632d122eb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.036+0000 7f3cbffff700 1 -- 172.21.15.5:0/3225826572 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0024040 con 0x5655148a8400 2023-09-23T04:50:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.036+0000 7f3cbffff700 1 -- 172.21.15.5:0/3225826572 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd0015070 con 0x5655148a8400 2023-09-23T04:50:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.036+0000 7f3cbffff700 1 -- 172.21.15.5:0/3225826572 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd001e7e0 con 0x5655148a8400 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.036+0000 7f3cef2edb80 1 -- 172.21.15.5:0/3225826572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 msgr2=0x5655148ba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.036+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5655148c62f0 tx=0x7f3cd0013000 comp rx=0 tx=0).stop 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 -- 172.21.15.5:0/3225826572 shutdown_connections 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8400 0x5655148ba6b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/3225826572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8d20 0x5655148b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 -- 172.21.15.5:0/3225826572 >> 172.21.15.5:0/3225826572 conn(0x56551481d260 msgr2=0x5655148abaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 -- 172.21.15.5:0/3225826572 shutdown_connections 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 -- 172.21.15.5:0/3225826572 wait complete. 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 -- start start 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.037+0000 7f3cef2edb80 1 Processor -- start 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655148ad740 con 0x5655148a8400 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655148ad8b0 con 0x5655148b3bd0 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655149173f0 con 0x5655148a8d20 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cef2edb80 10 client.0 _ll_register_callbacks cb 0x7f3cc409a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd8f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58826/0 (socket says 172.21.15.5:58826) 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/1152997457 learned_addr learned my addr 172.21.15.5:0/1152997457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd9f9c700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/1152997457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 msgr2=0x5655148ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.038+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 msgr2=0x5655148ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cd8f9a700 1 -- 172.21.15.5:0/1152997457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd0010990 con 0x5655148a8d20 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cd8f9a700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4098a80 tx=0x7f3cc4096400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d345d371cd76cb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40b6030 con 0x5655148a8d20 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3cc40b7070 con 0x5655148a8d20 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56551489c250 con 0x5655148a8d20 2023-09-23T04:50:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc4099c70 con 0x5655148a8d20 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cef2edb80 10 client.26605 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.039+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56551464d250 con 0x5655148a8d20 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56551489c250 con 0x5655148a8d20 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3cc4099690 con 0x5655148a8d20 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 10 client.26605 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 20 client.26605 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x565514918680 0x56551491aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 4 client.26605 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565514647f70 con 0x565514918680 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.040+0000 7f3cd979b700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x565514918680 0x56551491aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cbcff9700 10 client.26605 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cd979b700 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x565514918680 0x56551491aaa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1420e6802776774c server_cookie=eabab025621dde40 in_seq=0 out_seq=0 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cbcff9700 1 -- 172.21.15.5:0/1152997457 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x565514647f70 con 0x565514918680 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cbcff9700 10 client.26605 handle_command_reply: tid=0 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 client.26605 shutdown 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 msgr2=0x5655148acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4098a80 tx=0x7f3cc4096400 comp rx=0 tx=0).stop 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 shutdown_connections 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x565514918680 0x56551491aaa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5655148b3bd0 0x5655148ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5655148a8400 0x5655148ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 --2- 172.21.15.5:0/1152997457 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5655148a8d20 0x5655148acb10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 >> 172.21.15.5:0/1152997457 conn(0x56551481d260 msgr2=0x5655148b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cbcff9700 0 client.26605 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 shutdown_connections 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.041+0000 7f3cef2edb80 1 -- 172.21.15.5:0/1152997457 wait complete. 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cef2edb80 20 client.26605 trim_cache size 0 max 16384 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:50:02.053 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 msgr2=0x7f3cc00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 0x7f3cc00a3c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009bd0 tx=0x7f3cd0009260 comp rx=0 tx=0).stop 2023-09-23T04:50:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 msgr2=0x7f3cd4100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400eb30 tx=0x7f3cc400eef0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 shutdown_connections 2023-09-23T04:50:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3cc00a1790 0x7f3cc00a3c50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4106f30 0x7f3cd41a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3cd4068e10 0x7f3cd4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 --2- 172.21.15.5:0/3331306747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cd40697d0 0x7f3cd4100550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.042+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 >> 172.21.15.5:0/3331306747 conn(0x7f3cd4077ca0 msgr2=0x7f3cd40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.043+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 shutdown_connections 2023-09-23T04:50:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.043+0000 7f3cdb9ff700 1 -- 172.21.15.5:0/3331306747 wait complete. 2023-09-23T04:50:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:01 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:02.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:50:02.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:50:02.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:01 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:02.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 -- 172.21.15.5:0/1884299318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 msgr2=0x7f7c04101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 --2- 172.21.15.5:0/1884299318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c04101490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009990 tx=0x7f7bf4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 -- 172.21.15.5:0/1884299318 shutdown_connections 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 --2- 172.21.15.5:0/1884299318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c041061d0 0x7f7c0410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 --2- 172.21.15.5:0/1884299318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c04101490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 --2- 172.21.15.5:0/1884299318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c04101a60 0x7f7c04105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 -- 172.21.15.5:0/1884299318 >> 172.21.15.5:0/1884299318 conn(0x7f7c040fc930 msgr2=0x7f7c040fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 -- 172.21.15.5:0/1884299318 shutdown_connections 2023-09-23T04:50:02.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.311+0000 7f7c091ff700 1 -- 172.21.15.5:0/1884299318 wait complete. 2023-09-23T04:50:02.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 Processor -- start 2023-09-23T04:50:02.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 -- start start 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c041990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041061d0 0x7f7c04199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419e4e0 con 0x7f7c041061d0 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419e650 con 0x7f7c0419da60 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.313+0000 7f7c091ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419e810 con 0x7f7c041010b0 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c0359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c041990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c0359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39830/0 (socket says 172.21.15.5:39830) 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c0359e700 1 -- 172.21.15.5:0/251225498 learned_addr learned my addr 172.21.15.5:0/251225498 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c0259c700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041061d0 0x7f7c04199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 -- 172.21.15.5:0/251225498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 msgr2=0x7f7c0419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 -- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041061d0 msgr2=0x7f7c04199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041061d0 0x7f7c04199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 -- 172.21.15.5:0/251225498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40095f0 con 0x7f7c041010b0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c0359e700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.314+0000 7f7c02d9d700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c041990c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006010 tx=0x7f7bf4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.315+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf401d070 con 0x7f7c041010b0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.315+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c041a5d10 con 0x7f7c041010b0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.315+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf4003f60 con 0x7f7c041010b0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.315+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4022ba0 con 0x7f7c041010b0 2023-09-23T04:50:02.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.315+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c041a6190 con 0x7f7c041010b0 2023-09-23T04:50:02.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.316+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c04067ca0 con 0x7f7c041010b0 2023-09-23T04:50:02.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.319+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7bf4003aa0 con 0x7f7c041010b0 2023-09-23T04:50:02.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.320+0000 7f7bebfff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 0x7f7bec0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.320+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7bf40ced20 con 0x7f7c041010b0 2023-09-23T04:50:02.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.320+0000 7f7c0259c700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 0x7f7bec0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.321+0000 7f7c0259c700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 0x7f7bec0a3c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c0419a6d0 tx=0x7f7bf80073d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.321+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7bf4097c20 con 0x7f7c041010b0 2023-09-23T04:50:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.622+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c0419a410 con 0x7f7c041010b0 2023-09-23T04:50:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.622+0000 7f7bebfff700 1 -- 172.21.15.5:0/251225498 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11352 (secure 0 0 0) 0x7f7bf409b240 con 0x7f7c041010b0 2023-09-23T04:50:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:50:02.624 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:50:00.308963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:50:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 msgr2=0x7f7bec0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 0x7f7bec0a3c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c0419a6d0 tx=0x7f7bf80073d0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 msgr2=0x7f7c041990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c041990c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006010 tx=0x7f7bf4009580 comp rx=0 tx=0).stop 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 shutdown_connections 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7bec0a1790 0x7f7bec0a3c50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c0419da60 0x7f7c0419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7c041010b0 0x7f7c041990c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 --2- 172.21.15.5:0/251225498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c041061d0 0x7f7c04199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 >> 172.21.15.5:0/251225498 conn(0x7f7c040fc930 msgr2=0x7f7c0410d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 shutdown_connections 2023-09-23T04:50:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.625+0000 7f7c091ff700 1 -- 172.21.15.5:0/251225498 wait complete. 2023-09-23T04:50:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2023-09-23T04:50:02.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:50:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 -- 172.21.15.5:0/3530175917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c073af0 msgr2=0x7fd94c10d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 --2- 172.21.15.5:0/3530175917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c073af0 0x7fd94c10d1f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd9400098f0 tx=0x7fd940009c00 comp rx=0 tx=0).stop 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 -- 172.21.15.5:0/3530175917 shutdown_connections 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 --2- 172.21.15.5:0/3530175917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c10d730 0x7fd94c10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 --2- 172.21.15.5:0/3530175917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c0731d0 0x7fd94c0735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 --2- 172.21.15.5:0/3530175917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c073af0 0x7fd94c10d1f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 -- 172.21.15.5:0/3530175917 >> 172.21.15.5:0/3530175917 conn(0x7fd94c0fc980 msgr2=0x7fd94c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 -- 172.21.15.5:0/3530175917 shutdown_connections 2023-09-23T04:50:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.889+0000 7fd9509ff700 1 -- 172.21.15.5:0/3530175917 wait complete. 2023-09-23T04:50:02.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.891+0000 7fd9509ff700 1 Processor -- start 2023-09-23T04:50:02.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.891+0000 7fd9509ff700 1 -- start start 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.891+0000 7fd9509ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.891+0000 7fd9509ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 0x7fd94c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd9509ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 0x7fd94c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd9509ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c19e210 con 0x7fd94c0731d0 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd9509ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c19e380 con 0x7fd94c10d730 2023-09-23T04:50:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd9509ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c19e540 con 0x7fd94c073af0 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40330/0 (socket says 172.21.15.5:40330) 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94a59c700 1 -- 172.21.15.5:0/4086910309 learned_addr learned my addr 172.21.15.5:0/4086910309 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 0x7fd94c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 0x7fd94c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 -- 172.21.15.5:0/4086910309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 msgr2=0x7fd94c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 0x7fd94c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 -- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 msgr2=0x7fd94c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd949d9b700 1 -- 172.21.15.5:0/4086910309 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944009d00 con 0x7fd94c073af0 2023-09-23T04:50:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 0x7fd94c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.892+0000 7fd94a59c700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 0x7fd94c199770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd940009810 tx=0x7fd94000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd93b7fe700 1 -- 172.21.15.5:0/4086910309 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94001c070 con 0x7fd94c073af0 2023-09-23T04:50:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd93b7fe700 1 -- 172.21.15.5:0/4086910309 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94000bba0 con 0x7fd94c073af0 2023-09-23T04:50:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940009520 con 0x7fd94c073af0 2023-09-23T04:50:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd93b7fe700 1 -- 172.21.15.5:0/4086910309 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940003970 con 0x7fd94c073af0 2023-09-23T04:50:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.893+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c1a5e80 con 0x7fd94c073af0 2023-09-23T04:50:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.897+0000 7fd93b7fe700 1 -- 172.21.15.5:0/4086910309 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd940020030 con 0x7fd94c073af0 2023-09-23T04:50:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 Processor -- start 2023-09-23T04:50:02.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 -- start start 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce0640460 con 0x558ce06d7d20 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce0631c20 con 0x558ce06d7400 2023-09-23T04:50:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd93b7fe700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 0x7fd9340a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.899+0000 7fd96427ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce062f540 con 0x558ce06e2bd0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd93b7fe700 1 -- 172.21.15.5:0/4086910309 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd9400ce4a0 con 0x7fd94c073af0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39852/0 (socket says 172.21.15.5:39852) 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 -- 172.21.15.5:0/3446622849 learned_addr learned my addr 172.21.15.5:0/3446622849 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd949d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94a59c700 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 0x7fd9340a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 -- 172.21.15.5:0/3446622849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 msgr2=0x558ce06e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 -- 172.21.15.5:0/3446622849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 msgr2=0x558ce06e2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06e2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd94ad9d700 1 -- 172.21.15.5:0/3446622849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce068d8d0 con 0x558ce06d7400 2023-09-23T04:50:02.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.900+0000 7fd949d9b700 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.901+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd9440085d0 tx=0x7fd9440088e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c6af054effc292 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.901+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 0x7fd9340a3c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd944008c30 tx=0x7fd94401e000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.901+0000 7fd938ff9700 1 -- 172.21.15.5:0/3446622849 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944027070 con 0x558ce06d7400 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.901+0000 7fd938ff9700 1 -- 172.21.15.5:0/3446622849 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94400f040 con 0x558ce06d7400 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.901+0000 7fd938ff9700 1 -- 172.21.15.5:0/3446622849 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94400e020 con 0x558ce06d7400 2023-09-23T04:50:02.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 -- 172.21.15.5:0/3446622849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 msgr2=0x558ce06d77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd9440085d0 tx=0x7fd9440088e0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 -- 172.21.15.5:0/3446622849 shutdown_connections 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7400 0x558ce06d77e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 --2- 172.21.15.5:0/3446622849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 -- 172.21.15.5:0/3446622849 >> 172.21.15.5:0/3446622849 conn(0x558ce064c260 msgr2=0x558ce06d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 -- 172.21.15.5:0/3446622849 shutdown_connections 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.902+0000 7fd96427ab80 1 -- 172.21.15.5:0/3446622849 wait complete. 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.903+0000 7fd96427ab80 1 -- start start 2023-09-23T04:50:02.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.903+0000 7fd96427ab80 1 Processor -- start 2023-09-23T04:50:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:02 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/251225498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:02 smithi005 ceph-mon[97638]: pgmap v116: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce062ae30 con 0x558ce06d7d20 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce05c7710 con 0x558ce06e2bd0 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce05c73b0 con 0x558ce06d7400 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd96427ab80 10 client.0 _ll_register_callbacks cb 0x558ce0424800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58858/0 (socket says 172.21.15.5:58858) 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd94a59c700 1 -- 172.21.15.5:0/4277250152 learned_addr learned my addr 172.21.15.5:0/4277250152 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.904+0000 7fd949d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 -- 172.21.15.5:0/4277250152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 msgr2=0x558ce06e03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06e03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 -- 172.21.15.5:0/4277250152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 msgr2=0x558ce06dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 -- 172.21.15.5:0/4277250152 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c1a5e80 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd94a59c700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd93c011ca0 tx=0x7fd93c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca20de793f5ca07 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd931ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.905+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c016070 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ce06cb250 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd93c00fbc0 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93c00fd80 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 10 client.26623 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd94400ebe0 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ce06cb250 con 0x558ce06d7400 2023-09-23T04:50:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd93c003d00 con 0x558ce06d7400 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 10 client.26623 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 10 client.26623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 20 client.26623 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce06fadc0 0x558ce06fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 4 client.26623 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.906+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ce0623e10 con 0x558ce06fadc0 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.907+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce06fadc0 0x558ce06fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.907+0000 7fd949d9b700 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce06fadc0 0x558ce06fd1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2808d14c15cf94d server_cookie=c2ed0158a0360584 in_seq=0 out_seq=0 2023-09-23T04:50:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.907+0000 7fd931ffb700 10 client.26623 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd931ffb700 1 -- 172.21.15.5:0/4277250152 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ce0623e10 con 0x558ce06fadc0 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd931ffb700 10 client.26623 handle_command_reply: tid=0 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 client.26623 shutdown 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 msgr2=0x558ce06df930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd93c011ca0 tx=0x7fd93c00f3c0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 shutdown_connections 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06e03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7400 0x558ce06df930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce06fadc0 0x558ce06fd1e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 --2- 172.21.15.5:0/4277250152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7d20 0x558ce06dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd931ffb700 0 client.26623 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 >> 172.21.15.5:0/4277250152 conn(0x558ce064c260 msgr2=0x558ce06d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 shutdown_connections 2023-09-23T04:50:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.908+0000 7fd96427ab80 1 -- 172.21.15.5:0/4277250152 wait complete. 2023-09-23T04:50:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.909+0000 7fd96427ab80 20 client.26623 trim_cache size 0 max 16384 2023-09-23T04:50:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.924+0000 7fd96427ab80 1 Processor -- start 2023-09-23T04:50:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.924+0000 7fd96427ab80 1 -- start start 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06d99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06dcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce04ab920 con 0x558ce06d7400 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce06d5080 con 0x558ce06d7d20 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd96427ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce06d7a30 con 0x558ce06e2bd0 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd949d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06d99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd949d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39868/0 (socket says 172.21.15.5:39868) 2023-09-23T04:50:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd949d9b700 1 -- 172.21.15.5:0/404357335 learned_addr learned my addr 172.21.15.5:0/404357335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06dcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 -- 172.21.15.5:0/404357335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 msgr2=0x558ce06dcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06dcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 -- 172.21.15.5:0/404357335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 msgr2=0x558ce06e01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94a59c700 1 -- 172.21.15.5:0/404357335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce0455de0 con 0x558ce06d7400 2023-09-23T04:50:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06dcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.925+0000 7fd949d9b700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.926+0000 7fd94a59c700 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06d99a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd93c011c70 tx=0x7fd93c019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2462707063fe78fe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.926+0000 7fd9397fa700 1 -- 172.21.15.5:0/404357335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c016030 con 0x558ce06d7400 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.926+0000 7fd9397fa700 1 -- 172.21.15.5:0/404357335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c019e00 con 0x558ce06d7400 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.926+0000 7fd9397fa700 1 -- 172.21.15.5:0/404357335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c005640 con 0x558ce06d7400 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- 172.21.15.5:0/404357335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 msgr2=0x558ce06d99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06d99a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd93c011c70 tx=0x7fd93c019530 comp rx=0 tx=0).stop 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- 172.21.15.5:0/404357335 shutdown_connections 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06e2bd0 0x558ce06dcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06d99a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 --2- 172.21.15.5:0/404357335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06d7d20 0x558ce06e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- 172.21.15.5:0/404357335 >> 172.21.15.5:0/404357335 conn(0x558ce064c260 msgr2=0x558ce06d9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- 172.21.15.5:0/404357335 shutdown_connections 2023-09-23T04:50:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- 172.21.15.5:0/404357335 wait complete. 2023-09-23T04:50:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.927+0000 7fd96427ab80 1 -- start start 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 Processor -- start 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 0x558ce06eff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce06f0a10 con 0x558ce06d7400 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce06f0b80 con 0x558ce06e2bd0 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce06f0d40 con 0x558ce06d7d20 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd96427ab80 10 client.0 _ll_register_callbacks cb 0x7fd93c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd94ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd949d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 0x558ce06eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd94a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd94ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39882/0 (socket says 172.21.15.5:39882) 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.928+0000 7fd94ad9d700 1 -- 172.21.15.5:0/2421897753 learned_addr learned my addr 172.21.15.5:0/2421897753 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 -- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 msgr2=0x558ce06eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 0x558ce06eff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 -- 172.21.15.5:0/2421897753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 msgr2=0x558ce06f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 -- 172.21.15.5:0/2421897753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940009520 con 0x558ce06e2bd0 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd949d9b700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 0x558ce06eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:02.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd94ad9d700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd9440085d0 tx=0x7fd944023420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f65df13349be52f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd931ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944027050 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.929+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd94400f040 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd9440092d0 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd94400e040 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 10 client.26573 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd94400ebe0 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94400f040 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd94402e410 con 0x558ce06e2bd0 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 10 client.26573 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 10 client.26573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:50:02.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 20 client.26573 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce0750080 0x558ce07524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 4 client.26573 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.930+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ce0623b30 con 0x558ce0750080 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd94a59c700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce0750080 0x558ce07524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd931ffb700 10 client.26573 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd94a59c700 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce0750080 0x558ce07524a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f367c796f725c275 server_cookie=14f0f158d926a5dc in_seq=0 out_seq=0 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd931ffb700 1 -- 172.21.15.5:0/2421897753 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558ce0623b30 con 0x558ce0750080 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd931ffb700 10 client.26573 handle_command_reply: tid=0 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 client.26573 shutdown 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 msgr2=0x558ce06f04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd9440085d0 tx=0x7fd944023420 comp rx=0 tx=0).stop 2023-09-23T04:50:02.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 shutdown_connections 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce06e2bd0 0x558ce06f04d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.931+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x558ce0750080 0x558ce07524a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce06d7400 0x558ce06f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 1 --2- 172.21.15.5:0/2421897753 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce06d7d20 0x558ce06eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd931ffb700 0 client.26573 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 >> 172.21.15.5:0/2421897753 conn(0x558ce064c260 msgr2=0x558ce06d9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 shutdown_connections 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 1 -- 172.21.15.5:0/2421897753 wait complete. 2023-09-23T04:50:02.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd96427ab80 20 client.26573 trim_cache size 0 max 16384 2023-09-23T04:50:02.939 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:02.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 msgr2=0x7fd9340a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 0x7fd9340a3c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd944008c30 tx=0x7fd94401e000 comp rx=0 tx=0).stop 2023-09-23T04:50:02.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 msgr2=0x7fd94c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.932+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 0x7fd94c199770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd940009810 tx=0x7fd94000fa00 comp rx=0 tx=0).stop 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 shutdown_connections 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9340a1790 0x7fd9340a3c50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd94c10d730 0x7fd94c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd94c0731d0 0x7fd94c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 --2- 172.21.15.5:0/4086910309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd94c073af0 0x7fd94c199770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 >> 172.21.15.5:0/4086910309 conn(0x7fd94c0fc980 msgr2=0x7fd94c10f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 shutdown_connections 2023-09-23T04:50:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:02.933+0000 7fd9509ff700 1 -- 172.21.15.5:0/4086910309 wait complete. 2023-09-23T04:50:02.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 238928 7.97 MB/sec execute 38 sec latency 19.674 ms 2023-09-23T04:50:02.954 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:50:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:02 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/251225498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:02.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:02 smithi069 ceph-mon[133931]: pgmap v116: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2023-09-23T04:50:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:02 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/251225498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:02 smithi118 ceph-mon[134297]: pgmap v116: 129 pgs: 129 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 256 op/s 2023-09-23T04:50:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 -- 172.21.15.5:0/1969432385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0200ff910 msgr2=0x7fa0200ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 --2- 172.21.15.5:0/1969432385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0200ff910 0x7fa0200ffcf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa010009990 tx=0x7fa010009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 -- 172.21.15.5:0/1969432385 shutdown_connections 2023-09-23T04:50:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 --2- 172.21.15.5:0/1969432385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa020106250 0x7fa02010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 --2- 172.21.15.5:0/1969432385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0200ff910 0x7fa0200ffcf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 --2- 172.21.15.5:0/1969432385 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0201002c0 0x7fa020105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 -- 172.21.15.5:0/1969432385 >> 172.21.15.5:0/1969432385 conn(0x7fa020075070 msgr2=0x7fa020075480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 -- 172.21.15.5:0/1969432385 shutdown_connections 2023-09-23T04:50:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.193+0000 7fa0259ff700 1 -- 172.21.15.5:0/1969432385 wait complete. 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.194+0000 7fa0259ff700 1 Processor -- start 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.195+0000 7fa0259ff700 1 -- start start 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.195+0000 7fa0259ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.195+0000 7fa0259ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa020106250 0x7fa020100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.195+0000 7fa0259ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 0x7fa0201a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa0259ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201a5940 con 0x7fa020101300 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa0259ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201a5ab0 con 0x7fa020106250 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa0259ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201a5c70 con 0x7fa0200ff910 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58888/0 (socket says 172.21.15.5:58888) 2023-09-23T04:50:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 -- 172.21.15.5:0/75164281 learned_addr learned my addr 172.21.15.5:0/75164281 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 0x7fa0201a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 -- 172.21.15.5:0/75164281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa020106250 msgr2=0x7fa020100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa020106250 0x7fa020100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 -- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 msgr2=0x7fa0201a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 0x7fa0201a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 -- 172.21.15.5:0/75164281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0100095f0 con 0x7fa0200ff910 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 0x7fa0201a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.196+0000 7fa01effd700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa010005910 tx=0x7fa0100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.197+0000 7fa0249fd700 1 -- 172.21.15.5:0/75164281 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01001d070 con 0x7fa0200ff910 2023-09-23T04:50:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.197+0000 7fa0249fd700 1 -- 172.21.15.5:0/75164281 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa010022470 con 0x7fa0200ff910 2023-09-23T04:50:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.197+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201a5ef0 con 0x7fa0200ff910 2023-09-23T04:50:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.197+0000 7fa0249fd700 1 -- 172.21.15.5:0/75164281 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010017650 con 0x7fa0200ff910 2023-09-23T04:50:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.197+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201a62b0 con 0x7fa0200ff910 2023-09-23T04:50:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.200+0000 7fa0249fd700 1 -- 172.21.15.5:0/75164281 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa0100225e0 con 0x7fa0200ff910 2023-09-23T04:50:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.202+0000 7fa0393a6b80 1 Processor -- start 2023-09-23T04:50:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.202+0000 7fa0249fd700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 0x7fa0080a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.202+0000 7fa0249fd700 1 -- 172.21.15.5:0/75164281 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa0100ceac0 con 0x7fa0200ff910 2023-09-23T04:50:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.202+0000 7fa0393a6b80 1 -- start start 2023-09-23T04:50:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.202+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 0x7fa0080a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268d20 0x55d250273690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250273bd0 0x55d250275fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2501d1460 con 0x55d250268400 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2501c2c20 con 0x55d250268d20 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2501c0540 con 0x55d250273bd0 2023-09-23T04:50:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40402/0 (socket says 172.21.15.5:40402) 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 -- 172.21.15.5:0/1733743227 learned_addr learned my addr 172.21.15.5:0/1733743227 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268d20 0x55d250273690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250273bd0 0x55d250275fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 -- 172.21.15.5:0/1733743227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250273bd0 msgr2=0x55d250275fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250273bd0 0x55d250275fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 -- 172.21.15.5:0/1733743227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268d20 msgr2=0x55d250273690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268d20 0x55d250273690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01effd700 1 -- 172.21.15.5:0/1733743227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25021e8d0 con 0x55d250268400 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.203+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 0x7fa0080a3bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa018009fd0 tx=0x7fa01800b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.204+0000 7fa01effd700 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa0100a3450 tx=0x7fa010097af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800c946f73b3673 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.204+0000 7fa005ffb700 1 -- 172.21.15.5:0/1733743227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01009bd80 con 0x55d250268400 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.204+0000 7fa005ffb700 1 -- 172.21.15.5:0/1733743227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01009b680 con 0x55d250268400 2023-09-23T04:50:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.204+0000 7fa005ffb700 1 -- 172.21.15.5:0/1733743227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01009b8e0 con 0x55d250268400 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- 172.21.15.5:0/1733743227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 msgr2=0x55d2502687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa0100a3450 tx=0x7fa010097af0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- 172.21.15.5:0/1733743227 shutdown_connections 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250273bd0 0x55d250275fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268400 0x55d2502687e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/1733743227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268d20 0x55d250273690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- 172.21.15.5:0/1733743227 >> 172.21.15.5:0/1733743227 conn(0x55d2501dd260 msgr2=0x55d250264790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- 172.21.15.5:0/1733743227 shutdown_connections 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- 172.21.15.5:0/1733743227 wait complete. 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.205+0000 7fa0393a6b80 1 -- start start 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 Processor -- start 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268400 0x55d250273fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250273bd0 0x55d250274a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2501bbe30 con 0x55d250273bd0 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d250158710 con 0x55d250268400 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2501583b0 con 0x55d250268d20 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.206+0000 7fa0393a6b80 10 client.0 _ll_register_callbacks cb 0x55d24ffb5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58898/0 (socket says 172.21.15.5:58898) 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 -- 172.21.15.5:0/2425756698 learned_addr learned my addr 172.21.15.5:0/2425756698 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250273bd0 0x55d250274a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01effd700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268400 0x55d250273fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 -- 172.21.15.5:0/2425756698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268400 msgr2=0x55d250273fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268400 0x55d250273fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 -- 172.21.15.5:0/2425756698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250273bd0 msgr2=0x55d250274a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250273bd0 0x55d250274a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 -- 172.21.15.5:0/2425756698 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201a62b0 con 0x55d250268d20 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa014011490 tx=0x7fa0140117a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe5155daa275b94f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.207+0000 7f9ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014004520 con 0x55d250268d20 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa014004830 con 0x55d250268d20 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d25025c250 con 0x55d250268d20 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0140049f0 con 0x55d250268d20 2023-09-23T04:50:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7fa0393a6b80 10 client.26641 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d24ffe6de0 con 0x55d250268d20 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d25025c250 con 0x55d250268d20 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.208+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0140197c0 con 0x55d250268d20 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 10 client.26641 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 10 client.26641 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 20 client.26641 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502833b0 0x55d2502857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 4 client.26641 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2501b4e10 con 0x55d2502833b0 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa01effd700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502833b0 0x55d2502857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7fa01effd700 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502833b0 0x55d2502857d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be356d2bd251e304 server_cookie=6616110ccd1c0412 in_seq=0 out_seq=0 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.209+0000 7f9ffeffd700 10 client.26641 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2425756698 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d2501b4e10 con 0x55d2502833b0 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7f9ffeffd700 10 client.26641 handle_command_reply: tid=0 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 client.26641 shutdown 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 msgr2=0x55d2502744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa014011490 tx=0x7fa0140117a0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 shutdown_connections 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502833b0 0x55d2502857d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250273bd0 0x55d250274a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250268400 0x55d250273fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7f9ffeffd700 0 client.26641 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2425756698 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268d20 0x55d2502744f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.210+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 >> 172.21.15.5:0/2425756698 conn(0x55d2501dd260 msgr2=0x55d250264320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.211+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 shutdown_connections 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.211+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2425756698 wait complete. 2023-09-23T04:50:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.211+0000 7fa0393a6b80 20 client.26641 trim_cache size 0 max 16384 2023-09-23T04:50:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.225+0000 7fa0393a6b80 1 Processor -- start 2023-09-23T04:50:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 -- start start 2023-09-23T04:50:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d25026dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d25026e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25003c920 con 0x55d250268d20 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25027f100 con 0x55d250273bd0 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2502732d0 con 0x55d250268400 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.226+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58910/0 (socket says 172.21.15.5:58910) 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01effd700 1 -- 172.21.15.5:0/3968204623 learned_addr learned my addr 172.21.15.5:0/3968204623 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d25026e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d25026dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 -- 172.21.15.5:0/3968204623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 msgr2=0x55d250277fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 -- 172.21.15.5:0/3968204623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 msgr2=0x55d25026dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d25026dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01f7fe700 1 -- 172.21.15.5:0/3968204623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d250260480 con 0x55d250273bd0 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01effd700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.227+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d25026dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.228+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d25026e4f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa018009df0 tx=0x7fa01800d050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f207ca28f2b73540 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.228+0000 7fa0067fc700 1 -- 172.21.15.5:0/3968204623 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01801e070 con 0x55d250273bd0 2023-09-23T04:50:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.228+0000 7fa0067fc700 1 -- 172.21.15.5:0/3968204623 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01801f040 con 0x55d250273bd0 2023-09-23T04:50:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.228+0000 7fa0067fc700 1 -- 172.21.15.5:0/3968204623 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018024aa0 con 0x55d250273bd0 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- 172.21.15.5:0/3968204623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 msgr2=0x55d25026e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d25026e4f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa018009df0 tx=0x7fa01800d050 comp rx=0 tx=0).stop 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- 172.21.15.5:0/3968204623 shutdown_connections 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d25026e4f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/3968204623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d25026dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- 172.21.15.5:0/3968204623 >> 172.21.15.5:0/3968204623 conn(0x55d2501dd260 msgr2=0x55d25027f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- 172.21.15.5:0/3968204623 shutdown_connections 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- 172.21.15.5:0/3968204623 wait complete. 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 -- start start 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.229+0000 7fa0393a6b80 1 Processor -- start 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d250276020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d250276560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d250276af0 con 0x55d250268d20 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d250276c30 con 0x55d250273bd0 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d250276df0 con 0x55d250268400 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa0393a6b80 10 client.0 _ll_register_callbacks cb 0x7fa014009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa01effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58924/0 (socket says 172.21.15.5:58924) 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa01effd700 1 -- 172.21.15.5:0/2976577301 learned_addr learned my addr 172.21.15.5:0/2976577301 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa01f7fe700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d250276560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.230+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d250276020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 -- 172.21.15.5:0/2976577301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 msgr2=0x55d250276560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d250276560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 -- 172.21.15.5:0/2976577301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 msgr2=0x55d250276020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d250276020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 -- 172.21.15.5:0/2976577301 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d25021e8d0 con 0x55d250268400 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa01effd700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0100999d0 tx=0x7fa01009b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dccca88e8d6d5950 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7f9ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01009bd80 con 0x55d250268400 2023-09-23T04:50:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa01009f7d0 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa01009b680 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01009ab70 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa0393a6b80 10 client.26647 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.231+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d24ffe6de0 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01009b680 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa0100a34f0 con 0x55d250268400 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 10 client.26647 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 10 client.26647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 20 client.26647 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502d8350 0x55d2502da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 4 client.26647 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d2501b4b30 con 0x55d2502d8350 2023-09-23T04:50:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502d8350 0x55d2502da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7fa01e7fc700 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502d8350 0x55d2502da740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99e90340ba079d35 server_cookie=bb0693e46d977a95 in_seq=0 out_seq=0 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.232+0000 7f9ffeffd700 10 client.26647 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7f9ffeffd700 1 -- 172.21.15.5:0/2976577301 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d2501b4b30 con 0x55d2502d8350 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7f9ffeffd700 10 client.26647 handle_command_reply: tid=0 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 client.26647 shutdown 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 msgr2=0x55d250277ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0100999d0 tx=0x7fa01009b580 comp rx=0 tx=0).stop 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 shutdown_connections 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d250273bd0 0x55d250276560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d250268400 0x55d250277ce0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55d2502d8350 0x55d2502da740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 --2- 172.21.15.5:0/2976577301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d250268d20 0x55d250276020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7f9ffeffd700 0 client.26647 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 >> 172.21.15.5:0/2976577301 conn(0x55d2501dd260 msgr2=0x55d25026dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 shutdown_connections 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 1 -- 172.21.15.5:0/2976577301 wait complete. 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.233+0000 7fa0393a6b80 20 client.26647 trim_cache size 0 max 16384 2023-09-23T04:50:03.239 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 msgr2=0x7fa0080a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 0x7fa0080a3bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa018009fd0 tx=0x7fa01800b040 comp rx=0 tx=0).stop 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 msgr2=0x7fa020100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa010005910 tx=0x7fa0100047c0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 shutdown_connections 2023-09-23T04:50:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0080a1710 0x7fa0080a3bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa020101300 0x7fa0201a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa0200ff910 0x7fa020100880 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 --2- 172.21.15.5:0/75164281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa020106250 0x7fa020100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 >> 172.21.15.5:0/75164281 conn(0x7fa020075070 msgr2=0x7fa0200fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 shutdown_connections 2023-09-23T04:50:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.234+0000 7fa0259ff700 1 -- 172.21.15.5:0/75164281 wait complete. 2023-09-23T04:50:03.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:50:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.482+0000 7fd03a99d700 1 -- 172.21.15.5:0/1898645179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03410d470 msgr2=0x7fd03410f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.482+0000 7fd03a99d700 1 --2- 172.21.15.5:0/1898645179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03410d470 0x7fd03410f8a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009f90 tx=0x7fd02c00b330 comp rx=0 tx=0).stop 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 -- 172.21.15.5:0/1898645179 shutdown_connections 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 --2- 172.21.15.5:0/1898645179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03410d470 0x7fd03410f8a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 --2- 172.21.15.5:0/1898645179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd0340731f0 0x7fd0340735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 --2- 172.21.15.5:0/1898645179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd03410cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 -- 172.21.15.5:0/1898645179 >> 172.21.15.5:0/1898645179 conn(0x7fd0340fc680 msgr2=0x7fd0340feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 -- 172.21.15.5:0/1898645179 shutdown_connections 2023-09-23T04:50:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.483+0000 7fd03a99d700 1 -- 172.21.15.5:0/1898645179 wait complete. 2023-09-23T04:50:03.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.484+0000 7fd03a99d700 1 Processor -- start 2023-09-23T04:50:03.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.484+0000 7fd03a99d700 1 -- start start 2023-09-23T04:50:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 0x7fd03419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419e060 con 0x7fd0340731f0 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419e1d0 con 0x7fd034073b10 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a99d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419e390 con 0x7fd03410d470 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03919a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03999b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03919a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39950/0 (socket says 172.21.15.5:39950) 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03999b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40442/0 (socket says 172.21.15.5:40442) 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03919a700 1 -- 172.21.15.5:0/3508702726 learned_addr learned my addr 172.21.15.5:0/3508702726 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.485+0000 7fd03a19c700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 0x7fd03419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 -- 172.21.15.5:0/3508702726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 msgr2=0x7fd03419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 0x7fd03419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 -- 172.21.15.5:0/3508702726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 msgr2=0x7fd0341995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 -- 172.21.15.5:0/3508702726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0300095f0 con 0x7fd0340731f0 2023-09-23T04:50:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03919a700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03a19c700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 0x7fd03419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03999b700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd03000eb90 tx=0x7fd03000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd02affd700 1 -- 172.21.15.5:0/3508702726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030009970 con 0x7fd0340731f0 2023-09-23T04:50:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c009c40 con 0x7fd0340731f0 2023-09-23T04:50:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd02affd700 1 -- 172.21.15.5:0/3508702726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0300044e0 con 0x7fd0340731f0 2023-09-23T04:50:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd02affd700 1 -- 172.21.15.5:0/3508702726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030003e50 con 0x7fd0340731f0 2023-09-23T04:50:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.486+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341a5cd0 con 0x7fd0340731f0 2023-09-23T04:50:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.490+0000 7fd02affd700 1 -- 172.21.15.5:0/3508702726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd030004000 con 0x7fd0340731f0 2023-09-23T04:50:03.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.491+0000 7fd04f560b80 1 Processor -- start 2023-09-23T04:50:03.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 -- start start 2023-09-23T04:50:03.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd02affd700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 0x7fd0200a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd02affd700 1 -- 172.21.15.5:0/3508702726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd030014070 con 0x7fd0340731f0 2023-09-23T04:50:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b257800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 0x56238b25a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b1b5460 con 0x56238b24ce90 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b1a6c20 con 0x56238b257d40 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd04f560b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b1a4540 con 0x56238b24c570 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd03a19c700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 0x7fd0200a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd03999b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd03999b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd03999b700 1 -- 172.21.15.5:0/684290715 learned_addr learned my addr 172.21.15.5:0/684290715 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.492+0000 7fd03a19c700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 0x56238b25a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03919a700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b257800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 -- 172.21.15.5:0/684290715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 msgr2=0x56238b25a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 0x56238b25a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 -- 172.21.15.5:0/684290715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 msgr2=0x56238b257800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b257800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 -- 172.21.15.5:0/684290715 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56238b2028d0 con 0x56238b24c570 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03a19c700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 0x56238b25a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03919a700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b257800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03999b700 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a2580 tx=0x7fd03009a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d30dcce111ef5cb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd03a19c700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 0x7fd0200a3bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56238b24e000 tx=0x7fd02c0078d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd01ffff700 1 -- 172.21.15.5:0/684290715 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000ce20 con 0x56238b24c570 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd01ffff700 1 -- 172.21.15.5:0/684290715 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03009ae20 con 0x56238b24c570 2023-09-23T04:50:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.493+0000 7fd01ffff700 1 -- 172.21.15.5:0/684290715 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300ba070 con 0x56238b24c570 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.494+0000 7fd04f560b80 1 -- 172.21.15.5:0/684290715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 msgr2=0x56238b24c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.494+0000 7fd04f560b80 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a2580 tx=0x7fd03009a570 comp rx=0 tx=0).stop 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 -- 172.21.15.5:0/684290715 shutdown_connections 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b257d40 0x56238b25a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24c570 0x56238b24c950 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 --2- 172.21.15.5:0/684290715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b257800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 -- 172.21.15.5:0/684290715 >> 172.21.15.5:0/684290715 conn(0x56238b1c1260 msgr2=0x56238b2489b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 -- 172.21.15.5:0/684290715 shutdown_connections 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 -- 172.21.15.5:0/684290715 wait complete. 2023-09-23T04:50:03.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 -- start start 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.495+0000 7fd04f560b80 1 Processor -- start 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24c570 0x56238b257770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24ce90 0x56238b2549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b19fe30 con 0x56238b24c570 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b13c710 con 0x56238b24ce90 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b13c3b0 con 0x56238b257d40 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd04f560b80 10 client.0 _ll_register_callbacks cb 0x56238af99800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd03999b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24c570 0x56238b257770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd03a19c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd03a19c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58962/0 (socket says 172.21.15.5:58962) 2023-09-23T04:50:03.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd03a19c700 1 -- 172.21.15.5:0/1918577799 learned_addr learned my addr 172.21.15.5:0/1918577799 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.496+0000 7fd03919a700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24ce90 0x56238b2549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 -- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24ce90 msgr2=0x56238b2549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24ce90 0x56238b2549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 -- 172.21.15.5:0/1918577799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24c570 msgr2=0x56238b257770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24c570 0x56238b257770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 -- 172.21.15.5:0/1918577799 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341a5cd0 con 0x56238b257d40 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd02c015000 tx=0x7fd02c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e864d0a2c69a79 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd01d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.497+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c027070 con 0x56238b257d40 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56238b2404a0 con 0x56238b257d40 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd02c00bc00 con 0x56238b257d40 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02c00e040 con 0x56238b257d40 2023-09-23T04:50:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 10 client.26659 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56238afcade0 con 0x56238b257d40 2023-09-23T04:50:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56238b2404a0 con 0x56238b257d40 2023-09-23T04:50:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd02c01e460 con 0x56238b257d40 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 10 client.26659 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 10 client.26659 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 20 client.26659 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b267830 0x56238b269c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 4 client.26659 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:03.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.498+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56238b198e10 con 0x56238b267830 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.499+0000 7fd03999b700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b267830 0x56238b269c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.499+0000 7fd03999b700 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b267830 0x56238b269c50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d01991c73d0999d server_cookie=8c5fc50615836144 in_seq=0 out_seq=0 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.499+0000 7fd01d7fa700 10 client.26659 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1918577799 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56238b198e10 con 0x56238b267830 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd01d7fa700 10 client.26659 handle_command_reply: tid=0 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 client.26659 shutdown 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 msgr2=0x56238b255130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd02c015000 tx=0x7fd02c009920 comp rx=0 tx=0).stop 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 shutdown_connections 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b267830 0x56238b269c50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b255130 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24c570 0x56238b257770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd01d7fa700 0 client.26659 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1918577799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24ce90 0x56238b2549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 >> 172.21.15.5:0/1918577799 conn(0x56238b1c1260 msgr2=0x56238b248540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 shutdown_connections 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.500+0000 7fd04f560b80 1 -- 172.21.15.5:0/1918577799 wait complete. 2023-09-23T04:50:03.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.501+0000 7fd04f560b80 20 client.26659 trim_cache size 0 max 16384 2023-09-23T04:50:03.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 Processor -- start 2023-09-23T04:50:03.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 -- start start 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 0x56238b24ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 0x56238b2541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b020920 con 0x56238b257d40 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b242a50 con 0x56238b24c570 2023-09-23T04:50:03.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.517+0000 7fd04f560b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b240010 con 0x56238b24ce90 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03a19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 0x56238b2541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03a19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 0x56238b2541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40466/0 (socket says 172.21.15.5:40466) 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:39984/0 (socket says 172.21.15.5:39984) 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03a19c700 1 -- 172.21.15.5:0/1364839390 learned_addr learned my addr 172.21.15.5:0/1364839390 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03919a700 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 0x56238b24ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 -- 172.21.15.5:0/1364839390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 msgr2=0x56238b24ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 0x56238b24ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 -- 172.21.15.5:0/1364839390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 msgr2=0x56238b2541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 0x56238b2541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 -- 172.21.15.5:0/1364839390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56238afb3c60 con 0x56238b24c570 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03919a700 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 0x56238b24ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.518+0000 7fd03999b700 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a2580 tx=0x7fd03009af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3af5bde8cb61ee8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.519+0000 7fd028bff700 1 -- 172.21.15.5:0/1364839390 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300b6070 con 0x56238b24c570 2023-09-23T04:50:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.519+0000 7fd028bff700 1 -- 172.21.15.5:0/1364839390 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03009e870 con 0x56238b24c570 2023-09-23T04:50:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.519+0000 7fd028bff700 1 -- 172.21.15.5:0/1364839390 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300ba040 con 0x56238b24c570 2023-09-23T04:50:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.519+0000 7fd04f560b80 1 -- 172.21.15.5:0/1364839390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 msgr2=0x56238b25e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.519+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a2580 tx=0x7fd03009af90 comp rx=0 tx=0).stop 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 -- 172.21.15.5:0/1364839390 shutdown_connections 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b257d40 0x56238b2541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25e8f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1364839390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b24ce90 0x56238b24ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 -- 172.21.15.5:0/1364839390 >> 172.21.15.5:0/1364839390 conn(0x56238b1c1260 msgr2=0x56238b24e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 -- 172.21.15.5:0/1364839390 shutdown_connections 2023-09-23T04:50:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 -- 172.21.15.5:0/1364839390 wait complete. 2023-09-23T04:50:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 -- start start 2023-09-23T04:50:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.520+0000 7fd04f560b80 1 Processor -- start 2023-09-23T04:50:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b25d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b2b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b2b4b60 con 0x56238b24ce90 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b2b4cd0 con 0x56238b24c570 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238b2b4e90 con 0x56238b257d40 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd04f560b80 10 client.0 _ll_register_callbacks cb 0x7fd02c009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd03919a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b25d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd03a19c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b2b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd03919a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b25d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40474/0 (socket says 172.21.15.5:40474) 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd03919a700 1 -- 172.21.15.5:0/1056112444 learned_addr learned my addr 172.21.15.5:0/1056112444 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.521+0000 7fd03999b700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 -- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 msgr2=0x56238b25cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 -- 172.21.15.5:0/1056112444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 msgr2=0x56238b25d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b25d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 -- 172.21.15.5:0/1056112444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56238b2028d0 con 0x56238b257d40 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd03a19c700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b2b45f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009fd0 tx=0x7fd02c00be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10b1f77be53f16c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd01d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c027040 con 0x56238b257d40 2023-09-23T04:50:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03009ae20 con 0x56238b257d40 2023-09-23T04:50:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd02c0050b0 con 0x56238b257d40 2023-09-23T04:50:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.522+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02c00e040 con 0x56238b257d40 2023-09-23T04:50:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 10 client.26662 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56238aff1250 con 0x56238b257d40 2023-09-23T04:50:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd02c044020 con 0x56238b257d40 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd02c033050 con 0x56238b257d40 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 10 client.26662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 20 client.26662 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b2bc460 0x56238b2be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 4 client.26662 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56238b198b30 con 0x56238b2bc460 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.523+0000 7fd03999b700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b2bc460 0x56238b2be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd03999b700 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b2bc460 0x56238b2be880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4664ca09ec86d3b2 server_cookie=ac0112db44f841ed in_seq=0 out_seq=0 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd01d7fa700 10 client.26662 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd01d7fa700 1 -- 172.21.15.5:0/1056112444 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56238b198b30 con 0x56238b2bc460 2023-09-23T04:50:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd01d7fa700 10 client.26662 handle_command_reply: tid=0 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 client.26662 shutdown 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 msgr2=0x56238b2b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b2b45f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009fd0 tx=0x7fd02c00be90 comp rx=0 tx=0).stop 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 shutdown_connections 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x56238b2bc460 0x56238b2be880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56238b257d40 0x56238b2b45f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56238b24c570 0x56238b25cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd01d7fa700 0 client.26662 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 --2- 172.21.15.5:0/1056112444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56238b24ce90 0x56238b25d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 >> 172.21.15.5:0/1056112444 conn(0x56238b1c1260 msgr2=0x56238b24ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.524+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 shutdown_connections 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd04f560b80 1 -- 172.21.15.5:0/1056112444 wait complete. 2023-09-23T04:50:03.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd04f560b80 20 client.26662 trim_cache size 0 max 16384 2023-09-23T04:50:03.538 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:03.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 msgr2=0x7fd0200a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 0x7fd0200a3bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56238b24e000 tx=0x7fd02c0078d0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 msgr2=0x7fd034199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.525+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd03000eb90 tx=0x7fd03000eea0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 shutdown_connections 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0200a1710 0x7fd0200a3bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03410d470 0x7fd03419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0340731f0 0x7fd034199080 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 --2- 172.21.15.5:0/3508702726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd034073b10 0x7fd0341995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 >> 172.21.15.5:0/3508702726 conn(0x7fd0340fc680 msgr2=0x7fd03410f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 shutdown_connections 2023-09-23T04:50:03.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.526+0000 7fd03a99d700 1 -- 172.21.15.5:0/3508702726 wait complete. 2023-09-23T04:50:03.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 -- 172.21.15.5:0/1457727119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 msgr2=0x7ff81810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 --2- 172.21.15.5:0/1457727119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff81810cf60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff808009990 tx=0x7ff808009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 -- 172.21.15.5:0/1457727119 shutdown_connections 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 --2- 172.21.15.5:0/1457727119 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff81810d4a0 0x7ff81810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 --2- 172.21.15.5:0/1457727119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8180731f0 0x7ff8180735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 --2- 172.21.15.5:0/1457727119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff81810cf60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 -- 172.21.15.5:0/1457727119 >> 172.21.15.5:0/1457727119 conn(0x7ff8180fc710 msgr2=0x7ff8180feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 -- 172.21.15.5:0/1457727119 shutdown_connections 2023-09-23T04:50:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.789+0000 7ff81e19d700 1 -- 172.21.15.5:0/1457727119 wait complete. 2023-09-23T04:50:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.790+0000 7ff81e19d700 1 Processor -- start 2023-09-23T04:50:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.790+0000 7ff81e19d700 1 -- start start 2023-09-23T04:50:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8180731f0 0x7ff818199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81810d4a0 0x7ff81819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81819dff0 con 0x7ff81810d4a0 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81819e160 con 0x7ff818073b10 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81e19d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81819e320 con 0x7ff8180731f0 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81d99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81810d4a0 0x7ff81819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40012/0 (socket says 172.21.15.5:40012) 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 -- 172.21.15.5:0/4012314041 learned_addr learned my addr 172.21.15.5:0/4012314041 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81d19b700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8180731f0 0x7ff818199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 -- 172.21.15.5:0/4012314041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8180731f0 msgr2=0x7ff818199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8180731f0 0x7ff818199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 -- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81810d4a0 msgr2=0x7ff81819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81810d4a0 0x7ff81819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.791+0000 7ff81c99a700 1 -- 172.21.15.5:0/4012314041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009520 con 0x7ff818073b10 2023-09-23T04:50:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff81c99a700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff808005910 tx=0x7ff808005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff80e7fc700 1 -- 172.21.15.5:0/4012314041 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80801d070 con 0x7ff818073b10 2023-09-23T04:50:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff80e7fc700 1 -- 172.21.15.5:0/4012314041 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff808022470 con 0x7ff818073b10 2023-09-23T04:50:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8080095f0 con 0x7ff818073b10 2023-09-23T04:50:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff80e7fc700 1 -- 172.21.15.5:0/4012314041 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808017650 con 0x7ff818073b10 2023-09-23T04:50:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.792+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8181a5c60 con 0x7ff818073b10 2023-09-23T04:50:03.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.797+0000 7ff80e7fc700 1 -- 172.21.15.5:0/4012314041 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff8080225e0 con 0x7ff818073b10 2023-09-23T04:50:03.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.798+0000 7ff832c32b80 1 Processor -- start 2023-09-23T04:50:03.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 -- start start 2023-09-23T04:50:03.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16030950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b1603e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f99460 con 0x559b16030570 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f8ac20 con 0x559b16030e90 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff832c32b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f88540 con 0x559b1603bd40 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff80e7fc700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 0x7ff8040a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff81d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff81d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40022/0 (socket says 172.21.15.5:40022) 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff81d19b700 1 -- 172.21.15.5:0/2587390439 learned_addr learned my addr 172.21.15.5:0/2587390439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff81d99c700 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16030950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff80e7fc700 1 -- 172.21.15.5:0/4012314041 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff8080cea00 con 0x7ff818073b10 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.799+0000 7ff81c99a700 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b1603e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d99c700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 0x7ff8040a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 -- 172.21.15.5:0/2587390439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 msgr2=0x559b1603e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b1603e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 -- 172.21.15.5:0/2587390439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 msgr2=0x559b16030950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16030950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 -- 172.21.15.5:0/2587390439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b15fe68d0 con 0x559b16030e90 2023-09-23T04:50:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f10 tx=0x7ff814011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2bd1c8110439888 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff8037fe700 1 -- 172.21.15.5:0/2587390439 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814019630 con 0x559b16030e90 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff81d99c700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 0x7ff8040a3bd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff810009ec0 tx=0x7ff81000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff8037fe700 1 -- 172.21.15.5:0/2587390439 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff814019c50 con 0x559b16030e90 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.800+0000 7ff8037fe700 1 -- 172.21.15.5:0/2587390439 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ec80 con 0x559b16030e90 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 -- 172.21.15.5:0/2587390439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 msgr2=0x559b1603b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f10 tx=0x7ff814011720 comp rx=0 tx=0).stop 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 -- 172.21.15.5:0/2587390439 shutdown_connections 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b1603e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16030950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2587390439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b1603b800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 -- 172.21.15.5:0/2587390439 >> 172.21.15.5:0/2587390439 conn(0x559b15fa5260 msgr2=0x559b1602c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 -- 172.21.15.5:0/2587390439 shutdown_connections 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.801+0000 7ff832c32b80 1 -- 172.21.15.5:0/2587390439 wait complete. 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 -- start start 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 Processor -- start 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b1603b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b16039150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f83e30 con 0x559b16030570 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f20710 con 0x559b16030e90 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.802+0000 7ff832c32b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15f203b0 con 0x559b1603bd40 2023-09-23T04:50:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff832c32b80 10 client.0 _ll_register_callbacks cb 0x559b15d7d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40034/0 (socket says 172.21.15.5:40034) 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 -- 172.21.15.5:0/81878181 learned_addr learned my addr 172.21.15.5:0/81878181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81d99c700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b16039150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81d19b700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b1603b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 -- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 msgr2=0x559b16039150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b16039150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 -- 172.21.15.5:0/81878181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 msgr2=0x559b1603b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b1603b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 -- 172.21.15.5:0/81878181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff814009b70 con 0x559b16030e90 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff81c99a700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff80800f740 tx=0x7ff808005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ac3b1634060b95 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.803+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80809ba80 con 0x559b16030e90 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff80809bd20 con 0x559b16030e90 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b160244a0 con 0x559b16030e90 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff80809b160 con 0x559b16030e90 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 10 client.26618 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559b15daede0 con 0x559b16030e90 2023-09-23T04:50:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b160244a0 con 0x559b16030e90 2023-09-23T04:50:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff80809ab10 con 0x559b16030e90 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 10 client.26618 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 10 client.26618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 20 client.26618 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b1604b4d0 0x559b1604d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 4 client.26618 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.804+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b15f7ce10 con 0x559b1604b4d0 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.805+0000 7ff81d19b700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b1604b4d0 0x559b1604d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.805+0000 7ff81d19b700 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b1604b4d0 0x559b1604d8f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9b487a06f9a17c1 server_cookie=74ac4d18e543d426 in_seq=0 out_seq=0 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.805+0000 7ff800ff9700 10 client.26618 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff800ff9700 1 -- 172.21.15.5:0/81878181 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559b15f7ce10 con 0x559b1604b4d0 2023-09-23T04:50:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff800ff9700 10 client.26618 handle_command_reply: tid=0 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 client.26618 shutdown 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 msgr2=0x559b16038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff80800f740 tx=0x7ff808005f10 comp rx=0 tx=0).stop 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 shutdown_connections 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b16039150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b1603b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b1604b4d0 0x559b1604d8f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 --2- 172.21.15.5:0/81878181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030e90 0x559b16038a10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 >> 172.21.15.5:0/81878181 conn(0x559b15fa5260 msgr2=0x559b1602c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff800ff9700 0 client.26618 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.806+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 shutdown_connections 2023-09-23T04:50:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.807+0000 7ff832c32b80 1 -- 172.21.15.5:0/81878181 wait complete. 2023-09-23T04:50:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.807+0000 7ff832c32b80 20 client.26618 trim_cache size 0 max 16384 2023-09-23T04:50:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 Processor -- start 2023-09-23T04:50:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 -- start start 2023-09-23T04:50:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030e90 0x559b1603b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b160382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15e04920 con 0x559b16030e90 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16034880 con 0x559b16030570 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.822+0000 7ff832c32b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1602e2d0 con 0x559b1603bd40 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40040/0 (socket says 172.21.15.5:40040) 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 -- 172.21.15.5:0/1638393298 learned_addr learned my addr 172.21.15.5:0/1638393298 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d99c700 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b160382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81c99a700 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030e90 0x559b1603b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 -- 172.21.15.5:0/1638393298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 msgr2=0x559b160382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b160382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 -- 172.21.15.5:0/1638393298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030e90 msgr2=0x559b1603b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030e90 0x559b1603b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 -- 172.21.15.5:0/1638393298 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b15d97c60 con 0x559b16030570 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.823+0000 7ff81d19b700 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff814009e60 tx=0x7ff814005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199d5746eb24eb73 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff803fff700 1 -- 172.21.15.5:0/1638393298 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814029070 con 0x559b16030570 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff803fff700 1 -- 172.21.15.5:0/1638393298 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81400e4d0 con 0x559b16030570 2023-09-23T04:50:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff803fff700 1 -- 172.21.15.5:0/1638393298 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814005560 con 0x559b16030570 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 -- 172.21.15.5:0/1638393298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 msgr2=0x559b160334c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff814009e60 tx=0x7ff814005e40 comp rx=0 tx=0).stop 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 -- 172.21.15.5:0/1638393298 shutdown_connections 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b1603bd40 0x559b160382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b16030570 0x559b160334c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 --2- 172.21.15.5:0/1638393298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030e90 0x559b1603b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 -- 172.21.15.5:0/1638393298 >> 172.21.15.5:0/1638393298 conn(0x559b15fa5260 msgr2=0x559b16033d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 -- 172.21.15.5:0/1638393298 shutdown_connections 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.824+0000 7ff832c32b80 1 -- 172.21.15.5:0/1638393298 wait complete. 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.825+0000 7ff832c32b80 1 -- start start 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.825+0000 7ff832c32b80 1 Processor -- start 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16042620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b16030e90 0x559b16040950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b160413d0 con 0x559b16030570 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16041540 con 0x559b1603bd40 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b16041700 con 0x559b16030e90 2023-09-23T04:50:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff832c32b80 10 client.0 _ll_register_callbacks cb 0x7ff808005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff81d99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff81d99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40052/0 (socket says 172.21.15.5:40052) 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff81d99c700 1 -- 172.21.15.5:0/2592285406 learned_addr learned my addr 172.21.15.5:0/2592285406 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff81c99a700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b16030e90 0x559b16040950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.826+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16042620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 -- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b16030e90 msgr2=0x559b16040950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b16030e90 0x559b16040950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 -- 172.21.15.5:0/2592285406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 msgr2=0x559b16042620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16042620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 -- 172.21.15.5:0/2592285406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b15fe68d0 con 0x559b1603bd40 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16042620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff81d99c700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff810018d10 tx=0x7ff810004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae7cfa3e6455bd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81002a070 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff814019c50 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ff810007e40 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff81000e070 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff832c32b80 10 client.26624 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.827+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559b15dd5250 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff814019c50 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff81002d020 con 0x559b1603bd40 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 10 client.26624 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 10 client.26624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:50:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 20 client.26624 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b160a0720 0x559b160a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 4 client.26624 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b15f7cb30 con 0x559b160a0720 2023-09-23T04:50:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.828+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b160a0720 0x559b160a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:03.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff81d19b700 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b160a0720 0x559b160a2b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b9f781fd691d9b5 server_cookie=b0acfc0316f92448 in_seq=0 out_seq=0 2023-09-23T04:50:03.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff800ff9700 10 client.26624 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:03.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff800ff9700 1 -- 172.21.15.5:0/2592285406 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b15f7cb30 con 0x559b160a0720 2023-09-23T04:50:03.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff800ff9700 10 client.26624 handle_command_reply: tid=0 2023-09-23T04:50:03.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 client.26624 shutdown 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 msgr2=0x559b16040e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff810018d10 tx=0x7ff810004ad0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 shutdown_connections 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x559b160a0720 0x559b160a2b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b1603bd40 0x559b16040e90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff800ff9700 0 client.26624 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559b16030570 0x559b16042620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 --2- 172.21.15.5:0/2592285406 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559b16030e90 0x559b16040950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 >> 172.21.15.5:0/2592285406 conn(0x559b15fa5260 msgr2=0x559b160382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 shutdown_connections 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.829+0000 7ff832c32b80 1 -- 172.21.15.5:0/2592285406 wait complete. 2023-09-23T04:50:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff832c32b80 20 client.26624 trim_cache size 0 max 16384 2023-09-23T04:50:03.836 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 msgr2=0x7ff8040a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 0x7ff8040a3bd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff810009ec0 tx=0x7ff81000b040 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 msgr2=0x7ff818199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff808005910 tx=0x7ff808005720 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 shutdown_connections 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff8040a1710 0x7ff8040a3bd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81810d4a0 0x7ff81819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff8180731f0 0x7ff818199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 --2- 172.21.15.5:0/4012314041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff818073b10 0x7ff818199550 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 >> 172.21.15.5:0/4012314041 conn(0x7ff8180fc710 msgr2=0x7ff81810f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 shutdown_connections 2023-09-23T04:50:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:03.830+0000 7ff81e19d700 1 -- 172.21.15.5:0/4012314041 wait complete. 2023-09-23T04:50:03.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:50:03.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241301 8.17 MB/sec execute 39 sec latency 27.875 ms 2023-09-23T04:50:04.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/255769641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103310 msgr2=0x7fa9b41036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/255769641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103310 0x7fa9b41036f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009990 tx=0x7fa9a4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/255769641 shutdown_connections 2023-09-23T04:50:04.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/255769641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/255769641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103310 0x7fa9b41036f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/255769641 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103cc0 0x7fa9b4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/255769641 >> 172.21.15.5:0/255769641 conn(0x7fa9b40feb50 msgr2=0x7fa9b4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/255769641 shutdown_connections 2023-09-23T04:50:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.136+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/255769641 wait complete. 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.137+0000 7fa9ba9ff700 1 Processor -- start 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 -- start start 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b4199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 0x7fa9b419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b419e630 con 0x7fa9b419dbb0 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b419e7a0 con 0x7fa9b4108430 2023-09-23T04:50:04.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9ba9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b419e960 con 0x7fa9b4103310 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59060/0 (socket says 172.21.15.5:59060) 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3416068741 learned_addr learned my addr 172.21.15.5:0/3416068741 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b4199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.138+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 0x7fa9b419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3416068741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 msgr2=0x7fa9b4199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b4199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 msgr2=0x7fa9b419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 0x7fa9b419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3416068741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac009c40 con 0x7fa9b4103310 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 0x7fa9b419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:04.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4005910 tx=0x7fa9a4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b17fa700 1 -- 172.21.15.5:0/3416068741 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a401d070 con 0x7fa9b4103310 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40095f0 con 0x7fa9b4103310 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b17fa700 1 -- 172.21.15.5:0/3416068741 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a4022470 con 0x7fa9b4103310 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9b17fa700 1 -- 172.21.15.5:0/3416068741 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4017650 con 0x7fa9b4103310 2023-09-23T04:50:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.139+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41a6100 con 0x7fa9b4103310 2023-09-23T04:50:04.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.143+0000 7fa9b17fa700 1 -- 172.21.15.5:0/3416068741 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa9a40225e0 con 0x7fa9b4103310 2023-09-23T04:50:04.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.144+0000 7fa9ce348b80 1 Processor -- start 2023-09-23T04:50:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 -- start start 2023-09-23T04:50:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9b17fa700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 0x7fa99c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9b17fa700 1 -- 172.21.15.5:0/3416068741 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa9a40ceb20 con 0x7fa9b4103310 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x564509141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 0x7fa99c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645090a2460 con 0x564509142530 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564509093c20 con 0x56450914d3e0 2023-09-23T04:50:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.145+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564509091540 con 0x564509141c10 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b8f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b8f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40082/0 (socket says 172.21.15.5:40082) 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b8f9c700 1 -- 172.21.15.5:0/2791724787 learned_addr learned my addr 172.21.15.5:0/2791724787 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x564509141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2791724787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 msgr2=0x56450914f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2791724787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 msgr2=0x56450914cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2791724787 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645090ef8d0 con 0x564509141c10 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.146+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 0x7fa99c0a3bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009550 tx=0x7fa9ac007d40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x564509141ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cbe30 tx=0x7fa9a409b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59179a48d4c75616 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa99a7fc700 1 -- 172.21.15.5:0/2791724787 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40bb070 con 0x564509141c10 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa99a7fc700 1 -- 172.21.15.5:0/2791724787 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a409bde0 con 0x564509141c10 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa99a7fc700 1 -- 172.21.15.5:0/2791724787 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a409b940 con 0x564509141c10 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2791724787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 msgr2=0x564509141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.147+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x564509141ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cbe30 tx=0x7fa9a409b4b0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2791724787 shutdown_connections 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x56450914f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x564509141ff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2791724787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2791724787 >> 172.21.15.5:0/2791724787 conn(0x5645090ae260 msgr2=0x564509135790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2791724787 shutdown_connections 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.148+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2791724787 wait complete. 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 -- start start 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 Processor -- start 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x564509158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450908ce30 con 0x564509142530 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564509029710 con 0x56450914d3e0 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645090293b0 con 0x564509141c10 2023-09-23T04:50:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.149+0000 7fa9ce348b80 10 client.0 _ll_register_callbacks cb 0x564508e86800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59082/0 (socket says 172.21.15.5:59082) 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 -- 172.21.15.5:0/1250954332 learned_addr learned my addr 172.21.15.5:0/1250954332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x564509158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 -- 172.21.15.5:0/1250954332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 msgr2=0x564509158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x564509158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 -- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 msgr2=0x56450914dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 -- 172.21.15.5:0/1250954332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac0177e0 con 0x564509141c10 2023-09-23T04:50:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40a34b0 tx=0x7fa9a4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=191857ba35ee26ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.150+0000 7fa9937fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40b7070 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa9a409b6e0 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450912d250 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9a40bb0c0 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9ce348b80 10 client.26692 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fa9b41a6100 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.151+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56450912d250 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa9a409ba90 con 0x564509141c10 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 10 client.26692 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 20 client.26692 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56450915c650 0x56450915ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 4 client.26692 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564509085e10 con 0x56450915c650 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.152+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56450915c650 0x56450915ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.153+0000 7fa9937fe700 10 client.26692 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.153+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56450915c650 0x56450915ea90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf9914eb581eacf server_cookie=66b610725b0cc88d in_seq=0 out_seq=0 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9937fe700 1 -- 172.21.15.5:0/1250954332 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564509085e10 con 0x56450915c650 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9937fe700 10 client.26692 handle_command_reply: tid=0 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 client.26692 shutdown 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 msgr2=0x56450914d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40a34b0 tx=0x7fa9a4009310 comp rx=0 tx=0).stop 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 shutdown_connections 2023-09-23T04:50:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56450915c650 0x56450915ea90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56450914d3e0 0x564509158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564509141c10 0x56450914d7c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9937fe700 0 client.26692 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/1250954332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x56450914dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 >> 172.21.15.5:0/1250954332 conn(0x5645090ae260 msgr2=0x564509135320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.154+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 shutdown_connections 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.155+0000 7fa9ce348b80 1 -- 172.21.15.5:0/1250954332 wait complete. 2023-09-23T04:50:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.155+0000 7fa9ce348b80 20 client.26692 trim_cache size 0 max 16384 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 Processor -- start 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 -- start start 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x564509147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564508f0d920 con 0x564509142530 2023-09-23T04:50:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450914a9d0 con 0x564509141c10 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564509134580 con 0x56450914d3e0 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9b8f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40102/0 (socket says 172.21.15.5:40102) 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3753914665 learned_addr learned my addr 172.21.15.5:0/3753914665 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.173+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x564509147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3753914665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 msgr2=0x564509157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3753914665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 msgr2=0x564509147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x564509147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 -- 172.21.15.5:0/3753914665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564509131480 con 0x564509141c10 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x564509147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4099a30 tx=0x7fa9a4022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd9a734a97a8287b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa99affd700 1 -- 172.21.15.5:0/3753914665 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a409b6e0 con 0x564509141c10 2023-09-23T04:50:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa99affd700 1 -- 172.21.15.5:0/3753914665 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a409f830 con 0x564509141c10 2023-09-23T04:50:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.174+0000 7fa99affd700 1 -- 172.21.15.5:0/3753914665 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40a39e0 con 0x564509141c10 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 -- 172.21.15.5:0/3753914665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 msgr2=0x5645091477c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4099a30 tx=0x7fa9a4022f90 comp rx=0 tx=0).stop 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 -- 172.21.15.5:0/3753914665 shutdown_connections 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x5645091477c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/3753914665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x564509147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 -- 172.21.15.5:0/3753914665 >> 172.21.15.5:0/3753914665 conn(0x5645090ae260 msgr2=0x56450914d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 -- 172.21.15.5:0/3753914665 shutdown_connections 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.175+0000 7fa9ce348b80 1 -- 172.21.15.5:0/3753914665 wait complete. 2023-09-23T04:50:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 -- start start 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 Processor -- start 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x5645091504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509150a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.176+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645091af530 con 0x564509142530 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645091af6a0 con 0x564509141c10 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9ce348b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645091af860 con 0x56450914d3e0 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9ce348b80 10 client.0 _ll_register_callbacks cb 0x7fa9a8009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40118/0 (socket says 172.21.15.5:40118) 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2971971344 learned_addr learned my addr 172.21.15.5:0/2971971344 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b8f9c700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x5645091504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2971971344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 msgr2=0x564509150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 msgr2=0x5645091504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x5645091504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 -- 172.21.15.5:0/2971971344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac017ac0 con 0x564509141c10 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x5645091504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009520 tx=0x7fa9a4099c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4c867662ef3709 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.177+0000 7fa9937fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40b7030 con 0x564509141c10 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa9a40b8070 con 0x564509141c10 2023-09-23T04:50:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9a409bde0 con 0x564509141c10 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9a40a39e0 con 0x564509141c10 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 10 client.26642 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fa9b41a6100 con 0x564509141c10 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a409bde0 con 0x564509141c10 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa9a409a640 con 0x564509141c10 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 20 client.26642 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645091b0a70 0x5645091b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 4 client.26642 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.178+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564509085b30 con 0x5645091b0a70 2023-09-23T04:50:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645091b0a70 0x5645091b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645091b0a70 0x5645091b2e60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1dc804a9f65e841 server_cookie=2445a815a1263f38 in_seq=0 out_seq=0 2023-09-23T04:50:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9937fe700 10 client.26642 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9937fe700 1 -- 172.21.15.5:0/2971971344 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564509085b30 con 0x5645091b0a70 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9937fe700 10 client.26642 handle_command_reply: tid=0 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.179+0000 7fa9ce348b80 1 client.26642 shutdown 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 msgr2=0x56450914ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009520 tx=0x7fa9a4099c90 comp rx=0 tx=0).stop 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 shutdown_connections 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5645091b0a70 0x5645091b2e60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56450914d3e0 0x564509150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564509141c10 0x56450914ff90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 --2- 172.21.15.5:0/2971971344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564509142530 0x5645091504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9937fe700 0 client.26642 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 >> 172.21.15.5:0/2971971344 conn(0x5645090ae260 msgr2=0x56450914d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 shutdown_connections 2023-09-23T04:50:04.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 1 -- 172.21.15.5:0/2971971344 wait complete. 2023-09-23T04:50:04.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.180+0000 7fa9ce348b80 20 client.26642 trim_cache size 0 max 16384 2023-09-23T04:50:04.186 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 msgr2=0x7fa99c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 0x7fa99c0a3bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009550 tx=0x7fa9ac007d40 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 msgr2=0x7fa9b4199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4005910 tx=0x7fa9a4005740 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 shutdown_connections 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa99c0a1710 0x7fa99c0a3bd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b419dbb0 0x7fa9b419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa9b4103310 0x7fa9b4199210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 --2- 172.21.15.5:0/3416068741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9b4108430 0x7fa9b4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 >> 172.21.15.5:0/3416068741 conn(0x7fa9b40feb50 msgr2=0x7fa9b4106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 shutdown_connections 2023-09-23T04:50:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:04.181+0000 7fa9ba9ff700 1 -- 172.21.15.5:0/3416068741 wait complete. 2023-09-23T04:50:04.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 242112 8.02 MB/sec execute 40 sec latency 67.804 ms 2023-09-23T04:50:05.202 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:50:05.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:50:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/441258817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 msgr2=0x7fb4e41036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/441258817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e41036f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009990 tx=0x7fb4e0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/441258817 shutdown_connections 2023-09-23T04:50:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/441258817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e4108430 0x7fb4e410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/441258817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e41036f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/441258817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4103cc0 0x7fb4e4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/441258817 >> 172.21.15.5:0/441258817 conn(0x7fb4e40feb50 msgr2=0x7fb4e4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/441258817 shutdown_connections 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.445+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/441258817 wait complete. 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.446+0000 7fb4ec1ff700 1 Processor -- start 2023-09-23T04:50:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 -- start start 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e419dd20 0x7fb4e419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419e7a0 con 0x7fb4e419dd20 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419e910 con 0x7fb4e4108430 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.447+0000 7fb4ec1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419ead0 con 0x7fb4e4103310 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40132/0 (socket says 172.21.15.5:40132) 2023-09-23T04:50:05.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 -- 172.21.15.5:0/26220129 learned_addr learned my addr 172.21.15.5:0/26220129 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4ea79c700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e419dd20 0x7fb4e419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 -- 172.21.15.5:0/26220129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 msgr2=0x7fb4e4199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e4199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 -- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e419dd20 msgr2=0x7fb4e419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e419dd20 0x7fb4e419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 -- 172.21.15.5:0/26220129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00095f0 con 0x7fb4e4108430 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.448+0000 7fb4e979a700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e9f0 tx=0x7fb4d400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.449+0000 7fb4daffd700 1 -- 172.21.15.5:0/26220129 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4009750 con 0x7fb4e4108430 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.449+0000 7fb4daffd700 1 -- 172.21.15.5:0/26220129 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d4004cf0 con 0x7fb4e4108430 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.449+0000 7fb4daffd700 1 -- 172.21.15.5:0/26220129 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400eee0 con 0x7fb4e4108430 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.449+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e41a5fd0 con 0x7fb4e4108430 2023-09-23T04:50:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.449+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e41a6400 con 0x7fb4e4108430 2023-09-23T04:50:05.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.452+0000 7fb4daffd700 1 -- 172.21.15.5:0/26220129 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb4d4004750 con 0x7fb4e4108430 2023-09-23T04:50:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.454+0000 7fb4daffd700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 0x7fb4d00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.454+0000 7fb4daffd700 1 -- 172.21.15.5:0/26220129 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fb4d4014070 con 0x7fb4e4108430 2023-09-23T04:50:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.455+0000 7fb4ffa58b80 1 Processor -- start 2023-09-23T04:50:05.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.455+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 0x7fb4d00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.455+0000 7fb4ffa58b80 1 -- start start 2023-09-23T04:50:05.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c36860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c4c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b9f4e0 con 0x5650c4c4a460 2023-09-23T04:50:05.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b90ca0 con 0x5650c4c36480 2023-09-23T04:50:05.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b8e5c0 con 0x5650c4c36da0 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4ea79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c4c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c36860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 -- 172.21.15.5:0/479467686 learned_addr learned my addr 172.21.15.5:0/479467686 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 0x7fb4d00a3c50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b2e0 tx=0x7fb4e0005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 -- 172.21.15.5:0/479467686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 msgr2=0x5650c4c36860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c36860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 -- 172.21.15.5:0/479467686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 msgr2=0x5650c4c4c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.456+0000 7fb4e979a700 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c4c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.457+0000 7fb4e979a700 1 -- 172.21.15.5:0/479467686 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c4bec950 con 0x5650c4c36da0 2023-09-23T04:50:05.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.457+0000 7fb4e979a700 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40a2300 tx=0x7fb4d409a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371e44146b96e743 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.457+0000 7fb4cffff700 1 -- 172.21.15.5:0/479467686 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409aba0 con 0x5650c4c36da0 2023-09-23T04:50:05.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.457+0000 7fb4cffff700 1 -- 172.21.15.5:0/479467686 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40b0070 con 0x5650c4c36da0 2023-09-23T04:50:05.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.457+0000 7fb4cffff700 1 -- 172.21.15.5:0/479467686 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4096bb0 con 0x5650c4c36da0 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/479467686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 msgr2=0x5650c4c49f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40a2300 tx=0x7fb4d409a2f0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/479467686 shutdown_connections 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c4c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c36860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/479467686 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c49f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/479467686 >> 172.21.15.5:0/479467686 conn(0x5650c4bab2e0 msgr2=0x5650c4c32810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/479467686 shutdown_connections 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.458+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/479467686 wait complete. 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 -- start start 2023-09-23T04:50:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 Processor -- start 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 0x5650c4c471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36da0 0x5650c4c476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b89eb0 con 0x5650c4c36480 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b26790 con 0x5650c4c36da0 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.459+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4b26430 con 0x5650c4c4a460 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4ffa58b80 10 client.0 _ll_register_callbacks cb 0x5650c4983880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4ea79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 0x5650c4c471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36da0 0x5650c4c476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4ea79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c47c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59132/0 (socket says 172.21.15.5:59132) 2023-09-23T04:50:05.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4ea79c700 1 -- 172.21.15.5:0/1682902435 learned_addr learned my addr 172.21.15.5:0/1682902435 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/1682902435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 msgr2=0x5650c4c47c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c47c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/1682902435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36da0 msgr2=0x5650c4c476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36da0 0x5650c4c476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/1682902435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e41a6400 con 0x5650c4c36480 2023-09-23T04:50:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 0x5650c4c471b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0000c00 tx=0x7fb4e0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c6c15c38694fd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.460+0000 7fb4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0004d70 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650c4c2a2d0 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fb4e000f460 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e0005710 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4ffa58b80 10 client.16650 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.461+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650c49b4e60 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650c4c2a2d0 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb4e002e5c0 con 0x5650c4c36480 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 20 client.16650 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4c59f40 0x5650c4c5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 4 client.16650 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650c4b82e90 con 0x5650c4c59f40 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.462+0000 7fb4e979a700 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4c59f40 0x5650c4c5c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.463+0000 7fb4cd7fa700 10 client.16650 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.463+0000 7fb4e979a700 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4c59f40 0x5650c4c5c360 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8612c502dc3f8f4a server_cookie=bf7863af73d270d2 in_seq=0 out_seq=0 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.463+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/1682902435 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650c4b82e90 con 0x5650c4c59f40 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.463+0000 7fb4cd7fa700 10 client.16650 handle_command_reply: tid=0 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.463+0000 7fb4ffa58b80 1 client.16650 shutdown 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 msgr2=0x5650c4c471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 0x5650c4c471b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0000c00 tx=0x7fb4e0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 shutdown_connections 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4c59f40 0x5650c4c5c360 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c47c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36480 0x5650c4c471b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4cd7fa700 0 client.16650 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/1682902435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36da0 0x5650c4c476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 >> 172.21.15.5:0/1682902435 conn(0x5650c4bab2e0 msgr2=0x5650c4c323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 shutdown_connections 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/1682902435 wait complete. 2023-09-23T04:50:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.464+0000 7fb4ffa58b80 20 client.16650 trim_cache size 0 max 16384 2023-09-23T04:50:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 Processor -- start 2023-09-23T04:50:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 -- start start 2023-09-23T04:50:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c44460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4a0a9a0 con 0x5650c4c4a460 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4c33630 con 0x5650c4c36480 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4c4a240 con 0x5650c4c36da0 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.484+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59136/0 (socket says 172.21.15.5:59136) 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 -- 172.21.15.5:0/712414188 learned_addr learned my addr 172.21.15.5:0/712414188 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4ea79c700 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c44460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 -- 172.21.15.5:0/712414188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 msgr2=0x5650c4c47750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c47750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 -- 172.21.15.5:0/712414188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 msgr2=0x5650c4c44460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c44460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 -- 172.21.15.5:0/712414188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c499dce0 con 0x5650c4c36da0 2023-09-23T04:50:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4ea79c700 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4e979a700 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4d409a430 tx=0x7fb4d4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df051d121574204 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4d8ff9700 1 -- 172.21.15.5:0/712414188 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409e5f0 con 0x5650c4c36da0 2023-09-23T04:50:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4d8ff9700 1 -- 172.21.15.5:0/712414188 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40a2830 con 0x5650c4c36da0 2023-09-23T04:50:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.485+0000 7fb4d8ff9700 1 -- 172.21.15.5:0/712414188 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40caf00 con 0x5650c4c36da0 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/712414188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 msgr2=0x5650c4c47c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4d409a430 tx=0x7fb4d4005740 comp rx=0 tx=0).stop 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/712414188 shutdown_connections 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c4a460 0x5650c4c44460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c47750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/712414188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c36da0 0x5650c4c47c90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/712414188 >> 172.21.15.5:0/712414188 conn(0x5650c4bab2e0 msgr2=0x5650c4c38d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/712414188 shutdown_connections 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.486+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/712414188 wait complete. 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.487+0000 7fb4ffa58b80 1 -- start start 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.487+0000 7fb4ffa58b80 1 Processor -- start 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 0x5650c4c4fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c50300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4cadf50 con 0x5650c4c36da0 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4cae0c0 con 0x5650c4c36480 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c4cae280 con 0x5650c4c4a460 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ffa58b80 10 client.0 _ll_register_callbacks cb 0x7fb4e000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40168/0 (socket says 172.21.15.5:40168) 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/2794362681 learned_addr learned my addr 172.21.15.5:0/2794362681 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4ea79c700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c50300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e979a700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 0x5650c4c4fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/2794362681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 msgr2=0x5650c4c50300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c50300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 msgr2=0x5650c4c4fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 0x5650c4c4fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 -- 172.21.15.5:0/2794362681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d40b6040 con 0x5650c4c36480 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e979a700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 0x5650c4c4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4e9f9b700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b690 tx=0x7fb4e002ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3a1c2daf391586 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.488+0000 7fb4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000b310 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4d40b0070 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fb4e0004b70 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e0004d30 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4ffa58b80 10 client.26660 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650c49db2d0 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40b0070 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.489+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb4e0040430 con 0x5650c4c36480 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 10 client.26660 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 20 client.26660 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4caf600 0x5650c4cb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 4 client.26660 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:50:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5650c4b82bb0 con 0x5650c4caf600 2023-09-23T04:50:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4e979a700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4caf600 0x5650c4cb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4e979a700 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4caf600 0x5650c4cb1a20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdb0012885934c0 server_cookie=d606a8954634a7be in_seq=0 out_seq=0 2023-09-23T04:50:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.490+0000 7fb4cd7fa700 10 client.26660 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.491+0000 7fb4cd7fa700 1 -- 172.21.15.5:0/2794362681 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5650c4b82bb0 con 0x5650c4caf600 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.491+0000 7fb4cd7fa700 10 client.26660 handle_command_reply: tid=0 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 client.26660 shutdown 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 msgr2=0x5650c4c4f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b690 tx=0x7fb4e002ee40 comp rx=0 tx=0).stop 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 shutdown_connections 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5650c4caf600 0x5650c4cb1a20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5650c4c4a460 0x5650c4c50300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5650c4c36480 0x5650c4c4f840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 --2- 172.21.15.5:0/2794362681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650c4c36da0 0x5650c4c4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4cd7fa700 0 client.26660 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 >> 172.21.15.5:0/2794362681 conn(0x5650c4bab2e0 msgr2=0x5650c4c47740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 shutdown_connections 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.492+0000 7fb4ffa58b80 1 -- 172.21.15.5:0/2794362681 wait complete. 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.493+0000 7fb4ffa58b80 20 client.26660 trim_cache size 0 max 16384 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c6e82ee9-f7cf-432f-89ed-92eeed95bc4c", 2023-09-23T04:50:05.497 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.493+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 msgr2=0x7fb4d00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.493+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 0x7fb4d00a3c50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000b2e0 tx=0x7fb4e0005fd0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.493+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 msgr2=0x7fb4e41998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.493+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e9f0 tx=0x7fb4d400ed00 comp rx=0 tx=0).stop 2023-09-23T04:50:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 shutdown_connections 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4d00a1790 0x7fb4d00a3c50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e419dd20 0x7fb4e419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4e4103310 0x7fb4e4199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 --2- 172.21.15.5:0/26220129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4e4108430 0x7fb4e41998c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 >> 172.21.15.5:0/26220129 conn(0x7fb4e40feb50 msgr2=0x7fb4e40ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 shutdown_connections 2023-09-23T04:50:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.494+0000 7fb4ec1ff700 1 -- 172.21.15.5:0/26220129 wait complete. 2023-09-23T04:50:05.517 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 -- 172.21.15.5:0/2723641208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1010d0 msgr2=0x7fca1c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 --2- 172.21.15.5:0/2723641208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1010d0 0x7fca1c1014b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca18009990 tx=0x7fca18009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 -- 172.21.15.5:0/2723641208 shutdown_connections 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 --2- 172.21.15.5:0/2723641208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1061f0 0x7fca1c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 --2- 172.21.15.5:0/2723641208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1010d0 0x7fca1c1014b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 --2- 172.21.15.5:0/2723641208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c101a80 0x7fca1c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 -- 172.21.15.5:0/2723641208 >> 172.21.15.5:0/2723641208 conn(0x7fca1c0fc990 msgr2=0x7fca1c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 -- 172.21.15.5:0/2723641208 shutdown_connections 2023-09-23T04:50:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.762+0000 7fca235ff700 1 -- 172.21.15.5:0/2723641208 wait complete. 2023-09-23T04:50:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 Processor -- start 2023-09-23T04:50:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 -- start start 2023-09-23T04:50:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c19dae0 0x7fca1c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19e560 con 0x7fca1c19dae0 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19e6d0 con 0x7fca1c1061f0 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca235ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c19e890 con 0x7fca1c1010d0 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca2139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.764+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40188/0 (socket says 172.21.15.5:40188) 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca2139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59152/0 (socket says 172.21.15.5:59152) 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 -- 172.21.15.5:0/3156085712 learned_addr learned my addr 172.21.15.5:0/3156085712 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca21b9c700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c19dae0 0x7fca1c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 -- 172.21.15.5:0/3156085712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 msgr2=0x7fca1c199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 -- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c19dae0 msgr2=0x7fca1c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c19dae0 0x7fca1c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 -- 172.21.15.5:0/3156085712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180095f0 con 0x7fca1c1061f0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca2139b700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca20b9a700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e970 tx=0x7fca0c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.765+0000 7fca127fc700 1 -- 172.21.15.5:0/3156085712 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c009790 con 0x7fca1c1061f0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.766+0000 7fca127fc700 1 -- 172.21.15.5:0/3156085712 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c00c7e0 con 0x7fca1c1061f0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.766+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1a5df0 con 0x7fca1c1061f0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.766+0000 7fca127fc700 1 -- 172.21.15.5:0/3156085712 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0054e0 con 0x7fca1c1061f0 2023-09-23T04:50:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.766+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a6280 con 0x7fca1c1061f0 2023-09-23T04:50:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.770+0000 7fca127fc700 1 -- 172.21.15.5:0/3156085712 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fca0c00c950 con 0x7fca1c1061f0 2023-09-23T04:50:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.771+0000 7fca36f0ab80 1 Processor -- start 2023-09-23T04:50:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.771+0000 7fca127fc700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 0x7fca080a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.771+0000 7fca36f0ab80 1 -- start start 2023-09-23T04:50:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.771+0000 7fca127fc700 1 -- 172.21.15.5:0/3156085712 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fca0c014070 con 0x7fca1c1061f0 2023-09-23T04:50:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.771+0000 7fca21b9c700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 0x7fca080a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6dfa1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6deff460 con 0x55ad6df96400 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6def0c20 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6deee540 con 0x55ad6df96d20 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca2139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca2139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40670/0 (socket says 172.21.15.5:40670) 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6dfa1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca2139b700 1 -- 172.21.15.5:0/780333090 learned_addr learned my addr 172.21.15.5:0/780333090 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca21b9c700 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 -- 172.21.15.5:0/780333090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 msgr2=0x55ad6dfa3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 -- 172.21.15.5:0/780333090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 msgr2=0x55ad6df967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca20b9a700 1 -- 172.21.15.5:0/780333090 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad6df4c8d0 con 0x55ad6df96d20 2023-09-23T04:50:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.772+0000 7fca21b9c700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 0x7fca080a3c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca1400f590 tx=0x7fca1400b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.773+0000 7fca20b9a700 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6dfa1690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0cac30 tx=0x7fca0c09a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8c420b1059c972 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.773+0000 7fca077fe700 1 -- 172.21.15.5:0/780333090 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0b0070 con 0x55ad6df96d20 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.773+0000 7fca077fe700 1 -- 172.21.15.5:0/780333090 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c09a450 con 0x55ad6df96d20 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.773+0000 7fca077fe700 1 -- 172.21.15.5:0/780333090 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c005660 con 0x55ad6df96d20 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 -- 172.21.15.5:0/780333090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 msgr2=0x55ad6dfa1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6dfa1690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0cac30 tx=0x7fca0c09a330 comp rx=0 tx=0).stop 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 -- 172.21.15.5:0/780333090 shutdown_connections 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/780333090 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6dfa1690 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 -- 172.21.15.5:0/780333090 >> 172.21.15.5:0/780333090 conn(0x55ad6df0b260 msgr2=0x55ad6df92790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 -- 172.21.15.5:0/780333090 shutdown_connections 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.774+0000 7fca36f0ab80 1 -- 172.21.15.5:0/780333090 wait complete. 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.775+0000 7fca36f0ab80 1 -- start start 2023-09-23T04:50:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.775+0000 7fca36f0ab80 1 Processor -- start 2023-09-23T04:50:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 0x55ad6df9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 0x55ad6df9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6dee9e30 con 0x55ad6df96d20 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6de86710 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6de863b0 con 0x55ad6df96400 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca36f0ab80 10 client.0 _ll_register_callbacks cb 0x55ad6dce3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca2139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 0x55ad6df9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca2139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 0x55ad6df9e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59164/0 (socket says 172.21.15.5:59164) 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:40198/0 (socket says 172.21.15.5:40198) 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca2139b700 1 -- 172.21.15.5:0/666624787 learned_addr learned my addr 172.21.15.5:0/666624787 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca20b9a700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 0x55ad6df9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 -- 172.21.15.5:0/666624787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 msgr2=0x55ad6df9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 0x55ad6df9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 -- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 msgr2=0x55ad6df9eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 0x55ad6df9eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca21b9c700 1 -- 172.21.15.5:0/666624787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a6280 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.776+0000 7fca20b9a700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 0x55ad6df9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca21b9c700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca14009fd0 tx=0x7fca14004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e197db3beca2f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1400e040 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad6df8a250 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fca14008e20 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca14017cb0 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca36f0ab80 10 client.26672 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.777+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ad6dd14de0 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad6df8a250 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fca14004440 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 10 client.26672 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 20 client.26672 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6dfb16c0 0x55ad6dfb3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 4 client.26672 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad6dee2e10 con 0x55ad6dfb16c0 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.778+0000 7fca2139b700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6dfb16c0 0x55ad6dfb3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.779+0000 7fca04ff9700 10 client.26672 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.779+0000 7fca2139b700 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6dfb16c0 0x55ad6dfb3ae0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19a8962eafc22fc server_cookie=33813aea04817dc5 in_seq=0 out_seq=0 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca04ff9700 1 -- 172.21.15.5:0/666624787 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ad6dee2e10 con 0x55ad6dfb16c0 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca04ff9700 10 client.26672 handle_command_reply: tid=0 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 client.26672 shutdown 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 msgr2=0x55ad6df9f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca14009fd0 tx=0x7fca14004ad0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 shutdown_connections 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6dfb16c0 0x55ad6dfb3ae0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9f3f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96400 0x55ad6df9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/666624787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96d20 0x55ad6df9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca04ff9700 0 client.26672 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 >> 172.21.15.5:0/666624787 conn(0x55ad6df0b260 msgr2=0x55ad6df92320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 shutdown_connections 2023-09-23T04:50:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 1 -- 172.21.15.5:0/666624787 wait complete. 2023-09-23T04:50:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.780+0000 7fca36f0ab80 20 client.26672 trim_cache size 0 max 16384 2023-09-23T04:50:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.802+0000 7fca36f0ab80 1 Processor -- start 2023-09-23T04:50:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 -- start start 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6dd6a920 con 0x55ad6df96400 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6df96a30 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6df9a7e0 con 0x55ad6df96d20 2023-09-23T04:50:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca21b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59180/0 (socket says 172.21.15.5:59180) 2023-09-23T04:50:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.803+0000 7fca20b9a700 1 -- 172.21.15.5:0/828619214 learned_addr learned my addr 172.21.15.5:0/828619214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca2139b700 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 -- 172.21.15.5:0/828619214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 msgr2=0x55ad6df9bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 -- 172.21.15.5:0/828619214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 msgr2=0x55ad6df9f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df9f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 -- 172.21.15.5:0/828619214 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad6dcfdc60 con 0x55ad6df96d20 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca2139b700 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca20b9a700 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00efd0 tx=0x7fca0c09a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb2a4169f9eb7f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.804+0000 7fca07fff700 1 -- 172.21.15.5:0/828619214 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0b0040 con 0x55ad6df96d20 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.805+0000 7fca07fff700 1 -- 172.21.15.5:0/828619214 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c0a27e0 con 0x55ad6df96d20 2023-09-23T04:50:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.805+0000 7fca07fff700 1 -- 172.21.15.5:0/828619214 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c09e170 con 0x55ad6df96d20 2023-09-23T04:50:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.805+0000 7fca36f0ab80 1 -- 172.21.15.5:0/828619214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 msgr2=0x55ad6df9d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.805+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00efd0 tx=0x7fca0c09a040 comp rx=0 tx=0).stop 2023-09-23T04:50:05.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 -- 172.21.15.5:0/828619214 shutdown_connections 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6dfa1bd0 0x55ad6df9bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6df9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/828619214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6df96d20 0x55ad6df9d830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 -- 172.21.15.5:0/828619214 >> 172.21.15.5:0/828619214 conn(0x55ad6df0b260 msgr2=0x55ad6df9d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 -- 172.21.15.5:0/828619214 shutdown_connections 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 -- 172.21.15.5:0/828619214 wait complete. 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.806+0000 7fca36f0ab80 1 -- start start 2023-09-23T04:50:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 Processor -- start 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 0x55ad6dfa6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6dfa8410 con 0x55ad6df96400 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6dfa8580 con 0x55ad6df96d20 2023-09-23T04:50:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6dfa8740 con 0x55ad6dfa1bd0 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.807+0000 7fca36f0ab80 10 client.0 _ll_register_callbacks cb 0x7fca140093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca2139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 0x55ad6dfa6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca2139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40694/0 (socket says 172.21.15.5:40694) 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca2139b700 1 -- 172.21.15.5:0/4042053711 learned_addr learned my addr 172.21.15.5:0/4042053711 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca21b9c700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 -- 172.21.15.5:0/4042053711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 msgr2=0x55ad6dfa7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 -- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 msgr2=0x55ad6dfa6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 -- 172.21.15.5:0/4042053711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad6df4c8d0 con 0x55ad6df96d20 2023-09-23T04:50:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca21b9c700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca2139b700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca20b9a700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 0x55ad6dfa6dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55ad6df9fb40 tx=0x7fca0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3003dc6c5ae9f16a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0b0070 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fca0c09e370 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.808+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca0c09a450 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca0c09a6b0 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 10 client.26678 fetch_fsmap learned FSMap version 32 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ad6dd3b250 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0c0a6020 con 0x55ad6df96d20 2023-09-23T04:50:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fca0c0a23e0 con 0x55ad6df96d20 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 10 client.26678 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 20 client.26678 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6e00ea40 0x55ad6e010e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 4 client.26678 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.809+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad6dee2b30 con 0x55ad6e00ea40 2023-09-23T04:50:05.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca2139b700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6e00ea40 0x55ad6e010e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca2139b700 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6e00ea40 0x55ad6e010e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0a5f4e73c88de2 server_cookie=cebb5b8b2203fbb9 in_seq=0 out_seq=0 2023-09-23T04:50:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca04ff9700 10 client.26678 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca04ff9700 1 -- 172.21.15.5:0/4042053711 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ad6dee2b30 con 0x55ad6e00ea40 2023-09-23T04:50:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca04ff9700 10 client.26678 handle_command_reply: tid=0 2023-09-23T04:50:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 client.26678 shutdown 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 msgr2=0x55ad6dfa6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 0x55ad6dfa6dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55ad6df9fb40 tx=0x7fca0c0047c0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 shutdown_connections 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ad6e00ea40 0x55ad6e010e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.810+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ad6dfa1bd0 0x55ad6dfa7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca04ff9700 0 client.26678 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad6df96400 0x55ad6dfa6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 1 --2- 172.21.15.5:0/4042053711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ad6df96d20 0x55ad6dfa6dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 >> 172.21.15.5:0/4042053711 conn(0x55ad6df0b260 msgr2=0x55ad6df98b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 shutdown_connections 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 1 -- 172.21.15.5:0/4042053711 wait complete. 2023-09-23T04:50:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca36f0ab80 20 client.26678 trim_cache size 0 max 16384 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "c6e82ee9-f7cf-432f-89ed-92eeed95bc4c": { 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c6e82ee9-f7cf-432f-89ed-92eeed95bc4c", 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:50:05.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:50:05.817 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 msgr2=0x7fca080a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.811+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 0x7fca080a3c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca1400f590 tx=0x7fca1400b040 comp rx=0 tx=0).stop 2023-09-23T04:50:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 msgr2=0x7fca1c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e970 tx=0x7fca0c00ec80 comp rx=0 tx=0).stop 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 shutdown_connections 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca080a1790 0x7fca080a3c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c19dae0 0x7fca1c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca1c1010d0 0x7fca1c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 --2- 172.21.15.5:0/3156085712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca1c1061f0 0x7fca1c199680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 >> 172.21.15.5:0/3156085712 conn(0x7fca1c0fc990 msgr2=0x7fca1c10d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 shutdown_connections 2023-09-23T04:50:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:05.812+0000 7fca235ff700 1 -- 172.21.15.5:0/3156085712 wait complete. 2023-09-23T04:50:05.833 INFO:tasks.cephfs.filesystem:scrub status for tag:c6e82ee9-f7cf-432f-89ed-92eeed95bc4c - {'path': '/', 'tag': 'c6e82ee9-f7cf-432f-89ed-92eeed95bc4c', 'options': 'recursive,force'} 2023-09-23T04:50:05.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:05 smithi005 ceph-mon[97638]: pgmap v117: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2023-09-23T04:50:05.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:05 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:05.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244425 8.13 MB/sec execute 41 sec latency 17.043 ms 2023-09-23T04:50:05.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:05 smithi069 ceph-mon[133931]: pgmap v117: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2023-09-23T04:50:05.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:05 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:05 smithi118 ceph-mon[134297]: pgmap v117: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 245 op/s 2023-09-23T04:50:06.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:05 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:06.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244942 8.04 MB/sec execute 42 sec latency 737.150 ms 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:06.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:06 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:06 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:07.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:06 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:07.936 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244942 7.86 MB/sec execute 43 sec latency 1737.224 ms 2023-09-23T04:50:07.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:07 smithi069 ceph-mon[133931]: pgmap v118: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2023-09-23T04:50:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:07 smithi005 ceph-mon[97638]: pgmap v118: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2023-09-23T04:50:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:07 smithi118 ceph-mon[134297]: pgmap v118: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2023-09-23T04:50:08.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244942 7.68 MB/sec execute 44 sec latency 2737.281 ms 2023-09-23T04:50:08.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:08 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:08.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:08 smithi069 ceph-mon[133931]: pgmap v119: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 268 op/s 2023-09-23T04:50:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:08 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:08 smithi005 ceph-mon[97638]: pgmap v119: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 268 op/s 2023-09-23T04:50:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:08 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:50:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:08 smithi118 ceph-mon[134297]: pgmap v119: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 268 op/s 2023-09-23T04:50:09.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244942 7.51 MB/sec execute 45 sec latency 3737.422 ms 2023-09-23T04:50:10.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 245879 7.42 MB/sec execute 46 sec latency 3879.269 ms 2023-09-23T04:50:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:11 smithi005 ceph-mon[97638]: pgmap v120: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2023-09-23T04:50:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:11 smithi118 ceph-mon[134297]: pgmap v120: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2023-09-23T04:50:11.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248419 7.59 MB/sec execute 47 sec latency 29.535 ms 2023-09-23T04:50:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:11 smithi069 ceph-mon[133931]: pgmap v120: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.1 MiB/s wr, 178 op/s 2023-09-23T04:50:12.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248419 7.43 MB/sec execute 48 sec latency 1029.647 ms 2023-09-23T04:50:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:12 smithi005 ceph-mon[97638]: pgmap v121: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-23T04:50:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:12 smithi118 ceph-mon[134297]: pgmap v121: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-23T04:50:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:12 smithi069 ceph-mon[133931]: pgmap v121: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-09-23T04:50:13.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248419 7.28 MB/sec execute 49 sec latency 2029.756 ms 2023-09-23T04:50:14.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248419 7.14 MB/sec execute 50 sec latency 3029.894 ms 2023-09-23T04:50:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:14 smithi005 ceph-mon[97638]: pgmap v122: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 974 KiB/s wr, 153 op/s 2023-09-23T04:50:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:14 smithi118 ceph-mon[134297]: pgmap v122: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 974 KiB/s wr, 153 op/s 2023-09-23T04:50:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:14 smithi069 ceph-mon[133931]: pgmap v122: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 974 KiB/s wr, 153 op/s 2023-09-23T04:50:15.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 249243 7.04 MB/sec execute 51 sec latency 3169.113 ms 2023-09-23T04:50:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:15 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:16.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:15 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:15 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:16.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:15 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:15 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:16.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:15 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:16.937 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251819 7.22 MB/sec execute 52 sec latency 10.500 ms 2023-09-23T04:50:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:16 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:16 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:16 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:16 smithi005 ceph-mon[97638]: pgmap v123: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 780 KiB/s wr, 106 op/s 2023-09-23T04:50:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:16 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:16 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:16 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:16 smithi118 ceph-mon[134297]: pgmap v123: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 780 KiB/s wr, 106 op/s 2023-09-23T04:50:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:16 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:16 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:16 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:16 smithi069 ceph-mon[133931]: pgmap v123: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 780 KiB/s wr, 106 op/s 2023-09-23T04:50:17.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251992 7.09 MB/sec execute 53 sec latency 890.212 ms 2023-09-23T04:50:18.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251992 6.96 MB/sec execute 54 sec latency 1890.310 ms 2023-09-23T04:50:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:18 smithi005 ceph-mon[97638]: pgmap v124: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-23T04:50:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:18 smithi118 ceph-mon[134297]: pgmap v124: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-23T04:50:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:18 smithi069 ceph-mon[133931]: pgmap v124: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.0 MiB/s wr, 153 op/s 2023-09-23T04:50:19.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251992 6.84 MB/sec execute 55 sec latency 2890.436 ms 2023-09-23T04:50:20.938 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252033 6.72 MB/sec execute 56 sec latency 3025.756 ms 2023-09-23T04:50:21.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:21 smithi005 ceph-mon[97638]: pgmap v125: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 529 KiB/s wr, 86 op/s 2023-09-23T04:50:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:21 smithi118 ceph-mon[134297]: pgmap v125: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 529 KiB/s wr, 86 op/s 2023-09-23T04:50:21.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252033 6.60 MB/sec execute 57 sec latency 1819.825 ms 2023-09-23T04:50:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:21 smithi069 ceph-mon[133931]: pgmap v125: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 529 KiB/s wr, 86 op/s 2023-09-23T04:50:22.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252033 6.48 MB/sec execute 58 sec latency 2819.975 ms 2023-09-23T04:50:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:22 smithi005 ceph-mon[97638]: pgmap v126: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.2 MiB/s wr, 144 op/s 2023-09-23T04:50:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:22 smithi118 ceph-mon[134297]: pgmap v126: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.2 MiB/s wr, 144 op/s 2023-09-23T04:50:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:22 smithi069 ceph-mon[133931]: pgmap v126: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 2.2 MiB/s wr, 144 op/s 2023-09-23T04:50:23.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252033 6.37 MB/sec execute 59 sec latency 3820.111 ms 2023-09-23T04:50:24.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252033 6.27 MB/sec execute 60 sec latency 4820.258 ms 2023-09-23T04:50:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:24 smithi005 ceph-mon[97638]: pgmap v127: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2023-09-23T04:50:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:24 smithi118 ceph-mon[134297]: pgmap v127: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2023-09-23T04:50:25.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:24 smithi069 ceph-mon[133931]: pgmap v127: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2023-09-23T04:50:25.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252869 6.20 MB/sec execute 61 sec latency 4956.100 ms 2023-09-23T04:50:26.939 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255204 6.32 MB/sec execute 62 sec latency 27.868 ms 2023-09-23T04:50:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:26 smithi005 ceph-mon[97638]: pgmap v128: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2023-09-23T04:50:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:26 smithi118 ceph-mon[134297]: pgmap v128: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2023-09-23T04:50:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:26 smithi069 ceph-mon[133931]: pgmap v128: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2023-09-23T04:50:27.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255429 6.27 MB/sec execute 63 sec latency 885.275 ms 2023-09-23T04:50:28.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255429 6.17 MB/sec execute 64 sec latency 1885.407 ms 2023-09-23T04:50:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:28 smithi005 ceph-mon[97638]: pgmap v129: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2023-09-23T04:50:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:28 smithi118 ceph-mon[134297]: pgmap v129: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2023-09-23T04:50:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:28 smithi069 ceph-mon[133931]: pgmap v129: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2023-09-23T04:50:29.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255429 6.07 MB/sec execute 65 sec latency 2885.471 ms 2023-09-23T04:50:30.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:50:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:50:30.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:50:30.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255510 5.99 MB/sec execute 66 sec latency 3017.034 ms 2023-09-23T04:50:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:30 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:50:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:30 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:50:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:30 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:50:31.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:30 smithi005 ceph-mon[97638]: pgmap v130: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.3 MiB/s wr, 154 op/s 2023-09-23T04:50:31.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:30 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:50:31.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:30 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:50:31.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:30 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:50:31.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:30 smithi118 ceph-mon[134297]: pgmap v130: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.3 MiB/s wr, 154 op/s 2023-09-23T04:50:31.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:30 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:50:31.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:30 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:50:31.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:30 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:50:31.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:30 smithi069 ceph-mon[133931]: pgmap v130: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 3.3 MiB/s wr, 154 op/s 2023-09-23T04:50:31.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255510 5.90 MB/sec execute 67 sec latency 1829.321 ms 2023-09-23T04:50:32.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255510 5.81 MB/sec execute 68 sec latency 2829.461 ms 2023-09-23T04:50:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:32 smithi005 ceph-mon[97638]: pgmap v131: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.3 MiB/s wr, 163 op/s 2023-09-23T04:50:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:32 smithi118 ceph-mon[134297]: pgmap v131: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.3 MiB/s wr, 163 op/s 2023-09-23T04:50:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:32 smithi069 ceph-mon[133931]: pgmap v131: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.3 MiB/s wr, 163 op/s 2023-09-23T04:50:33.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255510 5.73 MB/sec execute 69 sec latency 3829.591 ms 2023-09-23T04:50:34.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255510 5.64 MB/sec execute 70 sec latency 4829.735 ms 2023-09-23T04:50:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:34 smithi005 ceph-mon[97638]: pgmap v132: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2023-09-23T04:50:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:34 smithi118 ceph-mon[134297]: pgmap v132: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2023-09-23T04:50:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:34 smithi069 ceph-mon[133931]: pgmap v132: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2023-09-23T04:50:35.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:50:35.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 256364 5.60 MB/sec execute 71 sec latency 4964.677 ms 2023-09-23T04:50:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 -- 172.21.15.5:0/680983362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658101a50 msgr2=0x7f7658105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 --2- 172.21.15.5:0/680983362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658101a50 0x7f7658105a90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76440098f0 tx=0x7f7644009c00 comp rx=0 tx=0).stop 2023-09-23T04:50:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 -- 172.21.15.5:0/680983362 shutdown_connections 2023-09-23T04:50:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 --2- 172.21.15.5:0/680983362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f765810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 --2- 172.21.15.5:0/680983362 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 --2- 172.21.15.5:0/680983362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658101a50 0x7f7658105a90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 -- 172.21.15.5:0/680983362 >> 172.21.15.5:0/680983362 conn(0x7f76580fc920 msgr2=0x7f76580fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 -- 172.21.15.5:0/680983362 shutdown_connections 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.060+0000 7f765f1ff700 1 -- 172.21.15.5:0/680983362 wait complete. 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.061+0000 7f765f1ff700 1 Processor -- start 2023-09-23T04:50:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.061+0000 7f765f1ff700 1 -- start start 2023-09-23T04:50:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.061+0000 7f765f1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.061+0000 7f765f1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f7658073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765f1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658073ba0 0x7f76581a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765f1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5810 con 0x7f76581061c0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765f1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5980 con 0x7f7658073ba0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765f1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5b40 con 0x7f76581010a0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f7657fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f7658073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33870/0 (socket says 172.21.15.5:33870) 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765d79c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658073ba0 0x7f76581a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 -- 172.21.15.5:0/4075626814 learned_addr learned my addr 172.21.15.5:0/4075626814 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 -- 172.21.15.5:0/4075626814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658073ba0 msgr2=0x7f76581a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658073ba0 0x7f76581a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 -- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 msgr2=0x7f7658073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f7658073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 -- 172.21.15.5:0/4075626814 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644009520 con 0x7f76581010a0 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f7657fff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f7658073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.062+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e9c0 tx=0x7f764c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.063+0000 7f7655ffb700 1 -- 172.21.15.5:0/4075626814 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c004a30 con 0x7f76581010a0 2023-09-23T04:50:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.063+0000 7f7655ffb700 1 -- 172.21.15.5:0/4075626814 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764c00cc80 con 0x7f76581010a0 2023-09-23T04:50:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.063+0000 7f7655ffb700 1 -- 172.21.15.5:0/4075626814 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0054e0 con 0x7f76581010a0 2023-09-23T04:50:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.063+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76581a5e20 con 0x7f76581010a0 2023-09-23T04:50:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.063+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a61f0 con 0x7f76581010a0 2023-09-23T04:50:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.066+0000 7f7655ffb700 1 -- 172.21.15.5:0/4075626814 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f764c004b90 con 0x7f76581010a0 2023-09-23T04:50:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.068+0000 7f7672ae8b80 1 Processor -- start 2023-09-23T04:50:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 -- start start 2023-09-23T04:50:36.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7655ffb700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 0x7f76480a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4d84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 0x5627e4d86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4ce2460 con 0x5627e4d79d20 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4cd3c20 con 0x5627e4d79400 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4cd1540 con 0x5627e4d84bd0 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.069+0000 7f7655ffb700 1 -- 172.21.15.5:0/4075626814 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f764c014070 con 0x7f76581010a0 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765d79c700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 0x7f76480a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48836/0 (socket says 172.21.15.5:48836) 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 -- 172.21.15.5:0/2420436924 learned_addr learned my addr 172.21.15.5:0/2420436924 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f7657fff700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4d84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 -- 172.21.15.5:0/2420436924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 msgr2=0x5627e4d86fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765d79c700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 0x5627e4d86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 0x5627e4d86fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 -- 172.21.15.5:0/2420436924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 msgr2=0x5627e4d84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4d84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 -- 172.21.15.5:0/2420436924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e4d2f8d0 con 0x5627e4d79400 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f7657fff700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4d84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765d79c700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 0x5627e4d86fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f764c0a2110 tx=0x7f764c099c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11dfd44eb47f5c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f765d79c700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 0x7f76480a3bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7650009ec0 tx=0x7f7650009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.070+0000 7f7642ffd700 1 -- 172.21.15.5:0/2420436924 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00cdf0 con 0x5627e4d79400 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7642ffd700 1 -- 172.21.15.5:0/2420436924 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764c09a9b0 con 0x5627e4d79400 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7642ffd700 1 -- 172.21.15.5:0/2420436924 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c098480 con 0x5627e4d79400 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2420436924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 msgr2=0x5627e4d797e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f764c0a2110 tx=0x7f764c099c90 comp rx=0 tx=0).stop 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2420436924 shutdown_connections 2023-09-23T04:50:36.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d84bd0 0x5627e4d86fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79400 0x5627e4d797e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2420436924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4d84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2420436924 >> 172.21.15.5:0/2420436924 conn(0x5627e4cee260 msgr2=0x5627e4d75790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2420436924 shutdown_connections 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.071+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2420436924 wait complete. 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.072+0000 7f7672ae8b80 1 -- start start 2023-09-23T04:50:36.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.072+0000 7f7672ae8b80 1 Processor -- start 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 0x5627e4d81920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4d823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4ccce30 con 0x5627e4d79400 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4c69710 con 0x5627e4d84bd0 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4c693b0 con 0x5627e4d79d20 2023-09-23T04:50:36.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7672ae8b80 10 client.0 _ll_register_callbacks cb 0x5627e4ac6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33882/0 (socket says 172.21.15.5:33882) 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 -- 172.21.15.5:0/2696642313 learned_addr learned my addr 172.21.15.5:0/2696642313 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f765d79c700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4d823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 0x5627e4d81920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 -- 172.21.15.5:0/2696642313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 msgr2=0x5627e4d823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4d823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 -- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 msgr2=0x5627e4d81920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.073+0000 7f7657fff700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 0x5627e4d81920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7657fff700 1 -- 172.21.15.5:0/2696642313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a61f0 con 0x5627e4d79d20 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 0x5627e4d81920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f765d79c700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4d823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7657fff700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7644005e90 tx=0x7f764400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3df5f87cc76fb02 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7637fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764401c040 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627e4d6d250 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f764400bb30 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7644005040 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7672ae8b80 10 client.26743 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.074+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5627e4af7de0 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627e4d6d250 con 0x5627e4d79d20 2023-09-23T04:50:36.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7644015020 con 0x5627e4d79d20 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 10 client.26743 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 10 client.26743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 20 client.26743 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4d94600 0x5627e4d96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 4 client.26743 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627e4cc5e10 con 0x5627e4d94600 2023-09-23T04:50:36.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.075+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4d94600 0x5627e4d96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.076+0000 7f7637fff700 10 client.26743 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:36.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.076+0000 7f765cf9b700 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4d94600 0x5627e4d96a20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a17cdd359366eb5 server_cookie=de4c43d948c75de2 in_seq=0 out_seq=0 2023-09-23T04:50:36.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7637fff700 1 -- 172.21.15.5:0/2696642313 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5627e4cc5e10 con 0x5627e4d94600 2023-09-23T04:50:36.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7637fff700 10 client.26743 handle_command_reply: tid=0 2023-09-23T04:50:36.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 client.26743 shutdown 2023-09-23T04:50:36.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 msgr2=0x5627e4d81e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7644005e90 tx=0x7f764400bef0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 shutdown_connections 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4d94600 0x5627e4d96a20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4d823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79400 0x5627e4d81920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2696642313 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79d20 0x5627e4d81e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7637fff700 0 client.26743 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 >> 172.21.15.5:0/2696642313 conn(0x5627e4cee260 msgr2=0x5627e4d75320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 shutdown_connections 2023-09-23T04:50:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.079+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2696642313 wait complete. 2023-09-23T04:50:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.080+0000 7f7672ae8b80 20 client.26743 trim_cache size 0 max 16384 2023-09-23T04:50:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 Processor -- start 2023-09-23T04:50:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 -- start start 2023-09-23T04:50:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4d7c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 0x5627e4d8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d84bd0 0x5627e4d7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4b4d920 con 0x5627e4d84bd0 2023-09-23T04:50:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4d7aef0 con 0x5627e4d79d20 2023-09-23T04:50:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4d745b0 con 0x5627e4d79400 2023-09-23T04:50:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.099+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4d7c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4d7c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33886/0 (socket says 172.21.15.5:33886) 2023-09-23T04:50:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f765cf9b700 1 -- 172.21.15.5:0/2835188801 learned_addr learned my addr 172.21.15.5:0/2835188801 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 0x5627e4d8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f765d79c700 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d84bd0 0x5627e4d7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 -- 172.21.15.5:0/2835188801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 msgr2=0x5627e4d7c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4d7c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 -- 172.21.15.5:0/2835188801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d84bd0 msgr2=0x5627e4d7eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d84bd0 0x5627e4d7eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 -- 172.21.15.5:0/2835188801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e4d71480 con 0x5627e4d79d20 2023-09-23T04:50:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f7657fff700 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 0x5627e4d8b720 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7644009810 tx=0x7f7644009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28a5c2b692bab67 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f76437fe700 1 -- 172.21.15.5:0/2835188801 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764401c040 con 0x5627e4d79d20 2023-09-23T04:50:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f76437fe700 1 -- 172.21.15.5:0/2835188801 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7644005920 con 0x5627e4d79d20 2023-09-23T04:50:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.100+0000 7f76437fe700 1 -- 172.21.15.5:0/2835188801 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644003680 con 0x5627e4d79d20 2023-09-23T04:50:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2835188801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 msgr2=0x5627e4d8b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 0x5627e4d8b720 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7644009810 tx=0x7f7644009c00 comp rx=0 tx=0).stop 2023-09-23T04:50:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2835188801 shutdown_connections 2023-09-23T04:50:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d84bd0 0x5627e4d7eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4d7c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/2835188801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d79d20 0x5627e4d8b720 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2835188801 >> 172.21.15.5:0/2835188801 conn(0x5627e4cee260 msgr2=0x5627e4d7cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2835188801 shutdown_connections 2023-09-23T04:50:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.101+0000 7f7672ae8b80 1 -- 172.21.15.5:0/2835188801 wait complete. 2023-09-23T04:50:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.102+0000 7f7672ae8b80 1 -- start start 2023-09-23T04:50:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.102+0000 7f7672ae8b80 1 Processor -- start 2023-09-23T04:50:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4ddcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4de7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4de79a0 con 0x5627e4d79d20 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4de7ae0 con 0x5627e4d84bd0 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e4de7c20 con 0x5627e4d79400 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7672ae8b80 10 client.0 _ll_register_callbacks cb 0x5627e4ac6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33892/0 (socket says 172.21.15.5:33892) 2023-09-23T04:50:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 -- 172.21.15.5:0/4192197963 learned_addr learned my addr 172.21.15.5:0/4192197963 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7657fff700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4ddcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765d79c700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4de7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 -- 172.21.15.5:0/4192197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 msgr2=0x5627e4de7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4de7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 -- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 msgr2=0x5627e4ddcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4ddcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f765cf9b700 1 -- 172.21.15.5:0/4192197963 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627e4af7de0 con 0x5627e4d79400 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.103+0000 7f7657fff700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4ddcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f765cf9b700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f764c09a240 tx=0x7f764c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b138b7a5c061caf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c09e400 con 0x5627e4d79400 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f764c0983f0 con 0x5627e4d79400 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f764c09a9b0 con 0x5627e4d79400 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f764c005640 con 0x5627e4d79400 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7672ae8b80 10 client.26749 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5627e4d910d0 con 0x5627e4d79400 2023-09-23T04:50:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764c09a9b0 con 0x5627e4d79400 2023-09-23T04:50:36.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.104+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f764c0cd300 con 0x5627e4d79400 2023-09-23T04:50:36.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 10 client.26749 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:36.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 10 client.26749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:36.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 20 client.26749 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:36.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4de9010 0x5627e4deb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 4 client.26749 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:50:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5627e4cc5b30 con 0x5627e4de9010 2023-09-23T04:50:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7657fff700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4de9010 0x5627e4deb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7637fff700 10 client.26749 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.105+0000 7f7657fff700 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4de9010 0x5627e4deb430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14864feaaa0f1e server_cookie=4844b5b88cdb418f in_seq=0 out_seq=0 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7637fff700 1 -- 172.21.15.5:0/4192197963 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5627e4cc5b30 con 0x5627e4de9010 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7637fff700 10 client.26749 handle_command_reply: tid=0 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 client.26749 shutdown 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 msgr2=0x5627e4ddc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f764c09a240 tx=0x7f764c000bc0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 shutdown_connections 2023-09-23T04:50:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5627e4de9010 0x5627e4deb430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627e4d84bd0 0x5627e4de7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5627e4d79400 0x5627e4ddc5d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7637fff700 0 client.26749 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 --2- 172.21.15.5:0/4192197963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627e4d79d20 0x5627e4ddcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 >> 172.21.15.5:0/4192197963 conn(0x5627e4cee260 msgr2=0x5627e4d83230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 shutdown_connections 2023-09-23T04:50:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 1 -- 172.21.15.5:0/4192197963 wait complete. 2023-09-23T04:50:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.106+0000 7f7672ae8b80 20 client.26749 trim_cache size 0 max 16384 2023-09-23T04:50:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:36.124 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:50:36.124 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:50:36.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 msgr2=0x7f76480a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 0x7f76480a3bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7650009ec0 tx=0x7f7650009920 comp rx=0 tx=0).stop 2023-09-23T04:50:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 msgr2=0x7f7658073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e9c0 tx=0x7f764c00ecd0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 shutdown_connections 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76480a1710 0x7f76480a3bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7658073ba0 0x7f76581a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010a0 0x7f7658073120 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 --2- 172.21.15.5:0/4075626814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581061c0 0x7f7658073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 >> 172.21.15.5:0/4075626814 conn(0x7f76580fc920 msgr2=0x7f76580fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 shutdown_connections 2023-09-23T04:50:36.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.107+0000 7f765f1ff700 1 -- 172.21.15.5:0/4075626814 wait complete. 2023-09-23T04:50:36.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:50:36.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.374+0000 7fb3565ff700 1 -- 172.21.15.5:0/1611002834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068dc0 msgr2=0x7fb350105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.374+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1611002834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068dc0 0x7fb350105aa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb344009990 tx=0x7fb344009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 -- 172.21.15.5:0/1611002834 shutdown_connections 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1611002834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3501061d0 0x7fb35010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1611002834 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb350068410 0x7fb3500687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1611002834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068dc0 0x7fb350105aa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 -- 172.21.15.5:0/1611002834 >> 172.21.15.5:0/1611002834 conn(0x7fb350076ea0 msgr2=0x7fb3500772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 -- 172.21.15.5:0/1611002834 shutdown_connections 2023-09-23T04:50:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.375+0000 7fb3565ff700 1 -- 172.21.15.5:0/1611002834 wait complete. 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.376+0000 7fb3565ff700 1 Processor -- start 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.376+0000 7fb3565ff700 1 -- start start 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 0x7fb350196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3501061d0 0x7fb3501974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019c3a0 con 0x7fb35019b920 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019c510 con 0x7fb350068410 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb3565ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019c6d0 con 0x7fb3501061d0 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51798/0 (socket says 172.21.15.5:51798) 2023-09-23T04:50:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 -- 172.21.15.5:0/1950021888 learned_addr learned my addr 172.21.15.5:0/1950021888 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb34ffff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 0x7fb350196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb34f7fe700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3501061d0 0x7fb3501974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 -- 172.21.15.5:0/1950021888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3501061d0 msgr2=0x7fb3501974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3501061d0 0x7fb3501974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 -- 172.21.15.5:0/1950021888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 msgr2=0x7fb350196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 0x7fb350196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb354b9c700 1 -- 172.21.15.5:0/1950021888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440095f0 con 0x7fb35019b920 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.377+0000 7fb34ffff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 0x7fb350196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb354b9c700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800dd20 con 0x7fb35019b920 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3500ffcf0 con 0x7fb35019b920 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb34800f040 con 0x7fb35019b920 2023-09-23T04:50:36.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480048a0 con 0x7fb35019b920 2023-09-23T04:50:36.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.378+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb350100200 con 0x7fb35019b920 2023-09-23T04:50:36.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.379+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb350067ca0 con 0x7fb35019b920 2023-09-23T04:50:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.382+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb348003a00 con 0x7fb35019b920 2023-09-23T04:50:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.383+0000 7fb34d7fa700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 0x7fb3380a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.384+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fb3480ccec0 con 0x7fb35019b920 2023-09-23T04:50:36.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.384+0000 7fb34ffff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 0x7fb3380a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.384+0000 7fb34ffff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 0x7fb3380a3bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb340011490 tx=0x7fb340009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.388+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fb34804eec0 con 0x7fb35019b920 2023-09-23T04:50:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.675+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb350068dc0 con 0x7fb35019b920 2023-09-23T04:50:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.676+0000 7fb34d7fa700 1 -- 172.21.15.5:0/1950021888 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11354 (secure 0 0 0) 0x7fb348095d90 con 0x7fb35019b920 2023-09-23T04:50:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:50:36.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:50:07.224731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 msgr2=0x7fb3380a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 0x7fb3380a3bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb340011490 tx=0x7fb340009260 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 msgr2=0x7fb35019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 shutdown_connections 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb3380a1710 0x7fb3380a3bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb35019b920 0x7fb35019bda0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb350068410 0x7fb350196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 --2- 172.21.15.5:0/1950021888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb3501061d0 0x7fb3501974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 >> 172.21.15.5:0/1950021888 conn(0x7fb350076ea0 msgr2=0x7fb3500fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 shutdown_connections 2023-09-23T04:50:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.679+0000 7fb3565ff700 1 -- 172.21.15.5:0/1950021888 wait complete. 2023-09-23T04:50:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2023-09-23T04:50:36.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.924+0000 7fe39d7ff700 1 -- 172.21.15.5:0/3411751263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3981020c0 msgr2=0x7fe3981024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.924+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/3411751263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3981020c0 0x7fe3981024a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe388009990 tx=0x7fe388009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 -- 172.21.15.5:0/3411751263 shutdown_connections 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/3411751263 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe39810b490 0x7fe39810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/3411751263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3981020c0 0x7fe3981024a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/3411751263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 0x7fe39810aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 -- 172.21.15.5:0/3411751263 >> 172.21.15.5:0/3411751263 conn(0x7fe398076f30 msgr2=0x7fe398077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 -- 172.21.15.5:0/3411751263 shutdown_connections 2023-09-23T04:50:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.925+0000 7fe39d7ff700 1 -- 172.21.15.5:0/3411751263 wait complete. 2023-09-23T04:50:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 Processor -- start 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 -- start start 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 0x7fe398100250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810b490 0x7fe3981a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398100a80 con 0x7fe3981029e0 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3981a5750 con 0x7fe39810b490 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.926+0000 7fe39d7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3981a5890 con 0x7fe3981020c0 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33922/0 (socket says 172.21.15.5:33922) 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 -- 172.21.15.5:0/2717747030 learned_addr learned my addr 172.21.15.5:0/2717747030 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe3967fc700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 0x7fe398100250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe3977fe700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810b490 0x7fe3981a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 -- 172.21.15.5:0/2717747030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810b490 msgr2=0x7fe3981a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810b490 0x7fe3981a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 -- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 msgr2=0x7fe398100250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 0x7fe398100250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 -- 172.21.15.5:0/2717747030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880095f0 con 0x7fe3981020c0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.927+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe388005910 tx=0x7fe388004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.928+0000 7fe37ffff700 1 -- 172.21.15.5:0/2717747030 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38801d070 con 0x7fe3981020c0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.928+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3981a5a90 con 0x7fe3981020c0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.928+0000 7fe37ffff700 1 -- 172.21.15.5:0/2717747030 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe388022470 con 0x7fe3981020c0 2023-09-23T04:50:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.928+0000 7fe37ffff700 1 -- 172.21.15.5:0/2717747030 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388017650 con 0x7fe3981020c0 2023-09-23T04:50:36.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.928+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981a5f10 con 0x7fe3981020c0 2023-09-23T04:50:36.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.931+0000 7fe37ffff700 1 -- 172.21.15.5:0/2717747030 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe3880225e0 con 0x7fe3981020c0 2023-09-23T04:50:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.933+0000 7fe37ffff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 0x7fe3800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.933+0000 7fe3b11c6b80 1 Processor -- start 2023-09-23T04:50:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.933+0000 7fe37ffff700 1 -- 172.21.15.5:0/2717747030 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe3880ce910 con 0x7fe3981020c0 2023-09-23T04:50:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.933+0000 7fe3967fc700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 0x7fe3800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.933+0000 7fe3b11c6b80 1 -- start start 2023-09-23T04:50:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb11c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb11c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1121460 con 0x557eb11b8d20 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1112c20 con 0x557eb11c3bd0 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1110540 con 0x557eb11b8400 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3967fc700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 0x7fe3800a3bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe398101500 tx=0x7fe38c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33932/0 (socket says 172.21.15.5:33932) 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 -- 172.21.15.5:0/823754798 learned_addr learned my addr 172.21.15.5:0/823754798 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3977fe700 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb11c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe3967fc700 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb11c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 -- 172.21.15.5:0/823754798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 msgr2=0x557eb11c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb11c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 -- 172.21.15.5:0/823754798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 msgr2=0x557eb11c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb11c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.934+0000 7fe396ffd700 1 -- 172.21.15.5:0/823754798 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb116e8d0 con 0x557eb11b8400 2023-09-23T04:50:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.935+0000 7fe396ffd700 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38809b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2103e14e076aafb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.935+0000 7fe37d7fa700 1 -- 172.21.15.5:0/823754798 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38809bbd0 con 0x557eb11b8400 2023-09-23T04:50:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.935+0000 7fe37d7fa700 1 -- 172.21.15.5:0/823754798 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38809b4d0 con 0x557eb11b8400 2023-09-23T04:50:36.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.935+0000 7fe37d7fa700 1 -- 172.21.15.5:0/823754798 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38809b990 con 0x557eb11b8400 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/823754798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 msgr2=0x557eb11b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe388006010 tx=0x7fe38809b370 comp rx=0 tx=0).stop 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/823754798 shutdown_connections 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb11c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb11b87e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/823754798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb11c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/823754798 >> 172.21.15.5:0/823754798 conn(0x557eb112d260 msgr2=0x557eb11b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/823754798 shutdown_connections 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/823754798 wait complete. 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.936+0000 7fe3b11c6b80 1 -- start start 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 Processor -- start 2023-09-23T04:50:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb110be30 con 0x557eb11b8400 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb10a8710 con 0x557eb11b8d20 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb10a83b0 con 0x557eb11c3bd0 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.937+0000 7fe3b11c6b80 10 client.0 _ll_register_callbacks cb 0x557eb0f05800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33942/0 (socket says 172.21.15.5:33942) 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 -- 172.21.15.5:0/2760973657 learned_addr learned my addr 172.21.15.5:0/2760973657 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3967fc700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 -- 172.21.15.5:0/2760973657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 msgr2=0x557eb11c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 -- 172.21.15.5:0/2760973657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 msgr2=0x557eb11c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 -- 172.21.15.5:0/2760973657 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981a5f10 con 0x557eb11c3bd0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe3977fe700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe390011ef0 tx=0x7fe390014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd37a14d69a6951 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.938+0000 7fe37aff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000e040 con 0x557eb11c3bd0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557eb11ac250 con 0x557eb11c3bd0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe3900093a0 con 0x557eb11c3bd0 2023-09-23T04:50:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe390014ce0 con 0x557eb11c3bd0 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 10 client.26767 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557eb0f36de0 con 0x557eb11c3bd0 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557eb11ac250 con 0x557eb11c3bd0 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe39001c020 con 0x557eb11c3bd0 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 10 client.26767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.939+0000 7fe3b11c6b80 20 client.26767 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb11d32f0 0x557eb11d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe3b11c6b80 4 client.26767 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557eb1104e10 con 0x557eb11d32f0 2023-09-23T04:50:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb11d32f0 0x557eb11d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe37aff5700 10 client.26767 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.940+0000 7fe396ffd700 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb11d32f0 0x557eb11d5710 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10da02476e2f921b server_cookie=3824a354fa2c7275 in_seq=0 out_seq=0 2023-09-23T04:50:36.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 258616 5.69 MB/sec execute 72 sec latency 17.041 ms 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe37aff5700 1 -- 172.21.15.5:0/2760973657 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557eb1104e10 con 0x557eb11d32f0 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe37aff5700 10 client.26767 handle_command_reply: tid=0 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe3b11c6b80 1 client.26767 shutdown 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 msgr2=0x557eb11c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe390011ef0 tx=0x7fe390014600 comp rx=0 tx=0).stop 2023-09-23T04:50:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.941+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 shutdown_connections 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11c4a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb11d32f0 0x557eb11d5710 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/2760973657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe37aff5700 0 client.26767 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 >> 172.21.15.5:0/2760973657 conn(0x557eb112d260 msgr2=0x557eb11b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 shutdown_connections 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/2760973657 wait complete. 2023-09-23T04:50:36.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.942+0000 7fe3b11c6b80 20 client.26767 trim_cache size 0 max 16384 2023-09-23T04:50:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.957+0000 7fe3b11c6b80 1 Processor -- start 2023-09-23T04:50:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.957+0000 7fe3b11c6b80 1 -- start start 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11bd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb0f8c920 con 0x557eb11b8400 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb11abd90 con 0x557eb11b8d20 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb11c1190 con 0x557eb11c3bd0 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3967fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11bd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3967fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11bd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48902/0 (socket says 172.21.15.5:48902) 2023-09-23T04:50:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe3967fc700 1 -- 172.21.15.5:0/3600460610 learned_addr learned my addr 172.21.15.5:0/3600460610 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.958+0000 7fe396ffd700 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 -- 172.21.15.5:0/3600460610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 msgr2=0x557eb11bd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11bd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 -- 172.21.15.5:0/3600460610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 msgr2=0x557eb11c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 -- 172.21.15.5:0/3600460610 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb0f1fc60 con 0x557eb11c3bd0 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe396ffd700 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe3977fe700 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11bfea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b010 tx=0x7fe390009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8998d7837c78ef32 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe37dffb700 1 -- 172.21.15.5:0/3600460610 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000e040 con 0x557eb11c3bd0 2023-09-23T04:50:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe37dffb700 1 -- 172.21.15.5:0/3600460610 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe39000f040 con 0x557eb11c3bd0 2023-09-23T04:50:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.959+0000 7fe37dffb700 1 -- 172.21.15.5:0/3600460610 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390007900 con 0x557eb11c3bd0 2023-09-23T04:50:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/3600460610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 msgr2=0x557eb11bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11bfea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b010 tx=0x7fe390009dd0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/3600460610 shutdown_connections 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11c3bd0 0x557eb11bfea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8400 0x557eb11c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/3600460610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11b8d20 0x557eb11bd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/3600460610 >> 172.21.15.5:0/3600460610 conn(0x557eb112d260 msgr2=0x557eb11bdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/3600460610 shutdown_connections 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.960+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/3600460610 wait complete. 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 -- start start 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 Processor -- start 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb121b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb1220130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb1220670 con 0x557eb11b8d20 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb12207e0 con 0x557eb11c3bd0 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb12209a0 con 0x557eb11b8400 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.961+0000 7fe3b11c6b80 10 client.0 _ll_register_callbacks cb 0x7fe390011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb1220130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe3967fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb121b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33962/0 (socket says 172.21.15.5:33962) 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 -- 172.21.15.5:0/482611081 learned_addr learned my addr 172.21.15.5:0/482611081 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 -- 172.21.15.5:0/482611081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 msgr2=0x557eb1220130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb1220130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 -- 172.21.15.5:0/482611081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 msgr2=0x557eb121b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb121b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 -- 172.21.15.5:0/482611081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557eb0f36de0 con 0x557eb11b8400 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe3977fe700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb1220130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe396ffd700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe38809b1d0 tx=0x7fe388005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83b6c4c099aebb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.962+0000 7fe37aff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880b3070 con 0x557eb11b8400 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fe38809f620 con 0x557eb11b8400 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe38809b4d0 con 0x557eb11b8400 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe38809bc50 con 0x557eb11b8400 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 10 client.26773 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557eb0f5d250 con 0x557eb11b8400 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38809b4d0 con 0x557eb11b8400 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe3880a3340 con 0x557eb11b8400 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 10 client.26773 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 20 client.26773 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb1227d70 0x557eb122a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 4 client.26773 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.963+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557eb1104b30 con 0x557eb1227d70 2023-09-23T04:50:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe3967fc700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb1227d70 0x557eb122a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe3967fc700 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb1227d70 0x557eb122a190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc8314a41176f39c server_cookie=61b24695f7a8942b in_seq=0 out_seq=0 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe37aff5700 10 client.26773 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe37aff5700 1 -- 172.21.15.5:0/482611081 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557eb1104b30 con 0x557eb1227d70 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe37aff5700 10 client.26773 handle_command_reply: tid=0 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.964+0000 7fe3b11c6b80 1 client.26773 shutdown 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 msgr2=0x557eb121b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe38809b1d0 tx=0x7fe388005ea0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 shutdown_connections 2023-09-23T04:50:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557eb11c3bd0 0x557eb1220130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557eb11b8400 0x557eb121b260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x557eb1227d70 0x557eb122a190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 --2- 172.21.15.5:0/482611081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eb11b8d20 0x557eb121b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe37aff5700 0 client.26773 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 >> 172.21.15.5:0/482611081 conn(0x557eb112d260 msgr2=0x557eb11c13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 shutdown_connections 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 1 -- 172.21.15.5:0/482611081 wait complete. 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.965+0000 7fe3b11c6b80 20 client.26773 trim_cache size 0 max 16384 2023-09-23T04:50:36.970 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 msgr2=0x7fe3800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 0x7fe3800a3bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe398101500 tx=0x7fe38c009260 comp rx=0 tx=0).stop 2023-09-23T04:50:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 msgr2=0x7fe3980ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe388005910 tx=0x7fe388004710 comp rx=0 tx=0).stop 2023-09-23T04:50:36.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 shutdown_connections 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe3800a1710 0x7fe3800a3bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe39810b490 0x7fe3981a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe3981020c0 0x7fe3980ffd10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 --2- 172.21.15.5:0/2717747030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3981029e0 0x7fe398100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 >> 172.21.15.5:0/2717747030 conn(0x7fe398076f30 msgr2=0x7fe3980fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 shutdown_connections 2023-09-23T04:50:36.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:36.966+0000 7fe39d7ff700 1 -- 172.21.15.5:0/2717747030 wait complete. 2023-09-23T04:50:36.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:36 smithi005 ceph-mon[97638]: pgmap v133: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1021 KiB/s wr, 89 op/s 2023-09-23T04:50:36.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:36 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1950021888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:36.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:50:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:36 smithi069 ceph-mon[133931]: pgmap v133: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1021 KiB/s wr, 89 op/s 2023-09-23T04:50:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:36 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1950021888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:36 smithi118 ceph-mon[134297]: pgmap v133: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1021 KiB/s wr, 89 op/s 2023-09-23T04:50:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:36 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1950021888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:50:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 -- 172.21.15.5:0/2774659559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1020c0 msgr2=0x7f2a1c1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 --2- 172.21.15.5:0/2774659559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c1024a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009990 tx=0x7f2a0c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 -- 172.21.15.5:0/2774659559 shutdown_connections 2023-09-23T04:50:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 --2- 172.21.15.5:0/2774659559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c10b490 0x7f2a1c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 --2- 172.21.15.5:0/2774659559 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c1024a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 --2- 172.21.15.5:0/2774659559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 -- 172.21.15.5:0/2774659559 >> 172.21.15.5:0/2774659559 conn(0x7f2a1c076f30 msgr2=0x7f2a1c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 -- 172.21.15.5:0/2774659559 shutdown_connections 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.230+0000 7f2a215ff700 1 -- 172.21.15.5:0/2774659559 wait complete. 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.231+0000 7f2a215ff700 1 Processor -- start 2023-09-23T04:50:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 -- start start 2023-09-23T04:50:37.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 0x7f2a1c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c19c020 con 0x7f2a1c10b490 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c19c190 con 0x7f2a1c1020c0 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.232+0000 7f2a215ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c19c350 con 0x7f2a1c1029e0 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33968/0 (socket says 172.21.15.5:33968) 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/1741811817 learned_addr learned my addr 172.21.15.5:0/1741811817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 0x7f2a1c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1affd700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/1741811817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c1020c0 msgr2=0x7f2a1c197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 msgr2=0x7f2a1c19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 0x7f2a1c19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/1741811817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0095f0 con 0x7f2a1c1029e0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 0x7f2a1c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.233+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000e9a0 tx=0x7f2a1000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.234+0000 7f2a03fff700 1 -- 172.21.15.5:0/1741811817 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10009790 con 0x7f2a1c1029e0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.234+0000 7f2a03fff700 1 -- 172.21.15.5:0/1741811817 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a10010430 con 0x7f2a1c1029e0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.234+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c19c630 con 0x7f2a1c1029e0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.234+0000 7f2a03fff700 1 -- 172.21.15.5:0/1741811817 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10018680 con 0x7f2a1c1029e0 2023-09-23T04:50:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.234+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a3e70 con 0x7f2a1c1029e0 2023-09-23T04:50:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.237+0000 7f2a03fff700 1 -- 172.21.15.5:0/1741811817 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2a100105a0 con 0x7f2a1c1029e0 2023-09-23T04:50:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.239+0000 7f2a34f16b80 1 Processor -- start 2023-09-23T04:50:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.239+0000 7f2a03fff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 0x7f2a040a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.239+0000 7f2a34f16b80 1 -- start start 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.239+0000 7f2a03fff700 1 -- 172.21.15.5:0/1741811817 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2a10014070 con 0x7f2a1c1029e0 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1affd700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 0x7f2a040a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99d400 0x56044e99d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e906460 con 0x56044e9a8bd0 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e8f7c20 con 0x56044e99dd20 2023-09-23T04:50:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e8f5540 con 0x56044e99d400 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99d400 0x56044e99d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48948/0 (socket says 172.21.15.5:48948) 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/4288392789 learned_addr learned my addr 172.21.15.5:0/4288392789 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.240+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/4288392789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99d400 msgr2=0x56044e99d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99d400 0x56044e99d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/4288392789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 msgr2=0x56044e9aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/4288392789 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044e9538d0 con 0x56044e99dd20 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1affd700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 0x7f2a040a3c50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b2e0 tx=0x7f2a0c00bb20 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a10094fd0 tx=0x7f2a10099d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6588e563e6f5e9a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a017fa700 1 -- 172.21.15.5:0/4288392789 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1009e510 con 0x56044e99dd20 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a017fa700 1 -- 172.21.15.5:0/4288392789 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a100a2750 con 0x56044e99dd20 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.241+0000 7f2a017fa700 1 -- 172.21.15.5:0/4288392789 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100cd690 con 0x56044e99dd20 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 -- 172.21.15.5:0/4288392789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 msgr2=0x56044e9a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2a10094fd0 tx=0x7f2a10099d20 comp rx=0 tx=0).stop 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 -- 172.21.15.5:0/4288392789 shutdown_connections 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99d400 0x56044e99d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/4288392789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99dd20 0x56044e9a8690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 -- 172.21.15.5:0/4288392789 >> 172.21.15.5:0/4288392789 conn(0x56044e912260 msgr2=0x56044e999790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 -- 172.21.15.5:0/4288392789 shutdown_connections 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.242+0000 7f2a34f16b80 1 -- 172.21.15.5:0/4288392789 wait complete. 2023-09-23T04:50:37.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.243+0000 7f2a34f16b80 1 -- start start 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.243+0000 7f2a34f16b80 1 Processor -- start 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.243+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.243+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.243+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e9a8bd0 0x56044e9a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e8f0e30 con 0x56044e99d400 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e88d710 con 0x56044e9a8bd0 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e88d3b0 con 0x56044e99dd20 2023-09-23T04:50:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a34f16b80 10 client.0 _ll_register_callbacks cb 0x56044e6ea800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33988/0 (socket says 172.21.15.5:33988) 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51892/0 (socket says 172.21.15.5:51892) 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/3692691766 learned_addr learned my addr 172.21.15.5:0/3692691766 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e9a8bd0 0x56044e9a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/3692691766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e9a8bd0 msgr2=0x56044e9a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e9a8bd0 0x56044e9a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/3692691766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 msgr2=0x56044e9a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/3692691766 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a3e70 con 0x56044e99dd20 2023-09-23T04:50:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.244+0000 7f2a1affd700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cb120 tx=0x7f2a100cd6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84eb75d0e95c13a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10010880 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56044e991250 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2a1009e510 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1009a830 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a34f16b80 10 client.26785 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56044e71bde0 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56044e991250 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2a10099320 con 0x56044e99dd20 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a34f16b80 10 client.26785 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.245+0000 7f2a34f16b80 10 client.26785 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a34f16b80 20 client.26785 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044e9bc670 0x56044e9bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a34f16b80 4 client.26785 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56044e8e9e10 con 0x56044e9bc670 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a1affd700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044e9bc670 0x56044e9bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f2a1affd700 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044e9bc670 0x56044e9bea90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=493e4de48ddfc347 server_cookie=6c7864f5f678c89e in_seq=0 out_seq=0 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.246+0000 7f29feff5700 10 client.26785 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f29feff5700 1 -- 172.21.15.5:0/3692691766 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56044e8e9e10 con 0x56044e9bc670 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f29feff5700 10 client.26785 handle_command_reply: tid=0 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 client.26785 shutdown 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 msgr2=0x56044e9a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cb120 tx=0x7f2a100cd6e0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 shutdown_connections 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e9a8bd0 0x56044e9a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044e9bc670 0x56044e9bea90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99d400 0x56044e9a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/3692691766 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9a5870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f29feff5700 0 client.26785 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.247+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 >> 172.21.15.5:0/3692691766 conn(0x56044e912260 msgr2=0x56044e999380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.248+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 shutdown_connections 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.248+0000 7f2a34f16b80 1 -- 172.21.15.5:0/3692691766 wait complete. 2023-09-23T04:50:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.248+0000 7f2a34f16b80 20 client.26785 trim_cache size 0 max 16384 2023-09-23T04:50:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.264+0000 7f2a34f16b80 1 Processor -- start 2023-09-23T04:50:37.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.264+0000 7f2a34f16b80 1 -- start start 2023-09-23T04:50:37.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99dd20 0x56044e9a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e9a8bd0 0x56044e9a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e771920 con 0x56044e99dd20 2023-09-23T04:50:37.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e99e130 con 0x56044e99d400 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e9a1700 con 0x56044e9a8bd0 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99dd20 0x56044e9a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:48976/0 (socket says 172.21.15.5:48976) 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 -- 172.21.15.5:0/122650883 learned_addr learned my addr 172.21.15.5:0/122650883 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e9a8bd0 0x56044e9a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 -- 172.21.15.5:0/122650883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e9a8bd0 msgr2=0x56044e9a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e9a8bd0 0x56044e9a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 -- 172.21.15.5:0/122650883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99dd20 msgr2=0x56044e9a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99dd20 0x56044e9a8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.265+0000 7f2a1affd700 1 -- 172.21.15.5:0/122650883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044e704c60 con 0x56044e99d400 2023-09-23T04:50:37.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.266+0000 7f2a1affd700 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0098e0 tx=0x7f2a0c027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2073b23995a595b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.266+0000 7f2a01ffb700 1 -- 172.21.15.5:0/122650883 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0343b0 con 0x56044e99d400 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.266+0000 7f2a01ffb700 1 -- 172.21.15.5:0/122650883 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a0c038070 con 0x56044e99d400 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.266+0000 7f2a01ffb700 1 -- 172.21.15.5:0/122650883 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c004c70 con 0x56044e99d400 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 -- 172.21.15.5:0/122650883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 msgr2=0x56044e9a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0098e0 tx=0x7f2a0c027000 comp rx=0 tx=0).stop 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 -- 172.21.15.5:0/122650883 shutdown_connections 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e9a8bd0 0x56044e9a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9a02a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/122650883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e99dd20 0x56044e9a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 -- 172.21.15.5:0/122650883 >> 172.21.15.5:0/122650883 conn(0x56044e912260 msgr2=0x56044e9a0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 -- 172.21.15.5:0/122650883 shutdown_connections 2023-09-23T04:50:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.267+0000 7f2a34f16b80 1 -- 172.21.15.5:0/122650883 wait complete. 2023-09-23T04:50:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 -- start start 2023-09-23T04:50:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 Processor -- start 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e9b25f0 con 0x56044e9a8bd0 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e9b2760 con 0x56044e99d400 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044e9b2920 con 0x56044e99dd20 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.268+0000 7f2a34f16b80 10 client.0 _ll_register_callbacks cb 0x7f2a10099f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34000/0 (socket says 172.21.15.5:34000) 2023-09-23T04:50:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/978199602 learned_addr learned my addr 172.21.15.5:0/978199602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1b7fe700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1affd700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/978199602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 msgr2=0x56044e9b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/978199602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 msgr2=0x56044e9b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 -- 172.21.15.5:0/978199602 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c02e040 con 0x56044e99dd20 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a1a7fc700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000c00 tx=0x7f2a10099fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f56fe21f36b637 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f29feff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1009f2f0 con 0x56044e99dd20 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a100a2750 con 0x56044e99dd20 2023-09-23T04:50:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.269+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2a1009e510 con 0x56044e99dd20 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1009f4e0 con 0x56044e99dd20 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 10 client.26791 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56044e742250 con 0x56044e99dd20 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a100a2750 con 0x56044e99dd20 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2a100a22c0 con 0x56044e99dd20 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 10 client.26791 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 20 client.26791 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044ea112f0 0x56044ea136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 4 client.26791 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56044e8e9b30 con 0x56044ea112f0 2023-09-23T04:50:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.270+0000 7f2a1affd700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044ea112f0 0x56044ea136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f29feff5700 10 client.26791 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a1affd700 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044ea112f0 0x56044ea136e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=faa31a8d69bfe8a1 server_cookie=857dfef50fe5664b in_seq=0 out_seq=0 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f29feff5700 1 -- 172.21.15.5:0/978199602 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56044e8e9b30 con 0x56044ea112f0 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f29feff5700 10 client.26791 handle_command_reply: tid=0 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 client.26791 shutdown 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 msgr2=0x56044e9b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a10000c00 tx=0x7f2a10099fb0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 shutdown_connections 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x56044ea112f0 0x56044ea136e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56044e9a8bd0 0x56044e9b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56044e99d400 0x56044e9b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f29feff5700 0 client.26791 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 --2- 172.21.15.5:0/978199602 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56044e99dd20 0x56044e9b1b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.271+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 >> 172.21.15.5:0/978199602 conn(0x56044e912260 msgr2=0x56044e9989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.272+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 shutdown_connections 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.272+0000 7f2a34f16b80 1 -- 172.21.15.5:0/978199602 wait complete. 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.272+0000 7f2a34f16b80 20 client.26791 trim_cache size 0 max 16384 2023-09-23T04:50:37.276 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 msgr2=0x7f2a040a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 0x7f2a040a3c50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b2e0 tx=0x7f2a0c00bb20 comp rx=0 tx=0).stop 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 msgr2=0x7f2a1c197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000e9a0 tx=0x7f2a1000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 shutdown_connections 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2a040a1790 0x7f2a040a3c50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a1c10b490 0x7f2a1c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2a1c1020c0 0x7f2a1c197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 --2- 172.21.15.5:0/1741811817 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2a1c1029e0 0x7f2a1c197580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 >> 172.21.15.5:0/1741811817 conn(0x7f2a1c076f30 msgr2=0x7f2a1c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 shutdown_connections 2023-09-23T04:50:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.273+0000 7f2a215ff700 1 -- 172.21.15.5:0/1741811817 wait complete. 2023-09-23T04:50:37.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.533+0000 7f243bfff700 1 -- 172.21.15.5:0/1259882734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 msgr2=0x7f243410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.533+0000 7f243bfff700 1 --2- 172.21.15.5:0/1259882734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f243410aec0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2424009990 tx=0x7f2424009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 -- 172.21.15.5:0/1259882734 shutdown_connections 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 --2- 172.21.15.5:0/1259882734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f243410b490 0x7f243410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 --2- 172.21.15.5:0/1259882734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24341020c0 0x7f24341024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 --2- 172.21.15.5:0/1259882734 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f243410aec0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 -- 172.21.15.5:0/1259882734 >> 172.21.15.5:0/1259882734 conn(0x7f2434076f30 msgr2=0x7f2434077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 -- 172.21.15.5:0/1259882734 shutdown_connections 2023-09-23T04:50:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.534+0000 7f243bfff700 1 -- 172.21.15.5:0/1259882734 wait complete. 2023-09-23T04:50:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.535+0000 7f243bfff700 1 Processor -- start 2023-09-23T04:50:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.535+0000 7f243bfff700 1 -- start start 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24341020c0 0x7f2434197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243410b490 0x7f243419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243419bfe0 con 0x7f24341020c0 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243419c150 con 0x7f243410b490 2023-09-23T04:50:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243bfff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243419c310 con 0x7f24341029e0 2023-09-23T04:50:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243959a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243959a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34014/0 (socket says 172.21.15.5:34014) 2023-09-23T04:50:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243959a700 1 -- 172.21.15.5:0/2854349703 learned_addr learned my addr 172.21.15.5:0/2854349703 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f2439d9b700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24341020c0 0x7f2434197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.536+0000 7f243a59c700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243410b490 0x7f243419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 -- 172.21.15.5:0/2854349703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243410b490 msgr2=0x7f243419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243410b490 0x7f243419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 -- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24341020c0 msgr2=0x7f2434197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24341020c0 0x7f2434197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 -- 172.21.15.5:0/2854349703 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24240095f0 con 0x7f24341029e0 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243959a700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2424005910 tx=0x7f2424005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f242affd700 1 -- 172.21.15.5:0/2854349703 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242401d070 con 0x7f24341029e0 2023-09-23T04:50:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f242affd700 1 -- 172.21.15.5:0/2854349703 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2424022470 con 0x7f24341029e0 2023-09-23T04:50:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243419c590 con 0x7f24341029e0 2023-09-23T04:50:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f242affd700 1 -- 172.21.15.5:0/2854349703 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424017650 con 0x7f24341029e0 2023-09-23T04:50:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.537+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24341a3e30 con 0x7f24341029e0 2023-09-23T04:50:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.541+0000 7f242affd700 1 -- 172.21.15.5:0/2854349703 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f242400fad0 con 0x7f24341029e0 2023-09-23T04:50:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.542+0000 7f244f859b80 1 Processor -- start 2023-09-23T04:50:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 -- start start 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2400 0x55fe3e2a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e20b460 con 0x55fe3e2b1fe0 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e1fcc20 con 0x55fe3e2a2400 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f244f859b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e1fa540 con 0x55fe3e2a2d20 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f242affd700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 0x7f24200a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.543+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34022/0 (socket says 172.21.15.5:34022) 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 -- 172.21.15.5:0/3113314401 learned_addr learned my addr 172.21.15.5:0/3113314401 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f243a59c700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2400 0x55fe3e2a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f242affd700 1 -- 172.21.15.5:0/2854349703 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f24240cec00 con 0x7f24341029e0 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f243959a700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f243a59c700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 0x7f24200a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 -- 172.21.15.5:0/3113314401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2400 msgr2=0x55fe3e2a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2400 0x55fe3e2a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 -- 172.21.15.5:0/3113314401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 msgr2=0x55fe3e2b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 -- 172.21.15.5:0/3113314401 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3e2588d0 con 0x55fe3e2a2d20 2023-09-23T04:50:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f243959a700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d80 tx=0x7f2430011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81915fa2d7f9239 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f241ffff700 1 -- 172.21.15.5:0/3113314401 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004230 con 0x55fe3e2a2d20 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.544+0000 7f243a59c700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 0x7f24200a3bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f242c009790 tx=0x7f242c009130 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f241ffff700 1 -- 172.21.15.5:0/3113314401 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2430004390 con 0x55fe3e2a2d20 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f241ffff700 1 -- 172.21.15.5:0/3113314401 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430019880 con 0x55fe3e2a2d20 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 -- 172.21.15.5:0/3113314401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 msgr2=0x55fe3e2b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d80 tx=0x7f2430011720 comp rx=0 tx=0).stop 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 -- 172.21.15.5:0/3113314401 shutdown_connections 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2400 0x55fe3e2a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 --2- 172.21.15.5:0/3113314401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b1aa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 -- 172.21.15.5:0/3113314401 >> 172.21.15.5:0/3113314401 conn(0x55fe3e217260 msgr2=0x55fe3e29e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 -- 172.21.15.5:0/3113314401 shutdown_connections 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.545+0000 7f244f859b80 1 -- 172.21.15.5:0/3113314401 wait complete. 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.546+0000 7f244f859b80 1 -- start start 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.546+0000 7f244f859b80 1 Processor -- start 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e1f5e30 con 0x55fe3e2a2d20 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e192710 con 0x55fe3e2b1fe0 2023-09-23T04:50:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e1923b0 con 0x55fe3e2a2400 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f244f859b80 10 client.0 _ll_register_callbacks cb 0x55fe3dfef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34030/0 (socket says 172.21.15.5:34030) 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f2439d9b700 1 -- 172.21.15.5:0/3489196754 learned_addr learned my addr 172.21.15.5:0/3489196754 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f243959a700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f243a59c700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.547+0000 7f2439d9b700 1 -- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2b1fe0 msgr2=0x55fe3e2af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f2439d9b700 1 -- 172.21.15.5:0/3489196754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2d20 msgr2=0x55fe3e2aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f2439d9b700 1 -- 172.21.15.5:0/3489196754 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24300099e0 con 0x55fe3e2a2400 2023-09-23T04:50:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f2439d9b700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2430005950 tx=0x7f24300038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf56884296d68ba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f241d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005450 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2430003ad0 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe3e296250 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2430009ed0 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.548+0000 7f244f859b80 10 client.26803 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55fe3e020de0 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe3e296250 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2430021040 con 0x55fe3e2a2400 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 10 client.26803 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 10 client.26803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:50:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 20 client.26803 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e2c1900 0x55fe3e2c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 4 client.26803 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe3e1eee10 con 0x55fe3e2c1900 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.549+0000 7f243959a700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e2c1900 0x55fe3e2c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.550+0000 7f243959a700 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e2c1900 0x55fe3e2c3cf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7360bb05d9abe054 server_cookie=d15b5f3fc62aafbd in_seq=0 out_seq=0 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.550+0000 7f241d7fa700 10 client.26803 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.550+0000 7f241d7fa700 1 -- 172.21.15.5:0/3489196754 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe3e1eee10 con 0x55fe3e2c1900 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.550+0000 7f241d7fa700 10 client.26803 handle_command_reply: tid=0 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 client.26803 shutdown 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 msgr2=0x55fe3e2b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2430005950 tx=0x7f24300038b0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 shutdown_connections 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e2c1900 0x55fe3e2c3cf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b1a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 --2- 172.21.15.5:0/3489196754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 >> 172.21.15.5:0/3489196754 conn(0x55fe3e217260 msgr2=0x55fe3e29e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f241d7fa700 0 client.26803 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 shutdown_connections 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 1 -- 172.21.15.5:0/3489196754 wait complete. 2023-09-23T04:50:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.551+0000 7f244f859b80 20 client.26803 trim_cache size 0 max 16384 2023-09-23T04:50:37.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.566+0000 7f244f859b80 1 Processor -- start 2023-09-23T04:50:37.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 -- start start 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e076920 con 0x55fe3e2a2400 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e298860 con 0x55fe3e2a2d20 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f244f859b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e2b0230 con 0x55fe3e2b1fe0 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f243a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51952/0 (socket says 172.21.15.5:51952) 2023-09-23T04:50:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f2439d9b700 1 -- 172.21.15.5:0/373563930 learned_addr learned my addr 172.21.15.5:0/373563930 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.567+0000 7f243959a700 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 -- 172.21.15.5:0/373563930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2b1fe0 msgr2=0x55fe3e2a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 -- 172.21.15.5:0/373563930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 msgr2=0x55fe3e2b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 -- 172.21.15.5:0/373563930 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3e009c60 con 0x55fe3e2a2d20 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f2439d9b700 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f243959a700 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2a49a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24240cbf10 tx=0x7f242401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987ad0a4ec4726a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f2428ff9700 1 -- 172.21.15.5:0/373563930 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242409a010 con 0x55fe3e2a2d20 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f2428ff9700 1 -- 172.21.15.5:0/373563930 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2424099e20 con 0x55fe3e2a2d20 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.568+0000 7f2428ff9700 1 -- 172.21.15.5:0/373563930 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240bb0b0 con 0x55fe3e2a2d20 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 -- 172.21.15.5:0/373563930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 msgr2=0x55fe3e2a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2a49a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24240cbf10 tx=0x7f242401a040 comp rx=0 tx=0).stop 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 -- 172.21.15.5:0/373563930 shutdown_connections 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 --2- 172.21.15.5:0/373563930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2a49a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 -- 172.21.15.5:0/373563930 >> 172.21.15.5:0/373563930 conn(0x55fe3e217260 msgr2=0x55fe3e2a4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 -- 172.21.15.5:0/373563930 shutdown_connections 2023-09-23T04:50:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.569+0000 7f244f859b80 1 -- 172.21.15.5:0/373563930 wait complete. 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.570+0000 7f244f859b80 1 -- start start 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.570+0000 7f244f859b80 1 Processor -- start 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e2b7200 con 0x55fe3e2b1fe0 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e2b7370 con 0x55fe3e2a2d20 2023-09-23T04:50:37.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3e2b7530 con 0x55fe3e2a2400 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f244f859b80 10 client.0 _ll_register_callbacks cb 0x7f242c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34040/0 (socket says 172.21.15.5:34040) 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 -- 172.21.15.5:0/4101836005 learned_addr learned my addr 172.21.15.5:0/4101836005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f243959a700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f243a59c700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 -- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 msgr2=0x55fe3e2b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 -- 172.21.15.5:0/4101836005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 msgr2=0x55fe3e2b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.571+0000 7f2439d9b700 1 -- 172.21.15.5:0/4101836005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe3e020de0 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f2439d9b700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2430019490 tx=0x7f2430005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e24c68de87b34 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f241d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430019cb0 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2430000e40 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2430004390 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2430009ed0 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f244f859b80 10 client.26809 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.572+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f24341a3e30 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2430004390 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f243003f070 con 0x55fe3e2a2400 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 10 client.26809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:50:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 20 client.26809 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e3163f0 0x55fe3e3187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 4 client.26809 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe3e1eeb30 con 0x55fe3e3163f0 2023-09-23T04:50:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.573+0000 7f243959a700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e3163f0 0x55fe3e3187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f243959a700 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e3163f0 0x55fe3e3187e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=446ecf7efc374481 server_cookie=b2fa1044b29e8613 in_seq=0 out_seq=0 2023-09-23T04:50:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f241d7fa700 10 client.26809 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f241d7fa700 1 -- 172.21.15.5:0/4101836005 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe3e1eeb30 con 0x55fe3e3163f0 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f241d7fa700 10 client.26809 handle_command_reply: tid=0 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 client.26809 shutdown 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 msgr2=0x55fe3e2b8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2430019490 tx=0x7f2430005e50 comp rx=0 tx=0).stop 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 shutdown_connections 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55fe3e3163f0 0x55fe3e3187e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe3e2b1fe0 0x55fe3e2b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe3e2a2400 0x55fe3e2b8390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 --2- 172.21.15.5:0/4101836005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe3e2a2d20 0x55fe3e2b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f241d7fa700 0 client.26809 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 >> 172.21.15.5:0/4101836005 conn(0x55fe3e217260 msgr2=0x55fe3e2a4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 shutdown_connections 2023-09-23T04:50:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.574+0000 7f244f859b80 1 -- 172.21.15.5:0/4101836005 wait complete. 2023-09-23T04:50:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f244f859b80 20 client.26809 trim_cache size 0 max 16384 2023-09-23T04:50:37.581 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 msgr2=0x7f24200a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 0x7f24200a3bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f242c009790 tx=0x7f242c009130 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 msgr2=0x7f2434197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2424005910 tx=0x7f2424005740 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 shutdown_connections 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f24200a1710 0x7f24200a3bd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243410b490 0x7f243419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24341020c0 0x7f2434197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 --2- 172.21.15.5:0/2854349703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f24341029e0 0x7f2434197540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.575+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 >> 172.21.15.5:0/2854349703 conn(0x7f2434076f30 msgr2=0x7f24340fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.576+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 shutdown_connections 2023-09-23T04:50:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.576+0000 7f243bfff700 1 -- 172.21.15.5:0/2854349703 wait complete. 2023-09-23T04:50:37.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:50:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 -- 172.21.15.5:0/2657545003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a0103cc0 msgr2=0x7f14a0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 --2- 172.21.15.5:0/2657545003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a0103cc0 0x7f14a0107d00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1490009990 tx=0x7f1490009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 -- 172.21.15.5:0/2657545003 shutdown_connections 2023-09-23T04:50:37.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 --2- 172.21.15.5:0/2657545003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 --2- 172.21.15.5:0/2657545003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 --2- 172.21.15.5:0/2657545003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a0103cc0 0x7f14a0107d00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 -- 172.21.15.5:0/2657545003 >> 172.21.15.5:0/2657545003 conn(0x7f14a00feb50 msgr2=0x7f14a0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 -- 172.21.15.5:0/2657545003 shutdown_connections 2023-09-23T04:50:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.828+0000 7f14a7bff700 1 -- 172.21.15.5:0/2657545003 wait complete. 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.829+0000 7f14a7bff700 1 Processor -- start 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.829+0000 7f14a7bff700 1 -- start start 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 0x7f14a019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a019e610 con 0x7f14a0103310 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a019e780 con 0x7f14a0108430 2023-09-23T04:50:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a7bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a019e940 con 0x7f14a019db90 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51988/0 (socket says 172.21.15.5:51988) 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 -- 172.21.15.5:0/4140256138 learned_addr learned my addr 172.21.15.5:0/4140256138 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a619c700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 0x7f14a019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a519a700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 -- 172.21.15.5:0/4140256138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 msgr2=0x7f14a019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 0x7f14a019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 -- 172.21.15.5:0/4140256138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 msgr2=0x7f14a0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.830+0000 7f14a599b700 1 -- 172.21.15.5:0/4140256138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900095f0 con 0x7f14a0103310 2023-09-23T04:50:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f14a619c700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 0x7f14a019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f14a519a700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f14a599b700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9a0 tx=0x7f149c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f1496ffd700 1 -- 172.21.15.5:0/4140256138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c009790 con 0x7f14a0103310 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f1496ffd700 1 -- 172.21.15.5:0/4140256138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149c010430 con 0x7f14a0103310 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a01a5ea0 con 0x7f14a0103310 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f1496ffd700 1 -- 172.21.15.5:0/4140256138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c018680 con 0x7f14a0103310 2023-09-23T04:50:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.831+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a01a6320 con 0x7f14a0103310 2023-09-23T04:50:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.835+0000 7f1496ffd700 1 -- 172.21.15.5:0/4140256138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f149c0105a0 con 0x7f14a0103310 2023-09-23T04:50:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.837+0000 7f14bb5c7b80 1 Processor -- start 2023-09-23T04:50:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.837+0000 7f1496ffd700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 0x7f148c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.837+0000 7f14bb5c7b80 1 -- start start 2023-09-23T04:50:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.837+0000 7f1496ffd700 1 -- 172.21.15.5:0/4140256138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f149c014070 con 0x7f14a0103310 2023-09-23T04:50:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.837+0000 7f14a619c700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 0x7f148c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e1839287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183935fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183891460 con 0x55e183933bd0 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183882c20 con 0x55e183928400 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183880540 con 0x55e183928d20 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34074/0 (socket says 172.21.15.5:34074) 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 -- 172.21.15.5:0/3902109140 learned_addr learned my addr 172.21.15.5:0/3902109140 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a599b700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e1839287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 -- 172.21.15.5:0/3902109140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 msgr2=0x55e1839287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e1839287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a619c700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183935fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 -- 172.21.15.5:0/3902109140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 msgr2=0x55e183935fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183935fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a519a700 1 -- 172.21.15.5:0/3902109140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1838de8d0 con 0x55e183928d20 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.838+0000 7f14a619c700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183935fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.839+0000 7f14a619c700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 0x7f148c0a3bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1498003e30 tx=0x7f149800b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.839+0000 7f14a519a700 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1490009990 tx=0x7f1490005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08302128ff0d318 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.839+0000 7f148bfff700 1 -- 172.21.15.5:0/3902109140 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149001d070 con 0x55e183928d20 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.839+0000 7f148bfff700 1 -- 172.21.15.5:0/3902109140 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149001e040 con 0x55e183928d20 2023-09-23T04:50:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.839+0000 7f148bfff700 1 -- 172.21.15.5:0/3902109140 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490004cf0 con 0x55e183928d20 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/3902109140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 msgr2=0x55e183933690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1490009990 tx=0x7f1490005ee0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/3902109140 shutdown_connections 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183935fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e1839287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/3902109140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183933690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/3902109140 >> 172.21.15.5:0/3902109140 conn(0x55e18389d260 msgr2=0x55e183924790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/3902109140 shutdown_connections 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.840+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/3902109140 wait complete. 2023-09-23T04:50:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.841+0000 7f14bb5c7b80 1 -- start start 2023-09-23T04:50:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.841+0000 7f14bb5c7b80 1 Processor -- start 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e1839335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e1839307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e183930de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18387be30 con 0x55e183928400 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183818710 con 0x55e183933bd0 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1838183b0 con 0x55e183928d20 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14bb5c7b80 10 client.0 _ll_register_callbacks cb 0x55e183675800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14a599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e1839335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14a519a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e1839307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14a599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e1839335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52014/0 (socket says 172.21.15.5:52014) 2023-09-23T04:50:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14a599b700 1 -- 172.21.15.5:0/1509827146 learned_addr learned my addr 172.21.15.5:0/1509827146 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.842+0000 7f14a619c700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e183930de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 -- 172.21.15.5:0/1509827146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 msgr2=0x55e183930de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e183930de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 -- 172.21.15.5:0/1509827146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 msgr2=0x55e1839335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e1839335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 -- 172.21.15.5:0/1509827146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a0110820 con 0x55e183928d20 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a619c700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e183930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14a519a700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e1839307d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1490000c00 tx=0x7f1490022b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f4de6df69a97cde server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14897fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.843+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149001d030 con 0x55e183928d20 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e18391c250 con 0x55e183928d20 2023-09-23T04:50:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1490004500 con 0x55e183928d20 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14900046f0 con 0x55e183928d20 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 10 client.26821 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e1836a6de0 con 0x55e183928d20 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e18391c250 con 0x55e183928d20 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f149002c400 con 0x55e183928d20 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 10 client.26821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 20 client.26821 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e183943020 0x55e183945410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 4 client.26821 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.844+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e183874e10 con 0x55e183943020 2023-09-23T04:50:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.845+0000 7f14a599b700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e183943020 0x55e183945410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.845+0000 7f14a599b700 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e183943020 0x55e183945410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eef534da0cf09c43 server_cookie=d77881eef3564998 in_seq=0 out_seq=0 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.845+0000 7f14897fa700 10 client.26821 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14897fa700 1 -- 172.21.15.5:0/1509827146 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e183874e10 con 0x55e183943020 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14897fa700 10 client.26821 handle_command_reply: tid=0 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 client.26821 shutdown 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 msgr2=0x55e1839307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e1839307d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1490000c00 tx=0x7f1490022b10 comp rx=0 tx=0).stop 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 shutdown_connections 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e183943020 0x55e183945410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e183930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e1839335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1509827146 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e1839307d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14897fa700 0 client.26821 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 >> 172.21.15.5:0/1509827146 conn(0x55e18389d260 msgr2=0x55e183924320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 shutdown_connections 2023-09-23T04:50:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.846+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1509827146 wait complete. 2023-09-23T04:50:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.847+0000 7f14bb5c7b80 20 client.26821 trim_cache size 0 max 16384 2023-09-23T04:50:37.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.860+0000 7f14bb5c7b80 1 Processor -- start 2023-09-23T04:50:37.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.860+0000 7f14bb5c7b80 1 -- start start 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e183930de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e18393f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1836fc920 con 0x55e183928400 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18392c6a0 con 0x55e183933bd0 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1839313e0 con 0x55e183928d20 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14a619c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14a599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e183930de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14a619c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49066/0 (socket says 172.21.15.5:49066) 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14a619c700 1 -- 172.21.15.5:0/1290480551 learned_addr learned my addr 172.21.15.5:0/1290480551 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.861+0000 7f14a519a700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e18393f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 -- 172.21.15.5:0/1290480551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 msgr2=0x55e18393f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e18393f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 -- 172.21.15.5:0/1290480551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 msgr2=0x55e183930de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e183930de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 -- 172.21.15.5:0/1290480551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e18368fc60 con 0x55e183933bd0 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a599b700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e183930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a519a700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e18393f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:37.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f14a619c700 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f149800e990 tx=0x7f1498014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9240ff0d895003f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f1494bff700 1 -- 172.21.15.5:0/1290480551 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498009020 con 0x55e183933bd0 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f1494bff700 1 -- 172.21.15.5:0/1290480551 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149800f040 con 0x55e183933bd0 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.862+0000 7f1494bff700 1 -- 172.21.15.5:0/1290480551 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498008930 con 0x55e183933bd0 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1290480551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 msgr2=0x55e1839330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f149800e990 tx=0x7f1498014000 comp rx=0 tx=0).stop 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1290480551 shutdown_connections 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183933bd0 0x55e1839330f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183928400 0x55e183930de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/1290480551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e18393f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1290480551 >> 172.21.15.5:0/1290480551 conn(0x55e18389d260 msgr2=0x55e183949d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1290480551 shutdown_connections 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.863+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/1290480551 wait complete. 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.864+0000 7f14bb5c7b80 1 -- start start 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.864+0000 7f14bb5c7b80 1 Processor -- start 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183939db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e18393a6c0 con 0x55e183933bd0 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183996340 con 0x55e183928400 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e183996480 con 0x55e183928d20 2023-09-23T04:50:37.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14bb5c7b80 10 client.0 _ll_register_callbacks cb 0x7f1490009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a599b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a619c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a599b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49078/0 (socket says 172.21.15.5:49078) 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a619c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52038/0 (socket says 172.21.15.5:52038) 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a599b700 1 -- 172.21.15.5:0/2032173449 learned_addr learned my addr 172.21.15.5:0/2032173449 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a519a700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183939db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a519a700 1 -- 172.21.15.5:0/2032173449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 msgr2=0x55e183939870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.865+0000 7f14a519a700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a519a700 1 -- 172.21.15.5:0/2032173449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 msgr2=0x55e183995e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a519a700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a519a700 1 -- 172.21.15.5:0/2032173449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a01a6320 con 0x55e183928d20 2023-09-23T04:50:37.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a599b700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a619c700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14a519a700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183939db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55e18391eb60 tx=0x7f149000bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b91d6a5d900c4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14897fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149001d050 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f149001e040 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1490000e40 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149002cbd0 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14bb5c7b80 10 client.26824 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.866+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f14980048a0 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149001e040 con 0x55e183928d20 2023-09-23T04:50:37.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f1490004620 con 0x55e183928d20 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 10 client.26824 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 10 client.26824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 20 client.26824 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e1839975e0 0x55e1839999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 4 client.26824 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e183874b30 con 0x55e1839975e0 2023-09-23T04:50:37.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.867+0000 7f14a599b700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e1839975e0 0x55e1839999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14a599b700 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e1839975e0 0x55e1839999d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=675525f2f10a037b server_cookie=a4b1aa8805a19970 in_seq=0 out_seq=0 2023-09-23T04:50:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14897fa700 10 client.26824 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14897fa700 1 -- 172.21.15.5:0/2032173449 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e183874b30 con 0x55e1839975e0 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14897fa700 10 client.26824 handle_command_reply: tid=0 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 client.26824 shutdown 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 msgr2=0x55e183939db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183939db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55e18391eb60 tx=0x7f149000bf80 comp rx=0 tx=0).stop 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 shutdown_connections 2023-09-23T04:50:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e183933bd0 0x55e183995e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55e1839975e0 0x55e1839999d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e183928400 0x55e183939870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 --2- 172.21.15.5:0/2032173449 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e183928d20 0x55e183939db0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 >> 172.21.15.5:0/2032173449 conn(0x55e18389d260 msgr2=0x55e18393f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14897fa700 0 client.26824 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 shutdown_connections 2023-09-23T04:50:37.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 1 -- 172.21.15.5:0/2032173449 wait complete. 2023-09-23T04:50:37.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.868+0000 7f14bb5c7b80 20 client.26824 trim_cache size 0 max 16384 2023-09-23T04:50:37.875 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:37.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 msgr2=0x7f148c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 0x7f148c0a3bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1498003e30 tx=0x7f149800b040 comp rx=0 tx=0).stop 2023-09-23T04:50:37.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 msgr2=0x7f14a01991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9a0 tx=0x7f149c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 shutdown_connections 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f148c0a1710 0x7f148c0a3bd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f14a019db90 0x7f14a019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a0103310 0x7f14a01991f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 --2- 172.21.15.5:0/4140256138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14a0108430 0x7f14a0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:37.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 >> 172.21.15.5:0/4140256138 conn(0x7f14a00feb50 msgr2=0x7f14a00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:37.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 shutdown_connections 2023-09-23T04:50:37.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:37.869+0000 7f14a7bff700 1 -- 172.21.15.5:0/4140256138 wait complete. 2023-09-23T04:50:37.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:50:37.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259693 5.70 MB/sec execute 73 sec latency 16.312 ms 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.115+0000 7f509c7ff700 1 -- 172.21.15.5:0/4152591792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50941020c0 msgr2=0x7f50941024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.115+0000 7f509c7ff700 1 --2- 172.21.15.5:0/4152591792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50941020c0 0x7f50941024a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5090009990 tx=0x7f5090009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 -- 172.21.15.5:0/4152591792 shutdown_connections 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 --2- 172.21.15.5:0/4152591792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f509410b490 0x7f509410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 --2- 172.21.15.5:0/4152591792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50941020c0 0x7f50941024a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 --2- 172.21.15.5:0/4152591792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941029e0 0x7f509410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 -- 172.21.15.5:0/4152591792 >> 172.21.15.5:0/4152591792 conn(0x7f5094076f30 msgr2=0x7f5094077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 -- 172.21.15.5:0/4152591792 shutdown_connections 2023-09-23T04:50:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.116+0000 7f509c7ff700 1 -- 172.21.15.5:0/4152591792 wait complete. 2023-09-23T04:50:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.117+0000 7f509c7ff700 1 Processor -- start 2023-09-23T04:50:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 -- start start 2023-09-23T04:50:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 0x7f5094197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 0x7f509419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509419c0b0 con 0x7f509410b490 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509419c220 con 0x7f50941029e0 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509c7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509419c3e0 con 0x7f50941020c0 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f5099d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 0x7f5094197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34118/0 (socket says 172.21.15.5:34118) 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509a59b700 1 -- 172.21.15.5:0/1442667567 learned_addr learned my addr 172.21.15.5:0/1442667567 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.118+0000 7f509ad9c700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 0x7f509419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 -- 172.21.15.5:0/1442667567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 msgr2=0x7f50941970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 -- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 msgr2=0x7f509419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 0x7f509419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 -- 172.21.15.5:0/1442667567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900095f0 con 0x7f50941029e0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f509ad9c700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 0x7f509419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f509a59b700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f5099d9a700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 0x7f5094197610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f508400e9f0 tx=0x7f508400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f508b7fe700 1 -- 172.21.15.5:0/1442667567 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840096d0 con 0x7f50941029e0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509419c6c0 con 0x7f50941029e0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f508b7fe700 1 -- 172.21.15.5:0/1442667567 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508400cbd0 con 0x7f50941029e0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.119+0000 7f508b7fe700 1 -- 172.21.15.5:0/1442667567 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084018680 con 0x7f50941029e0 2023-09-23T04:50:38.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.120+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50941a3f00 con 0x7f50941029e0 2023-09-23T04:50:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.124+0000 7f508b7fe700 1 -- 172.21.15.5:0/1442667567 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f508400cd40 con 0x7f50941029e0 2023-09-23T04:50:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.125+0000 7f50b0031b80 1 Processor -- start 2023-09-23T04:50:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 -- start start 2023-09-23T04:50:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f508b7fe700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 0x7f50800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374200a460 con 0x5637420a1d20 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741ffbc20 con 0x5637420b0fe0 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f50b0031b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741ff9540 con 0x5637420a1400 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f508b7fe700 1 -- 172.21.15.5:0/1442667567 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f5084014070 con 0x7f50941029e0 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.126+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34124/0 (socket says 172.21.15.5:34124) 2023-09-23T04:50:38.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f509a59b700 1 -- 172.21.15.5:0/3176480693 learned_addr learned my addr 172.21.15.5:0/3176480693 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f5099d9a700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f509a59b700 1 -- 172.21.15.5:0/3176480693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 msgr2=0x5637420b33d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f509ad9c700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 0x7f50800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.127+0000 7f509ad9c700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509a59b700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509a59b700 1 -- 172.21.15.5:0/3176480693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 msgr2=0x5637420b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509a59b700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509a59b700 1 -- 172.21.15.5:0/3176480693 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637420578d0 con 0x5637420a1400 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f5099d9a700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509ad9c700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.128+0000 7f509a59b700 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5090005f00 tx=0x7f5090005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd9d1b897484c60e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.129+0000 7f5088ff9700 1 -- 172.21.15.5:0/3176480693 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509001d070 con 0x5637420a1400 2023-09-23T04:50:38.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.129+0000 7f5088ff9700 1 -- 172.21.15.5:0/3176480693 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509001e040 con 0x5637420a1400 2023-09-23T04:50:38.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.129+0000 7f5088ff9700 1 -- 172.21.15.5:0/3176480693 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000f460 con 0x5637420a1400 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f509ad9c700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 0x7f50800a3bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f508c007610 tx=0x7f508c009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- 172.21.15.5:0/3176480693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 msgr2=0x5637420a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5090005f00 tx=0x7f5090005bc0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- 172.21.15.5:0/3176480693 shutdown_connections 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420a17e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3176480693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- 172.21.15.5:0/3176480693 >> 172.21.15.5:0/3176480693 conn(0x563742016260 msgr2=0x56374209d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- 172.21.15.5:0/3176480693 shutdown_connections 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- 172.21.15.5:0/3176480693 wait complete. 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.130+0000 7f50b0031b80 1 -- start start 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 Processor -- start 2023-09-23T04:50:38.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420ae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741ff4e30 con 0x5637420a1d20 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741f91710 con 0x5637420a1400 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741f913b0 con 0x5637420b0fe0 2023-09-23T04:50:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f50b0031b80 10 client.0 _ll_register_callbacks cb 0x563741dee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.131+0000 7f509a59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49120/0 (socket says 172.21.15.5:49120) 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 -- 172.21.15.5:0/3035046286 learned_addr learned my addr 172.21.15.5:0/3035046286 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509ad9c700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f5099d9a700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 -- 172.21.15.5:0/3035046286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 msgr2=0x5637420ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 -- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 msgr2=0x5637420adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 -- 172.21.15.5:0/3035046286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50941a3f00 con 0x5637420a1400 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f509a59b700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5090009ce0 tx=0x7f5090004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8121c7bb9d31df8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.132+0000 7f507dffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:38.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509001d070 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563742095250 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f5090005510 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50900056d0 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f50b0031b80 10 client.26783 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f509410e5d0 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563742095250 con 0x5637420a1400 2023-09-23T04:50:38.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.133+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f509002c400 con 0x5637420a1400 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 10 client.26783 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 10 client.26783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 20 client.26783 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637420c0450 0x5637420c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 4 client.26783 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563741fede10 con 0x5637420c0450 2023-09-23T04:50:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.134+0000 7f5099d9a700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637420c0450 0x5637420c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.135+0000 7f507dffb700 10 client.26783 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.135+0000 7f5099d9a700 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637420c0450 0x5637420c2840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d609222c8b234d8 server_cookie=c9ffaca33942afce in_seq=0 out_seq=0 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.135+0000 7f507dffb700 1 -- 172.21.15.5:0/3035046286 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563741fede10 con 0x5637420c0450 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.135+0000 7f507dffb700 10 client.26783 handle_command_reply: tid=0 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.135+0000 7f50b0031b80 1 client.26783 shutdown 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 msgr2=0x5637420b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5090009ce0 tx=0x7f5090004d70 comp rx=0 tx=0).stop 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 shutdown_connections 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637420c0450 0x5637420c2840 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1400 0x5637420b0a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f507dffb700 0 client.26783 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 --2- 172.21.15.5:0/3035046286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 >> 172.21.15.5:0/3035046286 conn(0x563742016260 msgr2=0x56374209d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 shutdown_connections 2023-09-23T04:50:38.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 1 -- 172.21.15.5:0/3035046286 wait complete. 2023-09-23T04:50:38.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.136+0000 7f50b0031b80 20 client.26783 trim_cache size 0 max 16384 2023-09-23T04:50:38.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 Processor -- start 2023-09-23T04:50:38.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 -- start start 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741e75920 con 0x5637420a1d20 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637420a8a80 con 0x5637420b0fe0 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.151+0000 7f50b0031b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637420a2bf0 con 0x5637420a1400 2023-09-23T04:50:38.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f5099d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49128/0 (socket says 172.21.15.5:49128) 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 -- 172.21.15.5:0/641363868 learned_addr learned my addr 172.21.15.5:0/641363868 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509a59b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34144/0 (socket says 172.21.15.5:34144) 2023-09-23T04:50:38.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f5099d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52088/0 (socket says 172.21.15.5:52088) 2023-09-23T04:50:38.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 -- 172.21.15.5:0/641363868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 msgr2=0x5637420b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 -- 172.21.15.5:0/641363868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 msgr2=0x5637420ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f509ad9c700 1 -- 172.21.15.5:0/641363868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563741e1fde0 con 0x5637420b0fe0 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.152+0000 7f5099d9a700 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.153+0000 7f509ad9c700 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f508c008010 tx=0x7f508c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e9cb24b689b770 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.153+0000 7f50897fa700 1 -- 172.21.15.5:0/641363868 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00e050 con 0x5637420b0fe0 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.153+0000 7f50897fa700 1 -- 172.21.15.5:0/641363868 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508c00f040 con 0x5637420b0fe0 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.153+0000 7f50897fa700 1 -- 172.21.15.5:0/641363868 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c016650 con 0x5637420b0fe0 2023-09-23T04:50:38.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- 172.21.15.5:0/641363868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 msgr2=0x5637420ada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f508c008010 tx=0x7f508c014000 comp rx=0 tx=0).stop 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- 172.21.15.5:0/641363868 shutdown_connections 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420b0fe0 0x5637420ada20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420a1400 0x5637420b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 --2- 172.21.15.5:0/641363868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1d20 0x5637420ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- 172.21.15.5:0/641363868 >> 172.21.15.5:0/641363868 conn(0x563742016260 msgr2=0x5637420a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- 172.21.15.5:0/641363868 shutdown_connections 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- 172.21.15.5:0/641363868 wait complete. 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 -- start start 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.154+0000 7f50b0031b80 1 Processor -- start 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1400 0x5637420b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637420b6690 con 0x5637420a1400 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637420b6800 con 0x5637420a1d20 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637420b69c0 con 0x5637420b0fe0 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f50b0031b80 10 client.0 _ll_register_callbacks cb 0x7f50840960e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f5099d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f509a59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1400 0x5637420b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f5099d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49138/0 (socket says 172.21.15.5:49138) 2023-09-23T04:50:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f5099d9a700 1 -- 172.21.15.5:0/2727097113 learned_addr learned my addr 172.21.15.5:0/2727097113 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.155+0000 7f509ad9c700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 -- 172.21.15.5:0/2727097113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 msgr2=0x5637420b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 -- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1400 msgr2=0x5637420b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1400 0x5637420b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 -- 172.21.15.5:0/2727097113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509419c6c0 con 0x5637420a1d20 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f5099d9a700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5084000c00 tx=0x7f5084005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd69f896fadaf6b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f507dffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508409e530 con 0x5637420a1d20 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f50840a2770 con 0x5637420a1d20 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f509001e040 con 0x5637420a1d20 2023-09-23T04:50:38.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f508409e150 con 0x5637420a1d20 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f50b0031b80 10 client.26789 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.156+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f508c020040 con 0x5637420a1d20 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f509001e040 con 0x5637420a1d20 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f508409a640 con 0x5637420a1d20 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 10 client.26789 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:38.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 20 client.26789 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637421157c0 0x563742117be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 4 client.26789 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:50:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563741fedb30 con 0x5637421157c0 2023-09-23T04:50:38.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.157+0000 7f509a59b700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637421157c0 0x563742117be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f507dffb700 10 client.26789 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:38.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f509a59b700 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637421157c0 0x563742117be0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d732d20db6314f47 server_cookie=673d835a520283c in_seq=0 out_seq=0 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f507dffb700 1 -- 172.21.15.5:0/2727097113 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563741fedb30 con 0x5637421157c0 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f507dffb700 10 client.26789 handle_command_reply: tid=0 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 client.26789 shutdown 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 msgr2=0x5637420b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5084000c00 tx=0x7f5084005bf0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 shutdown_connections 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5637421157c0 0x563742117be0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5637420b0fe0 0x5637420b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f507dffb700 0 client.26789 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637420a1400 0x5637420b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 --2- 172.21.15.5:0/2727097113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5637420a1d20 0x5637420b5c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 >> 172.21.15.5:0/2727097113 conn(0x563742016260 msgr2=0x5637420a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 shutdown_connections 2023-09-23T04:50:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 1 -- 172.21.15.5:0/2727097113 wait complete. 2023-09-23T04:50:38.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.158+0000 7f50b0031b80 20 client.26789 trim_cache size 0 max 16384 2023-09-23T04:50:38.169 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:50:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 msgr2=0x7f50800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 0x7f50800a3bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f508c007610 tx=0x7f508c009920 comp rx=0 tx=0).stop 2023-09-23T04:50:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 msgr2=0x7f5094197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 0x7f5094197610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f508400e9f0 tx=0x7f508400ed00 comp rx=0 tx=0).stop 2023-09-23T04:50:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 shutdown_connections 2023-09-23T04:50:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f50800a1710 0x7f50800a3bd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f509410b490 0x7f509419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f50941020c0 0x7f50941970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 --2- 172.21.15.5:0/1442667567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50941029e0 0x7f5094197610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 >> 172.21.15.5:0/1442667567 conn(0x7f5094076f30 msgr2=0x7f5094102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 shutdown_connections 2023-09-23T04:50:38.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:38.159+0000 7f509c7ff700 1 -- 172.21.15.5:0/1442667567 wait complete. 2023-09-23T04:50:38.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 261896 5.79 MB/sec execute 74 sec latency 65.653 ms 2023-09-23T04:50:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:38 smithi005 ceph-mon[97638]: pgmap v134: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-23T04:50:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:38 smithi118 ceph-mon[134297]: pgmap v134: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-23T04:50:39.179 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:50:39.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:50:39.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:38 smithi069 ceph-mon[133931]: pgmap v134: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.3 MiB/s wr, 137 op/s 2023-09-23T04:50:39.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 -- 172.21.15.5:0/3661199318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc103cc0 msgr2=0x7f8bdc107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 --2- 172.21.15.5:0/3661199318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc103cc0 0x7f8bdc107d00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009990 tx=0x7f8bcc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 -- 172.21.15.5:0/3661199318 shutdown_connections 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 --2- 172.21.15.5:0/3661199318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc108430 0x7f8bdc10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 --2- 172.21.15.5:0/3661199318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc103310 0x7f8bdc1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 --2- 172.21.15.5:0/3661199318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc103cc0 0x7f8bdc107d00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 -- 172.21.15.5:0/3661199318 >> 172.21.15.5:0/3661199318 conn(0x7f8bdc0feb50 msgr2=0x7f8bdc100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.410+0000 7f8be35ff700 1 -- 172.21.15.5:0/3661199318 shutdown_connections 2023-09-23T04:50:39.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.411+0000 7f8be35ff700 1 -- 172.21.15.5:0/3661199318 wait complete. 2023-09-23T04:50:39.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 Processor -- start 2023-09-23T04:50:39.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 -- start start 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 0x7f8bdc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc108430 0x7f8bdc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19e5f0 con 0x7f8bdc108430 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19e760 con 0x7f8bdc103310 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.412+0000 7f8be35ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19e920 con 0x7f8bdc19db70 2023-09-23T04:50:39.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be139b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 0x7f8bdc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34172/0 (socket says 172.21.15.5:34172) 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 -- 172.21.15.5:0/101650042 learned_addr learned my addr 172.21.15.5:0/101650042 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be0b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc108430 0x7f8bdc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 -- 172.21.15.5:0/101650042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 msgr2=0x7f8bdc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 0x7f8bdc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 -- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc108430 msgr2=0x7f8bdc199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc108430 0x7f8bdc199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 -- 172.21.15.5:0/101650042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0095f0 con 0x7f8bdc19db70 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be139b700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 0x7f8bdc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ef30 tx=0x7f8bd400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8bd27fc700 1 -- 172.21.15.5:0/101650042 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4018070 con 0x7f8bdc19db70 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8bd27fc700 1 -- 172.21.15.5:0/101650042 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd400f040 con 0x7f8bdc19db70 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.413+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc1a5e20 con 0x7f8bdc19db70 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.414+0000 7f8bd27fc700 1 -- 172.21.15.5:0/101650042 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400cd00 con 0x7f8bdc19db70 2023-09-23T04:50:39.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.414+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a62a0 con 0x7f8bdc19db70 2023-09-23T04:50:39.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.417+0000 7f8bd27fc700 1 -- 172.21.15.5:0/101650042 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f8bd4003a00 con 0x7f8bdc19db70 2023-09-23T04:50:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 Processor -- start 2023-09-23T04:50:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 -- start start 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bd27fc700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 0x7f8bc80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49fda0 0x55b7ce4aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bd27fc700 1 -- 172.21.15.5:0/101650042 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8bd40cd150 con 0x7f8bdc19db70 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce4084e0 con 0x55b7ce49f480 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce3f9ca0 con 0x55b7ce4aac50 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.420+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce3f75c0 con 0x55b7ce49fda0 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 0x7f8bc80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be139b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52126/0 (socket says 172.21.15.5:52126) 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be0b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49fda0 0x55b7ce4aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be139b700 1 -- 172.21.15.5:0/2089635609 learned_addr learned my addr 172.21.15.5:0/2089635609 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 -- 172.21.15.5:0/2089635609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49fda0 msgr2=0x55b7ce4aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49fda0 0x55b7ce4aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 -- 172.21.15.5:0/2089635609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 msgr2=0x55b7ce49f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 -- 172.21.15.5:0/2089635609 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ce455950 con 0x55b7ce4aac50 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be139b700 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4ad040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ca460 tx=0x7f8bd4099b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c6ec6ff67ffd24 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.421+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 0x7f8bc80a3c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b2e0 tx=0x7f8bcc005c20 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.422+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2089635609 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4004ad0 con 0x55b7ce4aac50 2023-09-23T04:50:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.422+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2089635609 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd401b320 con 0x55b7ce4aac50 2023-09-23T04:50:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.422+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2089635609 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4099f70 con 0x55b7ce4aac50 2023-09-23T04:50:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2089635609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 msgr2=0x55b7ce4ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4ad040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ca460 tx=0x7f8bd4099b30 comp rx=0 tx=0).stop 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2089635609 shutdown_connections 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4ad040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49f480 0x55b7ce49f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2089635609 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49fda0 0x55b7ce4aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2089635609 >> 172.21.15.5:0/2089635609 conn(0x55b7ce4142e0 msgr2=0x55b7ce49b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2089635609 shutdown_connections 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2089635609 wait complete. 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.423+0000 7f8bf7022b80 1 -- start start 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 Processor -- start 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce3f2eb0 con 0x55b7ce49fda0 2023-09-23T04:50:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce38f790 con 0x55b7ce4aac50 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce38f430 con 0x55b7ce49f480 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8bf7022b80 10 client.0 _ll_register_callbacks cb 0x55b7ce1ec880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8be139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8be139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34200/0 (socket says 172.21.15.5:34200) 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.424+0000 7f8be139b700 1 -- 172.21.15.5:0/734103353 learned_addr learned my addr 172.21.15.5:0/734103353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 -- 172.21.15.5:0/734103353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 msgr2=0x55b7ce4a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 -- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49fda0 msgr2=0x55b7ce4a7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 -- 172.21.15.5:0/734103353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd8009dd0 con 0x55b7ce49f480 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8be139b700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8012e30 tx=0x7f8bd800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05b4eff93d6d69 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8bc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x55b7ce49f480 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7ce4932d0 con 0x55b7ce49f480 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.425+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8bd800fb80 con 0x55b7ce49f480 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bd800fd40 con 0x55b7ce49f480 2023-09-23T04:50:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 10 client.26854 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f8bdc1a62a0 con 0x55b7ce49f480 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7ce4932d0 con 0x55b7ce49f480 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8bd801c420 con 0x55b7ce49f480 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 10 client.26854 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 20 client.26854 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce4ba800 0x55b7ce4bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 4 client.26854 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.426+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7ce3ebe90 con 0x55b7ce4ba800 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.427+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce4ba800 0x55b7ce4bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.428+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce4ba800 0x55b7ce4bcc20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6464a21a040bd8bd server_cookie=e0f0b23d30cc7580 in_seq=0 out_seq=0 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.428+0000 7f8bc4ff9700 10 client.26854 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.428+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/734103353 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7ce3ebe90 con 0x55b7ce4ba800 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.428+0000 7f8bc4ff9700 10 client.26854 handle_command_reply: tid=0 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.428+0000 7f8bf7022b80 1 client.26854 shutdown 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 msgr2=0x55b7ce4a79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8012e30 tx=0x7f8bd800f6a0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 shutdown_connections 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce4ba800 0x55b7ce4bcc20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a79a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/734103353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bc4ff9700 0 client.26854 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 >> 172.21.15.5:0/734103353 conn(0x55b7ce4142e0 msgr2=0x55b7ce49b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 shutdown_connections 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 1 -- 172.21.15.5:0/734103353 wait complete. 2023-09-23T04:50:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.429+0000 7f8bf7022b80 20 client.26854 trim_cache size 0 max 16384 2023-09-23T04:50:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 Processor -- start 2023-09-23T04:50:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 -- start start 2023-09-23T04:50:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce2739a0 con 0x55b7ce4aac50 2023-09-23T04:50:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce49d150 con 0x55b7ce49fda0 2023-09-23T04:50:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.449+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce4a27d0 con 0x55b7ce49f480 2023-09-23T04:50:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34216/0 (socket says 172.21.15.5:34216) 2023-09-23T04:50:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 -- 172.21.15.5:0/2173180422 learned_addr learned my addr 172.21.15.5:0/2173180422 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 -- 172.21.15.5:0/2173180422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 msgr2=0x55b7ce4a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 -- 172.21.15.5:0/2173180422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 msgr2=0x55b7ce4a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 -- 172.21.15.5:0/2173180422 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ce497500 con 0x55b7ce49f480 2023-09-23T04:50:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:50:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.450+0000 7f8be139b700 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8bd801c7d0 tx=0x7f8bd800f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccad67f7e6575820 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.451+0000 7f8bc7fff700 1 -- 172.21.15.5:0/2173180422 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017040 con 0x55b7ce49f480 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.451+0000 7f8bc7fff700 1 -- 172.21.15.5:0/2173180422 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd801c9f0 con 0x55b7ce49f480 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.451+0000 7f8bc7fff700 1 -- 172.21.15.5:0/2173180422 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80055d0 con 0x55b7ce49f480 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2173180422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 msgr2=0x55b7ce4a8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8bd801c7d0 tx=0x7f8bd800f9f0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2173180422 shutdown_connections 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce4a8250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/2173180422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce4a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2173180422 >> 172.21.15.5:0/2173180422 conn(0x55b7ce4142e0 msgr2=0x55b7ce4a1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2173180422 shutdown_connections 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- 172.21.15.5:0/2173180422 wait complete. 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.452+0000 7f8bf7022b80 1 -- start start 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 Processor -- start 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce511390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce4b0e00 con 0x55b7ce4aac50 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce4b0f70 con 0x55b7ce49fda0 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ce4b1130 con 0x55b7ce49f480 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8bf7022b80 10 client.0 _ll_register_callbacks cb 0x55b7ce1ec880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8be0b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8be0b9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49182/0 (socket says 172.21.15.5:49182) 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8be0b9a700 1 -- 172.21.15.5:0/3573436455 learned_addr learned my addr 172.21.15.5:0/3573436455 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.453+0000 7f8be139b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce511390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be1b9c700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 -- 172.21.15.5:0/3573436455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 msgr2=0x55b7ce511390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce511390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 -- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 msgr2=0x55b7ce4b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 -- 172.21.15.5:0/3573436455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a62a0 con 0x55b7ce49fda0 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8be0b9a700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc06a440 tx=0x7f8bcc029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eea735f68d0a010 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8bc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02f040 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bd401b320 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.454+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8bcc0349c0 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bcc034b80 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 10 client.26807 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b7ce21de60 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd401b320 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8bcc00f5f0 con 0x55b7ce49fda0 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 10 client.26807 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 10 client.26807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 20 client.26807 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce519360 0x55b7ce51b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 4 client.26807 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b7ce3ebbb0 con 0x55b7ce519360 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.455+0000 7f8be139b700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce519360 0x55b7ce51b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.456+0000 7f8be139b700 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce519360 0x55b7ce51b780 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac59458d809dd9f server_cookie=6b4c95ee34ddd71a in_seq=0 out_seq=0 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.456+0000 7f8bc4ff9700 10 client.26807 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.457+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3573436455 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b7ce3ebbb0 con 0x55b7ce519360 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.457+0000 7f8bc4ff9700 10 client.26807 handle_command_reply: tid=0 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.457+0000 7f8bf7022b80 1 client.26807 shutdown 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 msgr2=0x55b7ce5118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc06a440 tx=0x7f8bcc029000 comp rx=0 tx=0).stop 2023-09-23T04:50:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 shutdown_connections 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b7ce519360 0x55b7ce51b780 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ce4aac50 0x55b7ce4b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b7ce49f480 0x55b7ce511390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 --2- 172.21.15.5:0/3573436455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b7ce49fda0 0x55b7ce5118d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 >> 172.21.15.5:0/3573436455 conn(0x55b7ce4142e0 msgr2=0x55b7ce4a92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bc4ff9700 0 client.26807 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 shutdown_connections 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 1 -- 172.21.15.5:0/3573436455 wait complete. 2023-09-23T04:50:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.458+0000 7f8bf7022b80 20 client.26807 trim_cache size 0 max 16384 2023-09-23T04:50:39.466 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:50:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9330f133-c7f3-446a-9940-0a410598e031", 2023-09-23T04:50:39.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:50:39.466 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 msgr2=0x7f8bc80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 0x7f8bc80a3c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b2e0 tx=0x7f8bcc005c20 comp rx=0 tx=0).stop 2023-09-23T04:50:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 msgr2=0x7f8bdc19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ef30 tx=0x7f8bd400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 shutdown_connections 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8bc80a1790 0x7f8bc80a3c50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8bdc19db70 0x7f8bdc19dff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bdc103310 0x7f8bdc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 --2- 172.21.15.5:0/101650042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc108430 0x7f8bdc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 >> 172.21.15.5:0/101650042 conn(0x7f8bdc0feb50 msgr2=0x7f8bdc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 shutdown_connections 2023-09-23T04:50:39.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.459+0000 7f8be35ff700 1 -- 172.21.15.5:0/101650042 wait complete. 2023-09-23T04:50:39.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:50:39.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 -- 172.21.15.5:0/319740426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d4103310 msgr2=0x7f58d41036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 --2- 172.21.15.5:0/319740426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d4103310 0x7f58d41036f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009990 tx=0x7f58c4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 -- 172.21.15.5:0/319740426 shutdown_connections 2023-09-23T04:50:39.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 --2- 172.21.15.5:0/319740426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 0x7f58d410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 --2- 172.21.15.5:0/319740426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d4103310 0x7f58d41036f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 --2- 172.21.15.5:0/319740426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103cc0 0x7f58d4107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 -- 172.21.15.5:0/319740426 >> 172.21.15.5:0/319740426 conn(0x7f58d40feb50 msgr2=0x7f58d4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 -- 172.21.15.5:0/319740426 shutdown_connections 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.719+0000 7f58d89ff700 1 -- 172.21.15.5:0/319740426 wait complete. 2023-09-23T04:50:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.720+0000 7f58d89ff700 1 Processor -- start 2023-09-23T04:50:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 -- start start 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 0x7f58d4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 0x7f58d419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419e610 con 0x7f58d4103310 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419e780 con 0x7f58d4108430 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.721+0000 7f58d89ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d419e940 con 0x7f58d419db90 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 0x7f58d419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 0x7f58d4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52184/0 (socket says 172.21.15.5:52184) 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d259c700 1 -- 172.21.15.5:0/3926632126 learned_addr learned my addr 172.21.15.5:0/3926632126 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3926632126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 msgr2=0x7f58d4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 0x7f58d4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 msgr2=0x7f58d41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3926632126 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc009520 con 0x7f58d419db90 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d259c700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.722+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 0x7f58d419e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00efc0 tx=0x7f58cc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.723+0000 7f58cb7fe700 1 -- 172.21.15.5:0/3926632126 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc018070 con 0x7f58d419db90 2023-09-23T04:50:39.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.723+0000 7f58cb7fe700 1 -- 172.21.15.5:0/3926632126 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58cc00f040 con 0x7f58d419db90 2023-09-23T04:50:39.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.723+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c40095f0 con 0x7f58d419db90 2023-09-23T04:50:39.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.723+0000 7f58cb7fe700 1 -- 172.21.15.5:0/3926632126 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00cd40 con 0x7f58d419db90 2023-09-23T04:50:39.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.723+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d41a6190 con 0x7f58d419db90 2023-09-23T04:50:39.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.727+0000 7f58cb7fe700 1 -- 172.21.15.5:0/3926632126 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f58cc003a00 con 0x7f58d419db90 2023-09-23T04:50:39.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.727+0000 7f58ec2a5b80 1 Processor -- start 2023-09-23T04:50:39.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 -- start start 2023-09-23T04:50:39.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916ee4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916e42460 con 0x559916ed9d20 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916e33c20 con 0x559916ee4bd0 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.728+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916e31540 con 0x559916ed9400 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34238/0 (socket says 172.21.15.5:34238) 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 -- 172.21.15.5:0/1865977160 learned_addr learned my addr 172.21.15.5:0/1865977160 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58d259c700 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916ee4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 -- 172.21.15.5:0/1865977160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 msgr2=0x559916ee6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 -- 172.21.15.5:0/1865977160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 msgr2=0x559916ee4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916ee4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cb7fe700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 0x7f58c00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cbfff700 1 -- 172.21.15.5:0/1865977160 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559916e8f8d0 con 0x559916ed9400 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.729+0000 7f58cb7fe700 1 -- 172.21.15.5:0/3926632126 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f58cc0cd1e0 con 0x7f58d419db90 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.730+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58bc013810 tx=0x7f58bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea47e7848470913 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.730+0000 7f58cbfff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 0x7f58c00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.730+0000 7f58c8ff9700 1 -- 172.21.15.5:0/1865977160 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc018070 con 0x559916ed9400 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.730+0000 7f58c8ff9700 1 -- 172.21.15.5:0/1865977160 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58bc004510 con 0x559916ed9400 2023-09-23T04:50:39.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.730+0000 7f58c8ff9700 1 -- 172.21.15.5:0/1865977160 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc010690 con 0x559916ed9400 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1865977160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 msgr2=0x559916ed97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f58bc013810 tx=0x7f58bc009260 comp rx=0 tx=0).stop 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58cbfff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 0x7f58c00a3c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58bc013fd0 tx=0x7f58bc01f6d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1865977160 shutdown_connections 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916ed97e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1865977160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916ee4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1865977160 >> 172.21.15.5:0/1865977160 conn(0x559916e4e260 msgr2=0x559916ed5790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1865977160 shutdown_connections 2023-09-23T04:50:39.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.731+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1865977160 wait complete. 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.732+0000 7f58ec2a5b80 1 -- start start 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.732+0000 7f58ec2a5b80 1 Processor -- start 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9400 0x559916ee45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916e2ce30 con 0x559916ed9400 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916dc9710 con 0x559916ee4bd0 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916dc93b0 con 0x559916ed9d20 2023-09-23T04:50:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58ec2a5b80 10 client.0 _ll_register_callbacks cb 0x559916c26800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58cbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34252/0 (socket says 172.21.15.5:34252) 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58cbfff700 1 -- 172.21.15.5:0/1458490206 learned_addr learned my addr 172.21.15.5:0/1458490206 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58d259c700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9400 0x559916ee45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.733+0000 7f58cbfff700 1 -- 172.21.15.5:0/1458490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 msgr2=0x559916ee1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58cbfff700 1 -- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9400 msgr2=0x559916ee45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9400 0x559916ee45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58cbfff700 1 -- 172.21.15.5:0/1458490206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc013480 con 0x559916ed9d20 2023-09-23T04:50:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58cbfff700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003d60 tx=0x7f58bc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e2077cff2c6c56 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc018070 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559916ecd250 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.734+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f58bc003ab0 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58bc003880 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 10 client.26872 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559916c57de0 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559916ecd250 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f58bc028070 con 0x559916ed9d20 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 10 client.26872 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:39.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 20 client.26872 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916ef45f0 0x559916ef6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 4 client.26872 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.735+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559916e25e10 con 0x559916ef45f0 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.736+0000 7f58d259c700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916ef45f0 0x559916ef6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.736+0000 7f58d259c700 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916ef45f0 0x559916ef6a10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba0b90a3b294e4f server_cookie=18a797b3e31350f9 in_seq=0 out_seq=0 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.736+0000 7f58b9ffb700 10 client.26872 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58b9ffb700 1 -- 172.21.15.5:0/1458490206 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559916e25e10 con 0x559916ef45f0 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58b9ffb700 10 client.26872 handle_command_reply: tid=0 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 client.26872 shutdown 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 msgr2=0x559916ee1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003d60 tx=0x7f58bc01b000 comp rx=0 tx=0).stop 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 shutdown_connections 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916ef45f0 0x559916ef6a10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58b9ffb700 0 client.26872 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9400 0x559916ee45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/1458490206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9d20 0x559916ee1850 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 >> 172.21.15.5:0/1458490206 conn(0x559916e4e260 msgr2=0x559916ed5320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.737+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 shutdown_connections 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.738+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/1458490206 wait complete. 2023-09-23T04:50:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.738+0000 7f58ec2a5b80 20 client.26872 trim_cache size 0 max 16384 2023-09-23T04:50:39.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.760+0000 7f58ec2a5b80 1 Processor -- start 2023-09-23T04:50:39.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.760+0000 7f58ec2a5b80 1 -- start start 2023-09-23T04:50:39.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916edd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916cad920 con 0x559916ed9d20 2023-09-23T04:50:39.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916edfdf0 con 0x559916ee4bd0 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916ef0580 con 0x559916ed9400 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34268/0 (socket says 172.21.15.5:34268) 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 -- 172.21.15.5:0/477777506 learned_addr learned my addr 172.21.15.5:0/477777506 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58cbfff700 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916edd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 -- 172.21.15.5:0/477777506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 msgr2=0x559916ee1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 -- 172.21.15.5:0/477777506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 msgr2=0x559916edd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916edd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.761+0000 7f58d259c700 1 -- 172.21.15.5:0/477777506 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559916c40c60 con 0x559916ed9400 2023-09-23T04:50:39.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.762+0000 7f58d259c700 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559916ef72e0 tx=0x7f58c4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f2c0fe88f1bef8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.762+0000 7f58c97fa700 1 -- 172.21.15.5:0/477777506 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c401d040 con 0x559916ed9400 2023-09-23T04:50:39.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.762+0000 7f58c97fa700 1 -- 172.21.15.5:0/477777506 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c401e040 con 0x559916ed9400 2023-09-23T04:50:39.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.762+0000 7f58c97fa700 1 -- 172.21.15.5:0/477777506 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40055b0 con 0x559916ed9400 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/477777506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 msgr2=0x559916edcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x559916ef72e0 tx=0x7f58c4009550 comp rx=0 tx=0).stop 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/477777506 shutdown_connections 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916edcb90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/477777506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916edd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/477777506 >> 172.21.15.5:0/477777506 conn(0x559916e4e260 msgr2=0x559916eefe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/477777506 shutdown_connections 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/477777506 wait complete. 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.763+0000 7f58ec2a5b80 1 -- start start 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 Processor -- start 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916eeb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916ee9ff0 con 0x559916ed9d20 2023-09-23T04:50:39.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916eea160 con 0x559916ee4bd0 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916eea320 con 0x559916ed9400 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.764+0000 7f58ec2a5b80 10 client.0 _ll_register_callbacks cb 0x559916c26800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34272/0 (socket says 172.21.15.5:34272) 2023-09-23T04:50:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d259c700 1 -- 172.21.15.5:0/3871088819 learned_addr learned my addr 172.21.15.5:0/3871088819 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58cbfff700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916eeb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3871088819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 msgr2=0x559916eeb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 msgr2=0x559916eeb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916eeb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 -- 172.21.15.5:0/3871088819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d4110820 con 0x559916ee4bd0 2023-09-23T04:50:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d259c700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58d2d9d700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee98b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0ca4f0 tx=0x7f58cc0998e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95983d9625c3b47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.765+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc01b320 con 0x559916ee4bd0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58bc004510 con 0x559916ee4bd0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f58cc004ad0 con 0x559916ee4bd0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58cc09a4a0 con 0x559916ee4bd0 2023-09-23T04:50:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 10 client.26825 fetch_fsmap learned FSMap version 33 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559916c7e250 con 0x559916ee4bd0 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58cc07c080 con 0x559916ee4bd0 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f58cc099290 con 0x559916ee4bd0 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 10 client.26825 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 20 client.26825 populate_metadata read hostname 'smithi005' 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916f495b0 0x559916f4b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 4 client.26825 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559916e25b30 con 0x559916f495b0 2023-09-23T04:50:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.766+0000 7f58d259c700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916f495b0 0x559916f4b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58d259c700 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916f495b0 0x559916f4b9d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c954419c5245a5 server_cookie=34a570ac607b90aa in_seq=0 out_seq=0 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58b9ffb700 10 client.26825 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58b9ffb700 1 -- 172.21.15.5:0/3871088819 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559916e25b30 con 0x559916f495b0 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58b9ffb700 10 client.26825 handle_command_reply: tid=0 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58ec2a5b80 1 client.26825 shutdown 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 msgr2=0x559916ee98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.767+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee98b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0ca4f0 tx=0x7f58cc0998e0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 shutdown_connections 2023-09-23T04:50:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x559916f495b0 0x559916f4b9d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559916ee4bd0 0x559916ee98b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x559916ed9400 0x559916eeb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58b9ffb700 0 client.26825 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 --2- 172.21.15.5:0/3871088819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559916ed9d20 0x559916eeb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 >> 172.21.15.5:0/3871088819 conn(0x559916e4e260 msgr2=0x559916edcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 shutdown_connections 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 1 -- 172.21.15.5:0/3871088819 wait complete. 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.768+0000 7f58ec2a5b80 20 client.26825 trim_cache size 0 max 16384 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:50:39.775 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "9330f133-c7f3-446a-9940-0a410598e031": { 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9330f133-c7f3-446a-9940-0a410598e031", 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:50:39.776 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 msgr2=0x7f58c00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 0x7f58c00a3c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58bc013fd0 tx=0x7f58bc01f6d0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 msgr2=0x7f58d419e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 0x7f58d419e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00efc0 tx=0x7f58cc00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 shutdown_connections 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f58c00a1790 0x7f58c00a3c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f58d419db90 0x7f58d419e010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d4103310 0x7f58d41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 --2- 172.21.15.5:0/3926632126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58d4108430 0x7f58d4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:50:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 >> 172.21.15.5:0/3926632126 conn(0x7f58d40feb50 msgr2=0x7f58d4106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:50:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 shutdown_connections 2023-09-23T04:50:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:50:39.769+0000 7f58d89ff700 1 -- 172.21.15.5:0/3926632126 wait complete. 2023-09-23T04:50:39.789 INFO:tasks.cephfs.filesystem:scrub status for tag:9330f133-c7f3-446a-9940-0a410598e031 - {'path': '/', 'tag': '9330f133-c7f3-446a-9940-0a410598e031', 'options': 'recursive,force'} 2023-09-23T04:50:39.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:39 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:39.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:39 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:39.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:39 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:39.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:39 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:39.940 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263102 5.80 MB/sec execute 75 sec latency 20.199 ms 2023-09-23T04:50:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:39 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:39 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:39 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:39 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:39 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:39 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:39 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:40.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:39 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:40.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 265405 5.88 MB/sec execute 76 sec latency 16.086 ms 2023-09-23T04:50:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:50:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:40 smithi005 ceph-mon[97638]: pgmap v135: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 301 KiB/s wr, 59 op/s 2023-09-23T04:50:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:50:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:40 smithi118 ceph-mon[134297]: pgmap v135: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 301 KiB/s wr, 59 op/s 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:50:41.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:50:41.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:40 smithi069 ceph-mon[133931]: pgmap v135: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 301 KiB/s wr, 59 op/s 2023-09-23T04:50:41.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266661 5.89 MB/sec execute 77 sec latency 11.097 ms 2023-09-23T04:50:42.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 268838 5.96 MB/sec execute 78 sec latency 13.608 ms 2023-09-23T04:50:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:42 smithi005 ceph-mon[97638]: pgmap v136: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 836 KiB/s wr, 141 op/s 2023-09-23T04:50:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:42 smithi118 ceph-mon[134297]: pgmap v136: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 836 KiB/s wr, 141 op/s 2023-09-23T04:50:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:42 smithi069 ceph-mon[133931]: pgmap v136: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 836 KiB/s wr, 141 op/s 2023-09-23T04:50:43.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 270157 5.99 MB/sec execute 79 sec latency 20.211 ms 2023-09-23T04:50:44.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 272380 6.05 MB/sec execute 80 sec latency 12.062 ms 2023-09-23T04:50:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:44 smithi005 ceph-mon[97638]: pgmap v137: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 878 KiB/s wr, 139 op/s 2023-09-23T04:50:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:44 smithi118 ceph-mon[134297]: pgmap v137: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 878 KiB/s wr, 139 op/s 2023-09-23T04:50:45.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:44 smithi069 ceph-mon[133931]: pgmap v137: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 878 KiB/s wr, 139 op/s 2023-09-23T04:50:45.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273753 6.08 MB/sec execute 81 sec latency 10.386 ms 2023-09-23T04:50:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:45 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:45 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:45 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:45 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:46.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:45 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:46.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:45 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:46.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276050 6.15 MB/sec execute 82 sec latency 10.785 ms 2023-09-23T04:50:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:46 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:46 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:46 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:50:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:46 smithi005 ceph-mon[97638]: pgmap v138: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 136 op/s 2023-09-23T04:50:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:46 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:46 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:46 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:50:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:46 smithi118 ceph-mon[134297]: pgmap v138: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 136 op/s 2023-09-23T04:50:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:46 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:46 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:46 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:50:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:46 smithi069 ceph-mon[133931]: pgmap v138: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 136 op/s 2023-09-23T04:50:47.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 277313 6.16 MB/sec execute 83 sec latency 12.643 ms 2023-09-23T04:50:48.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 279185 6.18 MB/sec execute 84 sec latency 70.717 ms 2023-09-23T04:50:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:48 smithi005 ceph-mon[97638]: pgmap v139: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2023-09-23T04:50:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:48 smithi118 ceph-mon[134297]: pgmap v139: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2023-09-23T04:50:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:48 smithi069 ceph-mon[133931]: pgmap v139: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 229 op/s 2023-09-23T04:50:49.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280696 6.24 MB/sec execute 85 sec latency 15.918 ms 2023-09-23T04:50:50.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 282800 6.26 MB/sec execute 86 sec latency 11.694 ms 2023-09-23T04:50:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:51 smithi005 ceph-mon[97638]: pgmap v140: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-23T04:50:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:51 smithi118 ceph-mon[134297]: pgmap v140: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-23T04:50:51.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284249 6.32 MB/sec execute 87 sec latency 11.141 ms 2023-09-23T04:50:51.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:51 smithi069 ceph-mon[133931]: pgmap v140: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2023-09-23T04:50:52.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 286187 6.32 MB/sec execute 88 sec latency 73.704 ms 2023-09-23T04:50:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:52 smithi005 ceph-mon[97638]: pgmap v141: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 269 op/s 2023-09-23T04:50:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:52 smithi118 ceph-mon[134297]: pgmap v141: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 269 op/s 2023-09-23T04:50:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:52 smithi069 ceph-mon[133931]: pgmap v141: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 269 op/s 2023-09-23T04:50:53.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287670 6.39 MB/sec execute 89 sec latency 77.791 ms 2023-09-23T04:50:54.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 288550 6.34 MB/sec execute 90 sec latency 203.706 ms 2023-09-23T04:50:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:54 smithi005 ceph-mon[97638]: pgmap v142: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:54 smithi118 ceph-mon[134297]: pgmap v142: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:54 smithi069 ceph-mon[133931]: pgmap v142: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2023-09-23T04:50:55.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 290948 6.44 MB/sec execute 91 sec latency 32.095 ms 2023-09-23T04:50:56.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291495 6.39 MB/sec execute 92 sec latency 134.583 ms 2023-09-23T04:50:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:56 smithi005 ceph-mon[97638]: pgmap v143: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-23T04:50:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:56 smithi118 ceph-mon[134297]: pgmap v143: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-23T04:50:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:56 smithi069 ceph-mon[133931]: pgmap v143: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2023-09-23T04:50:57.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 293276 6.39 MB/sec execute 93 sec latency 94.502 ms 2023-09-23T04:50:58.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294811 6.46 MB/sec execute 94 sec latency 11.768 ms 2023-09-23T04:50:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:50:58 smithi005 ceph-mon[97638]: pgmap v144: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 347 op/s 2023-09-23T04:50:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:50:58 smithi118 ceph-mon[134297]: pgmap v144: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 347 op/s 2023-09-23T04:50:59.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:50:58 smithi069 ceph-mon[133931]: pgmap v144: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 347 op/s 2023-09-23T04:50:59.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 296857 6.46 MB/sec execute 95 sec latency 13.227 ms 2023-09-23T04:51:00.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 298165 6.51 MB/sec execute 96 sec latency 49.183 ms 2023-09-23T04:51:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:01 smithi005 ceph-mon[97638]: pgmap v145: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-09-23T04:51:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:01 smithi118 ceph-mon[134297]: pgmap v145: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-09-23T04:51:01.941 INFO:tasks.workunit.client.0.smithi005.stdout: 1 298706 6.46 MB/sec execute 97 sec latency 58.666 ms 2023-09-23T04:51:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:01 smithi069 ceph-mon[133931]: pgmap v145: 129 pgs: 129 active+clean; 176 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 254 op/s 2023-09-23T04:51:02.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 301141 6.52 MB/sec execute 98 sec latency 9.692 ms 2023-09-23T04:51:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:02 smithi005 ceph-mon[97638]: pgmap v146: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 348 op/s 2023-09-23T04:51:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:02 smithi118 ceph-mon[134297]: pgmap v146: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 348 op/s 2023-09-23T04:51:03.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:02 smithi069 ceph-mon[133931]: pgmap v146: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 348 op/s 2023-09-23T04:51:03.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 302303 6.52 MB/sec execute 99 sec latency 11.062 ms 2023-09-23T04:51:04.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 304519 6.58 MB/sec execute 100 sec latency 31.966 ms 2023-09-23T04:51:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:04 smithi005 ceph-mon[97638]: pgmap v147: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-23T04:51:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:04 smithi118 ceph-mon[134297]: pgmap v147: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-23T04:51:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:04 smithi069 ceph-mon[133931]: pgmap v147: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 286 op/s 2023-09-23T04:51:05.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 305776 6.58 MB/sec execute 101 sec latency 17.481 ms 2023-09-23T04:51:06.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 308058 6.63 MB/sec execute 102 sec latency 10.796 ms 2023-09-23T04:51:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:06 smithi005 ceph-mon[97638]: pgmap v148: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2023-09-23T04:51:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:06 smithi118 ceph-mon[134297]: pgmap v148: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2023-09-23T04:51:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:06 smithi069 ceph-mon[133931]: pgmap v148: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 284 op/s 2023-09-23T04:51:07.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.63 MB/sec execute 103 sec latency 100.068 ms 2023-09-23T04:51:08.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.57 MB/sec execute 104 sec latency 1100.188 ms 2023-09-23T04:51:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:08 smithi005 ceph-mon[97638]: pgmap v149: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 375 op/s 2023-09-23T04:51:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:08 smithi118 ceph-mon[134297]: pgmap v149: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 375 op/s 2023-09-23T04:51:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:08 smithi069 ceph-mon[133931]: pgmap v149: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 375 op/s 2023-09-23T04:51:09.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:51:09.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.51 MB/sec execute 105 sec latency 2100.274 ms 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 -- 172.21.15.5:0/2665629611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 msgr2=0x7fad2c10aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 --2- 172.21.15.5:0/2665629611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 0x7fad2c10aec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad20009990 tx=0x7fad20009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 -- 172.21.15.5:0/2665629611 shutdown_connections 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 --2- 172.21.15.5:0/2665629611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 0x7fad2c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 --2- 172.21.15.5:0/2665629611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 --2- 172.21.15.5:0/2665629611 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 0x7fad2c10aec0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 -- 172.21.15.5:0/2665629611 >> 172.21.15.5:0/2665629611 conn(0x7fad2c076f30 msgr2=0x7fad2c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.041+0000 7fad339ff700 1 -- 172.21.15.5:0/2665629611 shutdown_connections 2023-09-23T04:51:10.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.042+0000 7fad339ff700 1 -- 172.21.15.5:0/2665629611 wait complete. 2023-09-23T04:51:10.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.044+0000 7fad339ff700 1 Processor -- start 2023-09-23T04:51:10.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.044+0000 7fad339ff700 1 -- start start 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 0x7fad2c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 0x7fad2c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c19e220 con 0x7fad2c1020c0 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c19e390 con 0x7fad2c10b490 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad339ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c19e550 con 0x7fad2c1029e0 2023-09-23T04:51:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 0x7fad2c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad3179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad3179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36762/0 (socket says 172.21.15.5:36762) 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.045+0000 7fad3179b700 1 -- 172.21.15.5:0/1585264399 learned_addr learned my addr 172.21.15.5:0/1585264399 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.046+0000 7fad3179b700 1 -- 172.21.15.5:0/1585264399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 msgr2=0x7fad2c199780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.046+0000 7fad3179b700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 0x7fad2c199780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.046+0000 7fad3179b700 1 -- 172.21.15.5:0/1585264399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 msgr2=0x7fad2c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.046+0000 7fad3179b700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 0x7fad2c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.046+0000 7fad3179b700 1 -- 172.21.15.5:0/1585264399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009520 con 0x7fad2c1020c0 2023-09-23T04:51:10.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad3179b700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad2800eb30 tx=0x7fad2800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad1e7fc700 1 -- 172.21.15.5:0/1585264399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28009880 con 0x7fad2c1020c0 2023-09-23T04:51:10.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200095f0 con 0x7fad2c1020c0 2023-09-23T04:51:10.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad1e7fc700 1 -- 172.21.15.5:0/1585264399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad280044e0 con 0x7fad2c1020c0 2023-09-23T04:51:10.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad1e7fc700 1 -- 172.21.15.5:0/1585264399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28003e90 con 0x7fad2c1020c0 2023-09-23T04:51:10.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.047+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c19ea20 con 0x7fad2c1020c0 2023-09-23T04:51:10.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.051+0000 7fad1e7fc700 1 -- 172.21.15.5:0/1585264399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fad28004000 con 0x7fad2c1020c0 2023-09-23T04:51:10.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.052+0000 7fad1e7fc700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 0x7fad180a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.052+0000 7fad472fab80 1 Processor -- start 2023-09-23T04:51:10.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.052+0000 7fad1e7fc700 1 -- 172.21.15.5:0/1585264399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fad28014070 con 0x7fad2c1020c0 2023-09-23T04:51:10.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.052+0000 7fad472fab80 1 -- start start 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78824d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad30f9a700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 0x7fad180a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d788260ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d7882637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7881b6460 con 0x55d78824d400 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7881a7c20 con 0x55d78824dd20 2023-09-23T04:51:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad472fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7881a5540 con 0x55d7882613e0 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad30f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d788260ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d7882637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d7882637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42938/0 (socket says 172.21.15.5:42938) 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad30f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d788260ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:35900/0 (socket says 172.21.15.5:35900) 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad3179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78824d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.053+0000 7fad30f9a700 1 -- 172.21.15.5:0/3525191416 learned_addr learned my addr 172.21.15.5:0/3525191416 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 -- 172.21.15.5:0/3525191416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 msgr2=0x55d7882637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d7882637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 -- 172.21.15.5:0/3525191416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 msgr2=0x55d788260ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d788260ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 -- 172.21.15.5:0/3525191416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7882038d0 con 0x55d78824d400 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad3179b700 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78824d7e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad280cae70 tx=0x7fad2809a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292e34ae0c50f1d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad30f9a700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 0x7fad180a3c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad200098e0 tx=0x7fad2000b690 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad177fe700 1 -- 172.21.15.5:0/3525191416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280ba0d0 con 0x55d78824d400 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad177fe700 1 -- 172.21.15.5:0/3525191416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad28016330 con 0x55d78824d400 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.054+0000 7fad177fe700 1 -- 172.21.15.5:0/3525191416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280a2a90 con 0x55d78824d400 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 -- 172.21.15.5:0/3525191416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 msgr2=0x55d78824d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78824d7e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad280cae70 tx=0x7fad2809a560 comp rx=0 tx=0).stop 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 -- 172.21.15.5:0/3525191416 shutdown_connections 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d7882637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78824d7e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 --2- 172.21.15.5:0/3525191416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d788260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 -- 172.21.15.5:0/3525191416 >> 172.21.15.5:0/3525191416 conn(0x55d7881c2260 msgr2=0x55d788249790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 -- 172.21.15.5:0/3525191416 shutdown_connections 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.055+0000 7fad472fab80 1 -- 172.21.15.5:0/3525191416 wait complete. 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.056+0000 7fad472fab80 1 -- start start 2023-09-23T04:51:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.056+0000 7fad472fab80 1 Processor -- start 2023-09-23T04:51:10.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.056+0000 7fad472fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.056+0000 7fad472fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.056+0000 7fad472fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7882613e0 0x55d78825ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad472fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7881a0e30 con 0x55d7882613e0 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad472fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78813d710 con 0x55d78824dd20 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad472fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78813d3b0 con 0x55d78824d400 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad472fab80 10 client.0 _ll_register_callbacks cb 0x55d787f9a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42952/0 (socket says 172.21.15.5:42952) 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 -- 172.21.15.5:0/899858880 learned_addr learned my addr 172.21.15.5:0/899858880 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad30f9a700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad31f9c700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7882613e0 0x55d78825ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 -- 172.21.15.5:0/899858880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 msgr2=0x55d78825e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 -- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7882613e0 msgr2=0x55d78825ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7882613e0 0x55d78825ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 -- 172.21.15.5:0/899858880 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c19ea20 con 0x55d78824d400 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad3179b700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c5b0 tx=0x7fad28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23227e8ccebc69c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.057+0000 7fad14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800cc90 con 0x55d78824d400 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d788241250 con 0x55d78824d400 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fad2809a690 con 0x55d78824d400 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad2809e7e0 con 0x55d78824d400 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad472fab80 10 client.26887 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:10.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d787fcbde0 con 0x55d78824d400 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.058+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d788241250 con 0x55d78824d400 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fad28099b80 con 0x55d78824d400 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 10 client.26887 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 10 client.26887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 20 client.26887 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d788270af0 0x55d788272ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 4 client.26887 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d788199e10 con 0x55d788270af0 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad30f9a700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d788270af0 0x55d788272ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad30f9a700 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d788270af0 0x55d788272ee0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57fe7351a491ac90 server_cookie=8bbf352e3d7e442f in_seq=0 out_seq=0 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.059+0000 7fad14ff9700 10 client.26887 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad14ff9700 1 -- 172.21.15.5:0/899858880 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d788199e10 con 0x55d788270af0 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad14ff9700 10 client.26887 handle_command_reply: tid=0 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 client.26887 shutdown 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 msgr2=0x55d78825e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c5b0 tx=0x7fad28004750 comp rx=0 tx=0).stop 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 shutdown_connections 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d788270af0 0x55d788272ee0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7882613e0 0x55d78825ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d78824d400 0x55d78825e1e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 --2- 172.21.15.5:0/899858880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad14ff9700 0 client.26887 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.060+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 >> 172.21.15.5:0/899858880 conn(0x55d7881c2260 msgr2=0x55d788249320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.061+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 shutdown_connections 2023-09-23T04:51:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.061+0000 7fad472fab80 1 -- 172.21.15.5:0/899858880 wait complete. 2023-09-23T04:51:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.061+0000 7fad472fab80 20 client.26887 trim_cache size 0 max 16384 2023-09-23T04:51:10.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.079+0000 7fad472fab80 1 Processor -- start 2023-09-23T04:51:10.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 -- start start 2023-09-23T04:51:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78825ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788021920 con 0x55d78824d400 2023-09-23T04:51:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78825a0d0 con 0x55d78824dd20 2023-09-23T04:51:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad472fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78824e390 con 0x55d7882613e0 2023-09-23T04:51:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.080+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.081+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42966/0 (socket says 172.21.15.5:42966) 2023-09-23T04:51:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.081+0000 7fad31f9c700 1 -- 172.21.15.5:0/947579530 learned_addr learned my addr 172.21.15.5:0/947579530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad3179b700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78825ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 -- 172.21.15.5:0/947579530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 msgr2=0x55d78825d0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad30f9a700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 -- 172.21.15.5:0/947579530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 msgr2=0x55d78825ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78825ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 -- 172.21.15.5:0/947579530 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d787fb4c60 con 0x55d7882613e0 2023-09-23T04:51:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad3179b700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78825ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.082+0000 7fad31f9c700 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad24012900 tx=0x7fad24009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ba0da5b0c554fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.083+0000 7fad17fff700 1 -- 172.21.15.5:0/947579530 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400e040 con 0x55d7882613e0 2023-09-23T04:51:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.083+0000 7fad17fff700 1 -- 172.21.15.5:0/947579530 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad2400f040 con 0x55d7882613e0 2023-09-23T04:51:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.083+0000 7fad17fff700 1 -- 172.21.15.5:0/947579530 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad24007a60 con 0x55d7882613e0 2023-09-23T04:51:10.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 -- 172.21.15.5:0/947579530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 msgr2=0x55d78825b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad24012900 tx=0x7fad24009920 comp rx=0 tx=0).stop 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 -- 172.21.15.5:0/947579530 shutdown_connections 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d78825b470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824d400 0x55d78825ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.084+0000 7fad472fab80 1 --2- 172.21.15.5:0/947579530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824dd20 0x55d78825d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.085+0000 7fad472fab80 1 -- 172.21.15.5:0/947579530 >> 172.21.15.5:0/947579530 conn(0x55d7881c2260 msgr2=0x55d78825c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.086+0000 7fad472fab80 1 -- 172.21.15.5:0/947579530 shutdown_connections 2023-09-23T04:51:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.086+0000 7fad472fab80 1 -- 172.21.15.5:0/947579530 wait complete. 2023-09-23T04:51:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.087+0000 7fad472fab80 1 -- start start 2023-09-23T04:51:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.087+0000 7fad472fab80 1 Processor -- start 2023-09-23T04:51:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824d400 0x55d788267e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824dd20 0x55d788266150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788266ca0 con 0x55d78824dd20 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788266e10 con 0x55d78824d400 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d788266fd0 con 0x55d7882613e0 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad472fab80 10 client.0 _ll_register_callbacks cb 0x7fad240097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad31f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42978/0 (socket says 172.21.15.5:42978) 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.088+0000 7fad31f9c700 1 -- 172.21.15.5:0/1582670382 learned_addr learned my addr 172.21.15.5:0/1582670382 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.089+0000 7fad31f9c700 1 -- 172.21.15.5:0/1582670382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824d400 msgr2=0x55d788267e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.089+0000 7fad31f9c700 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824d400 0x55d788267e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.089+0000 7fad31f9c700 1 -- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824dd20 msgr2=0x55d788266150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.089+0000 7fad31f9c700 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824dd20 0x55d788266150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.089+0000 7fad31f9c700 1 -- 172.21.15.5:0/1582670382 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad240125e0 con 0x55d7882613e0 2023-09-23T04:51:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.090+0000 7fad31f9c700 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad24000f80 tx=0x7fad240089d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a1fc3c5ce7eddb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.090+0000 7fad14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.090+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400e070 con 0x55d7882613e0 2023-09-23T04:51:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.090+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad28016330 con 0x55d7882613e0 2023-09-23T04:51:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.090+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fad240092e0 con 0x55d7882613e0 2023-09-23T04:51:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad24017260 con 0x55d7882613e0 2023-09-23T04:51:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 10 client.26893 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d787ff2250 con 0x55d7882613e0 2023-09-23T04:51:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad28016330 con 0x55d7882613e0 2023-09-23T04:51:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fad240177f0 con 0x55d7882613e0 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 10 client.26893 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 10 client.26893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 20 client.26893 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d7882c5dd0 0x55d7882c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 4 client.26893 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:51:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.091+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d788199b30 con 0x55d7882c5dd0 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.092+0000 7fad3179b700 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d7882c5dd0 0x55d7882c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad3179b700 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d7882c5dd0 0x55d7882c81f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88c25168f80067e server_cookie=2ccbc8ee3aba900b in_seq=0 out_seq=0 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad14ff9700 10 client.26893 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad14ff9700 1 -- 172.21.15.5:0/1582670382 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d788199b30 con 0x55d7882c5dd0 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad14ff9700 10 client.26893 handle_command_reply: tid=0 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 client.26893 shutdown 2023-09-23T04:51:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 msgr2=0x55d788266690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad24000f80 tx=0x7fad240089d0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 shutdown_connections 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d7882c5dd0 0x55d7882c81f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d7882613e0 0x55d788266690 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d78824d400 0x55d788267e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad14ff9700 0 client.26893 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 --2- 172.21.15.5:0/1582670382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d78824dd20 0x55d788266150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 >> 172.21.15.5:0/1582670382 conn(0x55d7881c2260 msgr2=0x55d78825d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 shutdown_connections 2023-09-23T04:51:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.093+0000 7fad472fab80 1 -- 172.21.15.5:0/1582670382 wait complete. 2023-09-23T04:51:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad472fab80 20 client.26893 trim_cache size 0 max 16384 2023-09-23T04:51:10.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:51:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:51:10.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 msgr2=0x7fad180a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 0x7fad180a3c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad200098e0 tx=0x7fad2000b690 comp rx=0 tx=0).stop 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 msgr2=0x7fad2c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad2800eb30 tx=0x7fad2800ee40 comp rx=0 tx=0).stop 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 shutdown_connections 2023-09-23T04:51:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fad180a17d0 0x7fad180a3c90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad2c10b490 0x7fad2c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad2c1020c0 0x7fad2c199240 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 --2- 172.21.15.5:0/1585264399 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fad2c1029e0 0x7fad2c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.094+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 >> 172.21.15.5:0/1585264399 conn(0x7fad2c076f30 msgr2=0x7fad2c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.095+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 shutdown_connections 2023-09-23T04:51:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.095+0000 7fad339ff700 1 -- 172.21.15.5:0/1585264399 wait complete. 2023-09-23T04:51:10.112 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:51:10.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:51:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 -- 172.21.15.5:0/1740543992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26741010c0 msgr2=0x7f26741014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 --2- 172.21.15.5:0/1740543992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26741010c0 0x7f26741014a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2670009990 tx=0x7f2670009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 -- 172.21.15.5:0/1740543992 shutdown_connections 2023-09-23T04:51:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 --2- 172.21.15.5:0/1740543992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 0x7f267410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 --2- 172.21.15.5:0/1740543992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26741010c0 0x7f26741014a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 --2- 172.21.15.5:0/1740543992 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2674101a70 0x7f2674105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 -- 172.21.15.5:0/1740543992 >> 172.21.15.5:0/1740543992 conn(0x7f26740fc960 msgr2=0x7f26740fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 -- 172.21.15.5:0/1740543992 shutdown_connections 2023-09-23T04:51:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.326+0000 7f267b3ff700 1 -- 172.21.15.5:0/1740543992 wait complete. 2023-09-23T04:51:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.327+0000 7f267b3ff700 1 Processor -- start 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.327+0000 7f267b3ff700 1 -- start start 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 0x7f2674199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 0x7f267419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267419e640 con 0x7f267419dbc0 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267419e7b0 con 0x7f26741061e0 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267b3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267419e970 con 0x7f26741010c0 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 0x7f267419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267919b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267919b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42990/0 (socket says 172.21.15.5:42990) 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267919b700 1 -- 172.21.15.5:0/2869376187 learned_addr learned my addr 172.21.15.5:0/2869376187 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267899a700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 0x7f2674199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 -- 172.21.15.5:0/2869376187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 msgr2=0x7f2674199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 -- 172.21.15.5:0/2869376187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 msgr2=0x7f2674199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 0x7f2674199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267999c700 1 -- 172.21.15.5:0/2869376187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700095f0 con 0x7f267419dbc0 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.328+0000 7f267919b700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f267999c700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 0x7f267419e040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f266c007f10 tx=0x7f266c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c004db0 con 0x7f267419dbc0 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266c00f040 con 0x7f267419dbc0 2023-09-23T04:51:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26741a5ed0 con 0x7f267419dbc0 2023-09-23T04:51:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c005990 con 0x7f267419dbc0 2023-09-23T04:51:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.329+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26741a6360 con 0x7f267419dbc0 2023-09-23T04:51:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.330+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2674067ca0 con 0x7f267419dbc0 2023-09-23T04:51:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.332+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f266c00db20 con 0x7f267419dbc0 2023-09-23T04:51:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.333+0000 7f266a7fc700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 0x7f26600a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.334+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f266c0ccf70 con 0x7f267419dbc0 2023-09-23T04:51:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.334+0000 7f267919b700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 0x7f26600a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.334+0000 7f267919b700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 0x7f26600a3bd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2670006010 tx=0x7f267000b540 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.338+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f266c04eec0 con 0x7f267419dbc0 2023-09-23T04:51:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.595+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f267419a570 con 0x7f267419dbc0 2023-09-23T04:51:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.596+0000 7f266a7fc700 1 -- 172.21.15.5:0/2869376187 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11354 (secure 0 0 0) 0x7f266c095de0 con 0x7f267419dbc0 2023-09-23T04:51:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:51:10.736 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:50:07.224731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 msgr2=0x7f26600a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 0x7f26600a3bd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2670006010 tx=0x7f267000b540 comp rx=0 tx=0).stop 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 msgr2=0x7f267419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 0x7f267419e040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f266c007f10 tx=0x7f266c00d3b0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 shutdown_connections 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f26600a1710 0x7f26600a3bd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f267419dbc0 0x7f267419e040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f26741010c0 0x7f2674199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 --2- 172.21.15.5:0/2869376187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26741061e0 0x7f2674199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 >> 172.21.15.5:0/2869376187 conn(0x7f26740fc960 msgr2=0x7f26740fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 shutdown_connections 2023-09-23T04:51:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.598+0000 7f267b3ff700 1 -- 172.21.15.5:0/2869376187 wait complete. 2023-09-23T04:51:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2023-09-23T04:51:10.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:51:10.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.45 MB/sec execute 106 sec latency 3100.344 ms 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.948+0000 7f17af7ff700 1 -- 172.21.15.5:0/3358493314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a8101a80 msgr2=0x7f17a8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.948+0000 7f17af7ff700 1 --2- 172.21.15.5:0/3358493314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a8101a80 0x7f17a8105ac0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1798009990 tx=0x7f1798009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 -- 172.21.15.5:0/3358493314 shutdown_connections 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 --2- 172.21.15.5:0/3358493314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a81061f0 0x7f17a810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 --2- 172.21.15.5:0/3358493314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81010d0 0x7f17a81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 --2- 172.21.15.5:0/3358493314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a8101a80 0x7f17a8105ac0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 -- 172.21.15.5:0/3358493314 >> 172.21.15.5:0/3358493314 conn(0x7f17a80fc990 msgr2=0x7f17a80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 -- 172.21.15.5:0/3358493314 shutdown_connections 2023-09-23T04:51:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.949+0000 7f17af7ff700 1 -- 172.21.15.5:0/3358493314 wait complete. 2023-09-23T04:51:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 Processor -- start 2023-09-23T04:51:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 -- start start 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81061f0 0x7f17a8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a819b970 0x7f17a819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a819c3f0 con 0x7f17a81010d0 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a819c560 con 0x7f17a81061f0 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17af7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a819c720 con 0x7f17a819b970 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.950+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51174/0 (socket says 172.21.15.5:51174) 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 -- 172.21.15.5:0/2894848624 learned_addr learned my addr 172.21.15.5:0/2894848624 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17add9c700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a819b970 0x7f17a819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17acd9a700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81061f0 0x7f17a8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 -- 172.21.15.5:0/2894848624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a819b970 msgr2=0x7f17a819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a819b970 0x7f17a819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 -- 172.21.15.5:0/2894848624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81061f0 msgr2=0x7f17a8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81061f0 0x7f17a8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 -- 172.21.15.5:0/2894848624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980095f0 con 0x7f17a81010d0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9a0 tx=0x7f17a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f179e7fc700 1 -- 172.21.15.5:0/2894848624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4009790 con 0x7f17a81010d0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f179e7fc700 1 -- 172.21.15.5:0/2894848624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a4010430 con 0x7f17a81010d0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a81a3c80 con 0x7f17a81010d0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.951+0000 7f179e7fc700 1 -- 172.21.15.5:0/2894848624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4018680 con 0x7f17a81010d0 2023-09-23T04:51:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.952+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81a4160 con 0x7f17a81010d0 2023-09-23T04:51:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.955+0000 7f179e7fc700 1 -- 172.21.15.5:0/2894848624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f17a40105a0 con 0x7f17a81010d0 2023-09-23T04:51:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.956+0000 7f17c31a2b80 1 Processor -- start 2023-09-23T04:51:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.956+0000 7f179e7fc700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 0x7f17940a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.956+0000 7f17c31a2b80 1 -- start start 2023-09-23T04:51:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.956+0000 7f179e7fc700 1 -- 172.21.15.5:0/2894848624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f17a4014070 con 0x7f17a81010d0 2023-09-23T04:51:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17add9c700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 0x7f17940a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec5c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aebba460 con 0x5577aec51400 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aebabc20 con 0x5577aec5cbd0 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aeba9540 con 0x5577aec51d20 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51186/0 (socket says 172.21.15.5:51186) 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 -- 172.21.15.5:0/2217522085 learned_addr learned my addr 172.21.15.5:0/2217522085 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17acd9a700 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec5c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17add9c700 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 -- 172.21.15.5:0/2217522085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 msgr2=0x5577aec5c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec5c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 -- 172.21.15.5:0/2217522085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 msgr2=0x5577aec5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17add9c700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 0x7f17940a3bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17a000f540 tx=0x7f17a0009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.957+0000 7f17ad59b700 1 -- 172.21.15.5:0/2217522085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577aec078d0 con 0x5577aec51400 2023-09-23T04:51:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.958+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f17a4096910 tx=0x7f17a4099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8316c06d890dfa29 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.958+0000 7f17937fe700 1 -- 172.21.15.5:0/2217522085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a409e560 con 0x5577aec51400 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.958+0000 7f17937fe700 1 -- 172.21.15.5:0/2217522085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a40b1070 con 0x5577aec51400 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.958+0000 7f17937fe700 1 -- 172.21.15.5:0/2217522085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a409a870 con 0x5577aec51400 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2217522085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 msgr2=0x5577aec517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f17a4096910 tx=0x7f17a4099d70 comp rx=0 tx=0).stop 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2217522085 shutdown_connections 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec517e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2217522085 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec5c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2217522085 >> 172.21.15.5:0/2217522085 conn(0x5577aebc6260 msgr2=0x5577aec4d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2217522085 shutdown_connections 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2217522085 wait complete. 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.959+0000 7f17c31a2b80 1 -- start start 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 Processor -- start 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 0x5577aec59e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec5cbd0 0x5577aec5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aeba4e30 con 0x5577aec51400 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aeb41710 con 0x5577aec51d20 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aeb413b0 con 0x5577aec5cbd0 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17c31a2b80 10 client.0 _ll_register_callbacks cb 0x5577ae99e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17acd9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 0x5577aec59e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17ad59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51192/0 (socket says 172.21.15.5:51192) 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.960+0000 7f17ad59b700 1 -- 172.21.15.5:0/1669340948 learned_addr learned my addr 172.21.15.5:0/1669340948 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17add9c700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec5cbd0 0x5577aec5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 -- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec5cbd0 msgr2=0x5577aec5a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec5cbd0 0x5577aec5a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 -- 172.21.15.5:0/1669340948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 msgr2=0x5577aec598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 -- 172.21.15.5:0/1669340948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81a4160 con 0x5577aec51d20 2023-09-23T04:51:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17ad59b700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17acd9a700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 0x5577aec59e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1798000c00 tx=0x7f1798009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e64c6497ab3458 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f1790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179801d030 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.961+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577aec45250 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1798004500 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17980046c0 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 10 client.26855 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5577ae9cfde0 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577aec45250 con 0x5577aec51d20 2023-09-23T04:51:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f179800f7f0 con 0x5577aec51d20 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 10 client.26855 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 20 client.26855 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.962+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aec70a10 0x5577aec72e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.963+0000 7f17c31a2b80 4 client.26855 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.963+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577aeb9de10 con 0x5577aec70a10 2023-09-23T04:51:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.963+0000 7f17ad59b700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aec70a10 0x5577aec72e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.963+0000 7f17ad59b700 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aec70a10 0x5577aec72e30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d169c3542789d993 server_cookie=d805dc91f193e647 in_seq=0 out_seq=0 2023-09-23T04:51:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.963+0000 7f1790ff9700 10 client.26855 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.964+0000 7f1790ff9700 1 -- 172.21.15.5:0/1669340948 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577aeb9de10 con 0x5577aec70a10 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.964+0000 7f1790ff9700 10 client.26855 handle_command_reply: tid=0 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.964+0000 7f17c31a2b80 1 client.26855 shutdown 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.964+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 msgr2=0x5577aec59e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.964+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 0x5577aec59e30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1798000c00 tx=0x7f1798009570 comp rx=0 tx=0).stop 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 shutdown_connections 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec5cbd0 0x5577aec5a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aec70a10 0x5577aec72e30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51400 0x5577aec598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/1669340948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51d20 0x5577aec59e30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f1790ff9700 0 client.26855 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 >> 172.21.15.5:0/1669340948 conn(0x5577aebc6260 msgr2=0x5577aec4d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 shutdown_connections 2023-09-23T04:51:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 1 -- 172.21.15.5:0/1669340948 wait complete. 2023-09-23T04:51:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.965+0000 7f17c31a2b80 20 client.26855 trim_cache size 0 max 16384 2023-09-23T04:51:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.978+0000 7f17c31a2b80 1 Processor -- start 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 -- start start 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51400 0x5577aec54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aea25920 con 0x5577aec5cbd0 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aec5c9a0 con 0x5577aec51400 2023-09-23T04:51:10.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aec47860 con 0x5577aec51d20 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17add9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17add9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51200/0 (socket says 172.21.15.5:51200) 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17add9c700 1 -- 172.21.15.5:0/2911325470 learned_addr learned my addr 172.21.15.5:0/2911325470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17ad59b700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51400 0x5577aec54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.979+0000 7f17acd9a700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 -- 172.21.15.5:0/2911325470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 msgr2=0x5577aec59fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec59fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 -- 172.21.15.5:0/2911325470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51400 msgr2=0x5577aec54370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51400 0x5577aec54370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 -- 172.21.15.5:0/2911325470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577aec49480 con 0x5577aec5cbd0 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17acd9a700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f17add9c700 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17a0019980 tx=0x7f17a001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830a4345a166548c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f1793fff700 1 -- 172.21.15.5:0/2911325470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000e040 con 0x5577aec5cbd0 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f1793fff700 1 -- 172.21.15.5:0/2911325470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a000f040 con 0x5577aec5cbd0 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.980+0000 7f1793fff700 1 -- 172.21.15.5:0/2911325470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004280 con 0x5577aec5cbd0 2023-09-23T04:51:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2911325470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 msgr2=0x5577aec56bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17a0019980 tx=0x7f17a001c000 comp rx=0 tx=0).stop 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2911325470 shutdown_connections 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec5cbd0 0x5577aec56bc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec51400 0x5577aec54370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/2911325470 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51d20 0x5577aec59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2911325470 >> 172.21.15.5:0/2911325470 conn(0x5577aebc6260 msgr2=0x5577aec598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2911325470 shutdown_connections 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- 172.21.15.5:0/2911325470 wait complete. 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 -- start start 2023-09-23T04:51:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.981+0000 7f17c31a2b80 1 Processor -- start 2023-09-23T04:51:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51400 0x5577aecbda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51d20 0x5577aecbdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aec66050 con 0x5577aec51d20 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aec661c0 con 0x5577aec5cbd0 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577aec66380 con 0x5577aec51400 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17c31a2b80 10 client.0 _ll_register_callbacks cb 0x5577ae99e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17add9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17acd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51d20 0x5577aecbdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17add9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56272/0 (socket says 172.21.15.5:56272) 2023-09-23T04:51:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17add9c700 1 -- 172.21.15.5:0/516093173 learned_addr learned my addr 172.21.15.5:0/516093173 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.982+0000 7f17ad59b700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51400 0x5577aecbda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 -- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51400 msgr2=0x5577aecbda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51400 0x5577aecbda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 -- 172.21.15.5:0/516093173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51d20 msgr2=0x5577aecbdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51d20 0x5577aecbdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 -- 172.21.15.5:0/516093173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a0019600 con 0x5577aec5cbd0 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17add9c700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003cb0 tx=0x7f17a0003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b7fe2ef644e5fd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f1790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a000e030 con 0x5577aec5cbd0 2023-09-23T04:51:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17a40b1070 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f17a0019e40 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a0009030 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17c31a2b80 10 client.26861 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.983+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5577ae9b8c60 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a40b1070 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f17a0029030 con 0x5577aec5cbd0 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 10 client.26861 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 10 client.26861 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 20 client.26861 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aecc58e0 0x5577aecc7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 4 client.26861 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577aeb9db30 con 0x5577aecc58e0 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.984+0000 7f17ad59b700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aecc58e0 0x5577aecc7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f1790ff9700 10 client.26861 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17ad59b700 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aecc58e0 0x5577aecc7d00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db4762a06a1e154f server_cookie=8058d1848ce5acff in_seq=0 out_seq=0 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f1790ff9700 1 -- 172.21.15.5:0/516093173 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577aeb9db30 con 0x5577aecc58e0 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f1790ff9700 10 client.26861 handle_command_reply: tid=0 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 client.26861 shutdown 2023-09-23T04:51:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 msgr2=0x5577aec65b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003cb0 tx=0x7f17a0003ce0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 shutdown_connections 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577aec5cbd0 0x5577aec65b10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5577aec51400 0x5577aecbda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5577aecc58e0 0x5577aecc7d00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 --2- 172.21.15.5:0/516093173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577aec51d20 0x5577aecbdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f1790ff9700 0 client.26861 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 >> 172.21.15.5:0/516093173 conn(0x5577aebc6260 msgr2=0x5577aec58720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 shutdown_connections 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.985+0000 7f17c31a2b80 1 -- 172.21.15.5:0/516093173 wait complete. 2023-09-23T04:51:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17c31a2b80 20 client.26861 trim_cache size 0 max 16384 2023-09-23T04:51:10.992 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 msgr2=0x7f17940a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 0x7f17940a3bd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17a000f540 tx=0x7f17a0009920 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 msgr2=0x7f17a8196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17a400e9a0 tx=0x7f17a400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 shutdown_connections 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f17940a1710 0x7f17940a3bd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f17a819b970 0x7f17a819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81010d0 0x7f17a8196fd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 --2- 172.21.15.5:0/2894848624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17a81061f0 0x7f17a8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 >> 172.21.15.5:0/2894848624 conn(0x7f17a80fc990 msgr2=0x7f17a80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 shutdown_connections 2023-09-23T04:51:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:10.986+0000 7f17af7ff700 1 -- 172.21.15.5:0/2894848624 wait complete. 2023-09-23T04:51:11.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:51:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 -- 172.21.15.5:0/104851738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c1010d0 msgr2=0x7f187c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 --2- 172.21.15.5:0/104851738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c1010d0 0x7f187c1014b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f18780098f0 tx=0x7f1878009c00 comp rx=0 tx=0).stop 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 -- 172.21.15.5:0/104851738 shutdown_connections 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 --2- 172.21.15.5:0/104851738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 --2- 172.21.15.5:0/104851738 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c1010d0 0x7f187c1014b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 --2- 172.21.15.5:0/104851738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c101a80 0x7f187c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.211+0000 7f18837ff700 1 -- 172.21.15.5:0/104851738 >> 172.21.15.5:0/104851738 conn(0x7f187c0fc990 msgr2=0x7f187c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.212+0000 7f18837ff700 1 -- 172.21.15.5:0/104851738 shutdown_connections 2023-09-23T04:51:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.212+0000 7f18837ff700 1 -- 172.21.15.5:0/104851738 wait complete. 2023-09-23T04:51:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 Processor -- start 2023-09-23T04:51:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 -- start start 2023-09-23T04:51:11.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 0x7f187c19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c19e620 con 0x7f187c1010d0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c19e790 con 0x7f187c1061f0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f18837ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c19e950 con 0x7f187c19dba0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f1881d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 0x7f187c19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51226/0 (socket says 172.21.15.5:51226) 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f188159b700 1 -- 172.21.15.5:0/3213006199 learned_addr learned my addr 172.21.15.5:0/3213006199 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.213+0000 7f1880d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 -- 172.21.15.5:0/3213006199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 msgr2=0x7f187c19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 0x7f187c19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 -- 172.21.15.5:0/3213006199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 msgr2=0x7f187c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 -- 172.21.15.5:0/3213006199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878009520 con 0x7f187c1010d0 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f1881d9c700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 0x7f187c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f1880d9a700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f188159b700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1878009810 tx=0x7f187800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f18727fc700 1 -- 172.21.15.5:0/3213006199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187801c070 con 0x7f187c1010d0 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c1a5e50 con 0x7f187c1010d0 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f18727fc700 1 -- 172.21.15.5:0/3213006199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187800ba40 con 0x7f187c1010d0 2023-09-23T04:51:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.214+0000 7f18727fc700 1 -- 172.21.15.5:0/3213006199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878004930 con 0x7f187c1010d0 2023-09-23T04:51:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.215+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c1a6270 con 0x7f187c1010d0 2023-09-23T04:51:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.218+0000 7f18727fc700 1 -- 172.21.15.5:0/3213006199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f1878020070 con 0x7f187c1010d0 2023-09-23T04:51:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.220+0000 7f189710fb80 1 Processor -- start 2023-09-23T04:51:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.220+0000 7f189710fb80 1 -- start start 2023-09-23T04:51:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f18727fc700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 0x7f18680a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x55587360a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x55587360cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873568460 con 0x55587360abd0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873559c20 con 0x5558735ff400 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f189710fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873557540 con 0x5558735ffd20 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f18727fc700 1 -- 172.21.15.5:0/3213006199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f18780ce350 con 0x7f187c1010d0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f1881d9c700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 0x7f18680a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f188159b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f1881d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x55587360cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f188159b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56304/0 (socket says 172.21.15.5:56304) 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f188159b700 1 -- 172.21.15.5:0/67323262 learned_addr learned my addr 172.21.15.5:0/67323262 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f1880d9a700 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x55587360a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.221+0000 7f188159b700 1 -- 172.21.15.5:0/67323262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 msgr2=0x55587360a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f188159b700 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x55587360a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f188159b700 1 -- 172.21.15.5:0/67323262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 msgr2=0x55587360cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f188159b700 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x55587360cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f188159b700 1 -- 172.21.15.5:0/67323262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558735b58d0 con 0x5558735ff400 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f1881d9c700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 0x7f18680a3c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1874009fd0 tx=0x7f1874009310 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f188159b700 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1878097470 tx=0x7f187809ab50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ecc75a0274c9a25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f18677fe700 1 -- 172.21.15.5:0/67323262 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878063940 con 0x5558735ff400 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f18677fe700 1 -- 172.21.15.5:0/67323262 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187806dae0 con 0x5558735ff400 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.222+0000 7f18677fe700 1 -- 172.21.15.5:0/67323262 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187809f0c0 con 0x5558735ff400 2023-09-23T04:51:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 -- 172.21.15.5:0/67323262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 msgr2=0x5558735ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1878097470 tx=0x7f187809ab50 comp rx=0 tx=0).stop 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 -- 172.21.15.5:0/67323262 shutdown_connections 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x55587360cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x5558735ff7e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 --2- 172.21.15.5:0/67323262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x55587360a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 -- 172.21.15.5:0/67323262 >> 172.21.15.5:0/67323262 conn(0x555873574260 msgr2=0x5558735fb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 -- 172.21.15.5:0/67323262 shutdown_connections 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.223+0000 7f189710fb80 1 -- 172.21.15.5:0/67323262 wait complete. 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.224+0000 7f189710fb80 1 -- start start 2023-09-23T04:51:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.224+0000 7f189710fb80 1 Processor -- start 2023-09-23T04:51:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x555873616030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873552e30 con 0x5558735ff400 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558734ef710 con 0x5558735ffd20 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558734ef3b0 con 0x55587360abd0 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f189710fb80 10 client.0 _ll_register_callbacks cb 0x55587334c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51244/0 (socket says 172.21.15.5:51244) 2023-09-23T04:51:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f188159b700 1 -- 172.21.15.5:0/3309484276 learned_addr learned my addr 172.21.15.5:0/3309484276 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f1880d9a700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.225+0000 7f1881d9c700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x555873616030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 -- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 msgr2=0x555873616030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x555873616030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 -- 172.21.15.5:0/3309484276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 msgr2=0x55587360b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 -- 172.21.15.5:0/3309484276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c1a6270 con 0x5558735ff400 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1880d9a700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f188159b700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f187809ac70 tx=0x7f1878097330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf9f0a3dc64cff0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1864ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187809b670 con 0x5558735ff400 2023-09-23T04:51:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f18780a3270 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558735f3250 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f187809f0c0 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f189710fb80 10 client.16872 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55587337dde0 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.226+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18780b7080 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f18780cb940 con 0x5558735ff400 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 10 client.16872 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 10 client.16872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 20 client.16872 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x555873619f80 0x55587361c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 4 client.16872 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55587354be10 con 0x555873619f80 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.227+0000 7f1880d9a700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x555873619f80 0x55587361c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.228+0000 7f1864ff9700 10 client.16872 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.228+0000 7f1880d9a700 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x555873619f80 0x55587361c3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e70fdea8e98a9700 server_cookie=b1ee66bd6d7f6efb in_seq=0 out_seq=0 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.228+0000 7f1864ff9700 1 -- 172.21.15.5:0/3309484276 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55587354be10 con 0x555873619f80 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.228+0000 7f1864ff9700 10 client.16872 handle_command_reply: tid=0 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.228+0000 7f189710fb80 1 client.16872 shutdown 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 msgr2=0x55587360afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f187809ac70 tx=0x7f1878097330 comp rx=0 tx=0).stop 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 shutdown_connections 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x555873616030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x555873619f80 0x55587361c3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360afb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 --2- 172.21.15.5:0/3309484276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f1864ff9700 0 client.16872 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 >> 172.21.15.5:0/3309484276 conn(0x555873574260 msgr2=0x5558735fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 shutdown_connections 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 1 -- 172.21.15.5:0/3309484276 wait complete. 2023-09-23T04:51:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.229+0000 7f189710fb80 20 client.16872 trim_cache size 0 max 16384 2023-09-23T04:51:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 Processor -- start 2023-09-23T04:51:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 -- start start 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x555873605050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x555873605590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558733d3920 con 0x55587360abd0 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873604b90 con 0x5558735ff400 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f189710fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555873600100 con 0x5558735ffd20 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f1881d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f1881d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51256/0 (socket says 172.21.15.5:51256) 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f1881d9c700 1 -- 172.21.15.5:0/2909234174 learned_addr learned my addr 172.21.15.5:0/2909234174 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.243+0000 7f188159b700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x555873605050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1880d9a700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x555873605590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 -- 172.21.15.5:0/2909234174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 msgr2=0x555873605590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x555873605590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 -- 172.21.15.5:0/2909234174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 msgr2=0x555873605050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x555873605050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 -- 172.21.15.5:0/2909234174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555873366c60 con 0x55587360abd0 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f188159b700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x555873605050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1881d9c700 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1874009750 tx=0x7f187401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106fa0435f473086 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1867fff700 1 -- 172.21.15.5:0/2909234174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187402a070 con 0x55587360abd0 2023-09-23T04:51:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1867fff700 1 -- 172.21.15.5:0/2909234174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187400f040 con 0x55587360abd0 2023-09-23T04:51:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.244+0000 7f1867fff700 1 -- 172.21.15.5:0/2909234174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874004e30 con 0x55587360abd0 2023-09-23T04:51:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 -- 172.21.15.5:0/2909234174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 msgr2=0x555873615700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1874009750 tx=0x7f187401d000 comp rx=0 tx=0).stop 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 -- 172.21.15.5:0/2909234174 shutdown_connections 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55587360abd0 0x555873615700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ff400 0x555873605050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 --2- 172.21.15.5:0/2909234174 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558735ffd20 0x555873605590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 -- 172.21.15.5:0/2909234174 >> 172.21.15.5:0/2909234174 conn(0x555873574260 msgr2=0x5558736165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 -- 172.21.15.5:0/2909234174 shutdown_connections 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.245+0000 7f189710fb80 1 -- 172.21.15.5:0/2909234174 wait complete. 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 -- start start 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 Processor -- start 2023-09-23T04:51:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x55587366cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587360fe60 con 0x5558735ff400 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587360ffd0 con 0x5558735ffd20 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55587366d320 con 0x55587360abd0 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f189710fb80 10 client.0 _ll_register_callbacks cb 0x7f1874008770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.246+0000 7f188159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51268/0 (socket says 172.21.15.5:51268) 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1880d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 -- 172.21.15.5:0/1158313278 learned_addr learned my addr 172.21.15.5:0/1158313278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1881d9c700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x55587366cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 -- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 msgr2=0x55587366cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x55587366cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 -- 172.21.15.5:0/1158313278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 msgr2=0x55587360f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 -- 172.21.15.5:0/1158313278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55587337dde0 con 0x5558735ff400 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1881d9c700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x55587366cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1880d9a700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f188159b700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1878011040 tx=0x7f18780054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b08a3351c2b173 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1864ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18780a3270 con 0x5558735ff400 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f187809f0c0 con 0x5558735ff400 2023-09-23T04:51:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.247+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f187806dae0 con 0x5558735ff400 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18780b9020 con 0x5558735ff400 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 10 client.16878 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f1874020040 con 0x5558735ff400 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187806c030 con 0x5558735ff400 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f187809a560 con 0x5558735ff400 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 10 client.16878 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 10 client.16878 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 20 client.16878 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55587366e620 0x555873670a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 4 client.16878 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55587354bb30 con 0x55587366e620 2023-09-23T04:51:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.248+0000 7f1880d9a700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55587366e620 0x555873670a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f1880d9a700 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55587366e620 0x555873670a40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfbfb1050501cdf8 server_cookie=8ae3bc302f234fe2 in_seq=0 out_seq=0 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f1864ff9700 10 client.16878 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f1864ff9700 1 -- 172.21.15.5:0/1158313278 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55587354bb30 con 0x55587366e620 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f1864ff9700 10 client.16878 handle_command_reply: tid=0 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 client.16878 shutdown 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 msgr2=0x55587360f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1878011040 tx=0x7f18780054c0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 shutdown_connections 2023-09-23T04:51:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55587360abd0 0x55587366cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55587366e620 0x555873670a40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558735ff400 0x55587360f150 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 --2- 172.21.15.5:0/1158313278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558735ffd20 0x55587360f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 >> 172.21.15.5:0/1158313278 conn(0x555873574260 msgr2=0x555873605050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.249+0000 7f1864ff9700 0 client.16878 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 shutdown_connections 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f189710fb80 1 -- 172.21.15.5:0/1158313278 wait complete. 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f189710fb80 20 client.16878 trim_cache size 0 max 16384 2023-09-23T04:51:11.255 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 msgr2=0x7f18680a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 0x7f18680a3c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1874009fd0 tx=0x7f1874009310 comp rx=0 tx=0).stop 2023-09-23T04:51:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 msgr2=0x7f187c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1878009810 tx=0x7f187800fa00 comp rx=0 tx=0).stop 2023-09-23T04:51:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 shutdown_connections 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f18680a1790 0x7f18680a3c50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f187c19dba0 0x7f187c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c1010d0 0x7f187c199200 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 --2- 172.21.15.5:0/3213006199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f187c1061f0 0x7f187c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.250+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 >> 172.21.15.5:0/3213006199 conn(0x7f187c0fc990 msgr2=0x7f187c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.251+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 shutdown_connections 2023-09-23T04:51:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.251+0000 7f18837ff700 1 -- 172.21.15.5:0/3213006199 wait complete. 2023-09-23T04:51:11.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:51:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 -- 172.21.15.5:0/2725433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba81061f0 msgr2=0x7faba810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2725433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba81061f0 0x7faba810dd70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faba000b0a0 tx=0x7faba000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 -- 172.21.15.5:0/2725433252 shutdown_connections 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2725433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba81061f0 0x7faba810dd70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2725433252 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81010d0 0x7faba81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2725433252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba8101a80 0x7faba8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 -- 172.21.15.5:0/2725433252 >> 172.21.15.5:0/2725433252 conn(0x7faba80fc990 msgr2=0x7faba80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 -- 172.21.15.5:0/2725433252 shutdown_connections 2023-09-23T04:51:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.474+0000 7fabad7ff700 1 -- 172.21.15.5:0/2725433252 wait complete. 2023-09-23T04:51:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.476+0000 7fabad7ff700 1 Processor -- start 2023-09-23T04:51:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.476+0000 7fabad7ff700 1 -- start start 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba81010d0 0x7faba8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81061f0 0x7faba8073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a5920 con 0x7faba8073ba0 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a5a90 con 0x7faba81010d0 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7fabad7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81a5c50 con 0x7faba81061f0 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51282/0 (socket says 172.21.15.5:51282) 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 -- 172.21.15.5:0/2072075314 learned_addr learned my addr 172.21.15.5:0/2072075314 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba81010d0 0x7faba8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba67fc700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81061f0 0x7faba8073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 -- 172.21.15.5:0/2072075314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81061f0 msgr2=0x7faba8073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81061f0 0x7faba8073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 -- 172.21.15.5:0/2072075314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba81010d0 msgr2=0x7faba8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba81010d0 0x7faba8073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.477+0000 7faba77fe700 1 -- 172.21.15.5:0/2072075314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0009d00 con 0x7faba8073ba0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7faba77fe700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faba000b070 tx=0x7faba000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7fab8ffff700 1 -- 172.21.15.5:0/2072075314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba000e040 con 0x7faba8073ba0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7fab8ffff700 1 -- 172.21.15.5:0/2072075314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba0009480 con 0x7faba8073ba0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba81a5ed0 con 0x7faba8073ba0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7fab8ffff700 1 -- 172.21.15.5:0/2072075314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba00052a0 con 0x7faba8073ba0 2023-09-23T04:51:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.478+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba81a6350 con 0x7faba8073ba0 2023-09-23T04:51:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.481+0000 7fab8ffff700 1 -- 172.21.15.5:0/2072075314 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7faba001d480 con 0x7faba8073ba0 2023-09-23T04:51:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.482+0000 7fabc1155b80 1 Processor -- start 2023-09-23T04:51:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 -- start start 2023-09-23T04:51:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fab8ffff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 0x7fab900a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fab8ffff700 1 -- 172.21.15.5:0/2072075314 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7faba00ce690 con 0x7faba8073ba0 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed519460 con 0x55e3ed5b0400 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed50ac20 con 0x55e3ed5b0d20 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7fabc1155b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed508540 con 0x55e3ed5bbbd0 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7faba67fc700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 0x7fab900a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51288/0 (socket says 172.21.15.5:51288) 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7faba6ffd700 1 -- 172.21.15.5:0/1290454806 learned_addr learned my addr 172.21.15.5:0/1290454806 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.483+0000 7faba67fc700 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba77fe700 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 -- 172.21.15.5:0/1290454806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 msgr2=0x55e3ed5bdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 -- 172.21.15.5:0/1290454806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 msgr2=0x55e3ed5bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 -- 172.21.15.5:0/1290454806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3ed5668d0 con 0x55e3ed5b0400 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab98009d80 tx=0x7fab98011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9d445b6971eb29 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7faba67fc700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 0x7fab900a3c50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faba8074910 tx=0x7fab9c009aa0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7fab8d7fa700 1 -- 172.21.15.5:0/1290454806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98004230 con 0x55e3ed5b0400 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7fab8d7fa700 1 -- 172.21.15.5:0/1290454806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab98004390 con 0x55e3ed5b0400 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.484+0000 7fab8d7fa700 1 -- 172.21.15.5:0/1290454806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980197a0 con 0x55e3ed5b0400 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 -- 172.21.15.5:0/1290454806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 msgr2=0x55e3ed5b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab98009d80 tx=0x7fab98011720 comp rx=0 tx=0).stop 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 -- 172.21.15.5:0/1290454806 shutdown_connections 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b07e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1290454806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 -- 172.21.15.5:0/1290454806 >> 172.21.15.5:0/1290454806 conn(0x55e3ed525260 msgr2=0x55e3ed5ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 -- 172.21.15.5:0/1290454806 shutdown_connections 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.485+0000 7fabc1155b80 1 -- 172.21.15.5:0/1290454806 wait complete. 2023-09-23T04:51:11.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.486+0000 7fabc1155b80 1 -- start start 2023-09-23T04:51:11.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.486+0000 7fabc1155b80 1 Processor -- start 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed503e30 con 0x55e3ed5b0d20 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed4a0710 con 0x55e3ed5b0400 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed4a03b0 con 0x55e3ed5bbbd0 2023-09-23T04:51:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7fabc1155b80 10 client.0 _ll_register_callbacks cb 0x55e3ed2fd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51294/0 (socket says 172.21.15.5:51294) 2023-09-23T04:51:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 -- 172.21.15.5:0/1905429147 learned_addr learned my addr 172.21.15.5:0/1905429147 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba77fe700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 -- 172.21.15.5:0/1905429147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 msgr2=0x55e3ed5b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 -- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0400 msgr2=0x55e3ed5b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.487+0000 7faba67fc700 1 -- 172.21.15.5:0/1905429147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab980099e0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7faba67fc700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab9c010880 tx=0x7fab9c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614a9b08cce7d5ff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fab8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c01e6f0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3ed5a4250 con 0x55e3ed5b0d20 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fab9c01e850 con 0x55e3ed5b0d20 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9c01ea10 con 0x55e3ed5b0d20 2023-09-23T04:51:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fabc1155b80 10 client.16890 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.488+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e3ed32ede0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3ed5a4250 con 0x55e3ed5b0d20 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fab9c028030 con 0x55e3ed5b0d20 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 10 client.16890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 20 client.16890 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed5cb750 0x55e3ed5cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 4 client.16890 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3ed4fce10 con 0x55e3ed5cb750 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.489+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed5cb750 0x55e3ed5cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.490+0000 7faba6ffd700 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed5cb750 0x55e3ed5cdb70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec5299f294ab5c3 server_cookie=93f862d9e0648457 in_seq=0 out_seq=0 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.490+0000 7fab8aff5700 10 client.16890 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fab8aff5700 1 -- 172.21.15.5:0/1905429147 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3ed4fce10 con 0x55e3ed5cb750 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fab8aff5700 10 client.16890 handle_command_reply: tid=0 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 client.16890 shutdown 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 msgr2=0x55e3ed5b8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab9c010880 tx=0x7fab9c01c000 comp rx=0 tx=0).stop 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 shutdown_connections 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed5cb750 0x55e3ed5cdb70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0400 0x55e3ed5b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 --2- 172.21.15.5:0/1905429147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b8e70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fab8aff5700 0 client.16890 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 >> 172.21.15.5:0/1905429147 conn(0x55e3ed525260 msgr2=0x55e3ed5ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 shutdown_connections 2023-09-23T04:51:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 1 -- 172.21.15.5:0/1905429147 wait complete. 2023-09-23T04:51:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.491+0000 7fabc1155b80 20 client.16890 trim_cache size 0 max 16384 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 Processor -- start 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 -- start start 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed384920 con 0x55e3ed5b0400 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed5ad5e0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7fabc1155b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed5b47a0 con 0x55e3ed5bbbd0 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7faba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51298/0 (socket says 172.21.15.5:51298) 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7faba6ffd700 1 -- 172.21.15.5:0/2683291729 learned_addr learned my addr 172.21.15.5:0/2683291729 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.505+0000 7faba67fc700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba77fe700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 -- 172.21.15.5:0/2683291729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 msgr2=0x55e3ed5b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 -- 172.21.15.5:0/2683291729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 msgr2=0x55e3ed5b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 -- 172.21.15.5:0/2683291729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3ed317c60 con 0x55e3ed5b0400 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba77fe700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab98009cd0 tx=0x7fab98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e734330c8b4e6a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7fab8dffb700 1 -- 172.21.15.5:0/2683291729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9802b530 con 0x55e3ed5b0400 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7fab8dffb700 1 -- 172.21.15.5:0/2683291729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab98003b40 con 0x55e3ed5b0400 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.506+0000 7fab8dffb700 1 -- 172.21.15.5:0/2683291729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9802ba70 con 0x55e3ed5b0400 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- 172.21.15.5:0/2683291729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 msgr2=0x55e3ed5c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab98009cd0 tx=0x7fab98009260 comp rx=0 tx=0).stop 2023-09-23T04:51:11.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- 172.21.15.5:0/2683291729 shutdown_connections 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c6e70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2683291729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- 172.21.15.5:0/2683291729 >> 172.21.15.5:0/2683291729 conn(0x55e3ed525260 msgr2=0x55e3ed5b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- 172.21.15.5:0/2683291729 shutdown_connections 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- 172.21.15.5:0/2683291729 wait complete. 2023-09-23T04:51:11.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.507+0000 7fabc1155b80 1 -- start start 2023-09-23T04:51:11.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 Processor -- start 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed5c2bf0 con 0x55e3ed5bbbd0 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed5bf170 con 0x55e3ed5b0d20 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3ed5bf330 con 0x55e3ed5b0400 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7fabc1155b80 10 client.0 _ll_register_callbacks cb 0x7fab9c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.508+0000 7faba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba77fe700 1 -- 172.21.15.5:0/2362732276 learned_addr learned my addr 172.21.15.5:0/2362732276 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 -- 172.21.15.5:0/2362732276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5b0400 msgr2=0x55e3ed5c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 -- 172.21.15.5:0/2362732276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 msgr2=0x55e3ed5bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 -- 172.21.15.5:0/2362732276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3ed5668d0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba77fe700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7faba67fc700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5be6f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0059d0 tx=0x7fab9c020d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885909b4f9b6832 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.509+0000 7fab8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c020410 con 0x55e3ed5b0d20 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab98004390 con 0x55e3ed5b0d20 2023-09-23T04:51:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fab9c020570 con 0x55e3ed5b0d20 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9c020730 con 0x55e3ed5b0d20 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fabc1155b80 10 client.26897 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e3ed355250 con 0x55e3ed5b0d20 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab98004390 con 0x55e3ed5b0d20 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.510+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fab9c0108b0 con 0x55e3ed5b0d20 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 10 client.26897 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 10 client.26897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 20 client.26897 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed6207d0 0x55e3ed622bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 4 client.26897 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3ed4fcb30 con 0x55e3ed6207d0 2023-09-23T04:51:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.511+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed6207d0 0x55e3ed622bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7faba6ffd700 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed6207d0 0x55e3ed622bf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c276ea50104a0316 server_cookie=bc1dbed09940e991 in_seq=0 out_seq=0 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fab8aff5700 10 client.26897 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fab8aff5700 1 -- 172.21.15.5:0/2362732276 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3ed4fcb30 con 0x55e3ed6207d0 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fab8aff5700 10 client.26897 handle_command_reply: tid=0 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 client.26897 shutdown 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 msgr2=0x55e3ed5be6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5be6f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0059d0 tx=0x7fab9c020d00 comp rx=0 tx=0).stop 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 shutdown_connections 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55e3ed6207d0 0x55e3ed622bf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ed5bbbd0 0x55e3ed5bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fab8aff5700 0 client.26897 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e3ed5b0400 0x55e3ed5c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 --2- 172.21.15.5:0/2362732276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e3ed5b0d20 0x55e3ed5be6f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 >> 172.21.15.5:0/2362732276 conn(0x55e3ed525260 msgr2=0x55e3ed5c6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 shutdown_connections 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 1 -- 172.21.15.5:0/2362732276 wait complete. 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.512+0000 7fabc1155b80 20 client.26897 trim_cache size 0 max 16384 2023-09-23T04:51:11.517 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 msgr2=0x7fab900a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 0x7fab900a3c50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faba8074910 tx=0x7fab9c009aa0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 msgr2=0x7faba81a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faba000b070 tx=0x7faba000bc80 comp rx=0 tx=0).stop 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 shutdown_connections 2023-09-23T04:51:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fab900a1790 0x7fab900a3c50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8073ba0 0x7faba81a5290 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faba81010d0 0x7faba8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 --2- 172.21.15.5:0/2072075314 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7faba81061f0 0x7faba8073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 >> 172.21.15.5:0/2072075314 conn(0x7faba80fc990 msgr2=0x7faba80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 shutdown_connections 2023-09-23T04:51:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.513+0000 7fabad7ff700 1 -- 172.21.15.5:0/2072075314 wait complete. 2023-09-23T04:51:11.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:51:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/2228254915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb8101a80 msgr2=0x7f7cb8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/2228254915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb8101a80 0x7f7cb8105ac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009990 tx=0x7f7ca0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/2228254915 shutdown_connections 2023-09-23T04:51:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/2228254915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/2228254915 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 0x7f7cb81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/2228254915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb8101a80 0x7f7cb8105ac0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/2228254915 >> 172.21.15.5:0/2228254915 conn(0x7f7cb80fc990 msgr2=0x7f7cb80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/2228254915 shutdown_connections 2023-09-23T04:51:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.740+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/2228254915 wait complete. 2023-09-23T04:51:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.741+0000 7f7cbd3ff700 1 Processor -- start 2023-09-23T04:51:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 -- start start 2023-09-23T04:51:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 0x7f7cb81a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb810c950 0x7f7cb810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d4e0 con 0x7f7cb810c950 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d650 con 0x7f7cb81061f0 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.742+0000 7f7cbd3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb810d810 con 0x7f7cb81010d0 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56400/0 (socket says 172.21.15.5:56400) 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 -- 172.21.15.5:0/1009383325 learned_addr learned my addr 172.21.15.5:0/1009383325 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 0x7f7cb81a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb810c950 0x7f7cb810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 -- 172.21.15.5:0/1009383325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 msgr2=0x7f7cb81a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 0x7f7cb81a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 -- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb810c950 msgr2=0x7f7cb810ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb810c950 0x7f7cb810ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.743+0000 7f7caffff700 1 -- 172.21.15.5:0/1009383325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00095f0 con 0x7f7cb81061f0 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7caffff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0004760 tx=0x7f7ca0004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7cb4ff9700 1 -- 172.21.15.5:0/1009383325 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca001d070 con 0x7f7cb81061f0 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7cb4ff9700 1 -- 172.21.15.5:0/1009383325 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca0004830 con 0x7f7cb81061f0 2023-09-23T04:51:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb81aa2b0 con 0x7f7cb81061f0 2023-09-23T04:51:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7cb4ff9700 1 -- 172.21.15.5:0/1009383325 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0005280 con 0x7f7cb81061f0 2023-09-23T04:51:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.744+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81aa670 con 0x7f7cb81061f0 2023-09-23T04:51:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.747+0000 7f7cb4ff9700 1 -- 172.21.15.5:0/1009383325 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7ca0005400 con 0x7f7cb81061f0 2023-09-23T04:51:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.749+0000 7f7cd0cd5b80 1 Processor -- start 2023-09-23T04:51:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 -- start start 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921c967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921bff460 con 0x55d921c96d20 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921bf0c20 con 0x55d921ca1bd0 2023-09-23T04:51:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921bee540 con 0x55d921c96400 2023-09-23T04:51:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.750+0000 7f7cb4ff9700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 0x7f7ca40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb4ff9700 1 -- 172.21.15.5:0/1009383325 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7ca00ce820 con 0x7f7cb81061f0 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51340/0 (socket says 172.21.15.5:51340) 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/814497542 learned_addr learned my addr 172.21.15.5:0/814497542 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56404/0 (socket says 172.21.15.5:56404) 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921c967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 0x7f7ca40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 -- 172.21.15.5:0/814497542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 msgr2=0x55d921c967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921c967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 -- 172.21.15.5:0/814497542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 msgr2=0x55d921ca1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 -- 172.21.15.5:0/814497542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d921c4c8d0 con 0x55d921ca1bd0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.751+0000 7f7caffff700 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00cbb30 tx=0x7f7ca0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be222e9673f168 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cad7fa700 1 -- 172.21.15.5:0/814497542 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca009bae0 con 0x55d921ca1bd0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cad7fa700 1 -- 172.21.15.5:0/814497542 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca00a3930 con 0x55d921ca1bd0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cad7fa700 1 -- 172.21.15.5:0/814497542 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca009c940 con 0x55d921ca1bd0 2023-09-23T04:51:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 0x7f7ca40a3bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009ec0 tx=0x7f7cb000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/814497542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 msgr2=0x55d921ca3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.752+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00cbb30 tx=0x7f7ca0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/814497542 shutdown_connections 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921ca3fc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921c967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/814497542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/814497542 >> 172.21.15.5:0/814497542 conn(0x55d921c0b260 msgr2=0x55d921c92790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/814497542 shutdown_connections 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/814497542 wait complete. 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 -- start start 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.753+0000 7f7cd0cd5b80 1 Processor -- start 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921be9e30 con 0x55d921c96d20 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921b86710 con 0x55d921ca1bd0 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921b863b0 con 0x55d921c96400 2023-09-23T04:51:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cd0cd5b80 10 client.0 _ll_register_callbacks cb 0x55d9219e3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54910/0 (socket says 172.21.15.5:54910) 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/1149842491 learned_addr learned my addr 172.21.15.5:0/1149842491 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7caffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.754+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/1149842491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 msgr2=0x55d921cad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/1149842491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 msgr2=0x55d921ca24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/1149842491 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80099e0 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8012a10 tx=0x7f7ca8012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3748403af3c17e0f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7c9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017070 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d921c8a250 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.755+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7ca8004050 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca8004210 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 10 client.26962 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f7cb81aa670 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d921c8a250 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7ca80037f0 con 0x55d921c96400 2023-09-23T04:51:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 10 client.26962 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 10 client.26962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 20 client.26962 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921cb1360 0x55d921cb3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 4 client.26962 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.756+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d921be2e10 con 0x55d921cb1360 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.757+0000 7f7caffff700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921cb1360 0x55d921cb3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.757+0000 7f7caffff700 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921cb1360 0x55d921cb3750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f5a77c81b740456 server_cookie=dfb534f6e47f2a5b in_seq=0 out_seq=0 2023-09-23T04:51:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.757+0000 7f7c9e7fc700 10 client.26962 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:11.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/1149842491 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d921be2e10 con 0x55d921cb1360 2023-09-23T04:51:11.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7c9e7fc700 10 client.26962 handle_command_reply: tid=0 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 client.26962 shutdown 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 msgr2=0x55d921ca1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8012a10 tx=0x7f7ca8012d20 comp rx=0 tx=0).stop 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 shutdown_connections 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921cb1360 0x55d921cb3750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96400 0x55d921ca1fb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7c9e7fc700 0 client.26962 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/1149842491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96d20 0x55d921ca24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 >> 172.21.15.5:0/1149842491 conn(0x55d921c0b260 msgr2=0x55d921c92320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 shutdown_connections 2023-09-23T04:51:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.758+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/1149842491 wait complete. 2023-09-23T04:51:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.759+0000 7f7cd0cd5b80 20 client.26962 trim_cache size 0 max 16384 2023-09-23T04:51:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 Processor -- start 2023-09-23T04:51:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 -- start start 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921c9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921a6a920 con 0x55d921c96400 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921c97620 con 0x55d921ca1bd0 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.772+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921c97bf0 con 0x55d921c96d20 2023-09-23T04:51:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51356/0 (socket says 172.21.15.5:51356) 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/3986652677 learned_addr learned my addr 172.21.15.5:0/3986652677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7caffff700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921c9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/3986652677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 msgr2=0x55d921c9b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921c9b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/3986652677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 msgr2=0x55d921cac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/3986652677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d921c8e480 con 0x55d921c96400 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009340 tx=0x7f7ca80123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a80acaf61a5d1d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cadffb700 1 -- 172.21.15.5:0/3986652677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca802d070 con 0x55d921c96400 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.773+0000 7f7cadffb700 1 -- 172.21.15.5:0/3986652677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca80037c0 con 0x55d921c96400 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cadffb700 1 -- 172.21.15.5:0/3986652677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017040 con 0x55d921c96400 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/3986652677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 msgr2=0x55d921c9b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009340 tx=0x7f7ca80123a0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/3986652677 shutdown_connections 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921c9b3d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/3986652677 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921c9b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/3986652677 >> 172.21.15.5:0/3986652677 conn(0x55d921c0b260 msgr2=0x55d921c9c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/3986652677 shutdown_connections 2023-09-23T04:51:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.774+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/3986652677 wait complete. 2023-09-23T04:51:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 -- start start 2023-09-23T04:51:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 Processor -- start 2023-09-23T04:51:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921ca5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921ca56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cfe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921cfe750 con 0x55d921c96400 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921cfe8c0 con 0x55d921ca1bd0 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d921cfea80 con 0x55d921c96d20 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.775+0000 7f7cd0cd5b80 10 client.0 _ll_register_callbacks cb 0x7f7ca80124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7caffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921ca56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921ca5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7caffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921ca56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54920/0 (socket says 172.21.15.5:54920) 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7caffff700 1 -- 172.21.15.5:0/895528310 learned_addr learned my addr 172.21.15.5:0/895528310 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb77fe700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cfe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 msgr2=0x55d921ca56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921ca56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/895528310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 msgr2=0x55d921cfe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cfe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 -- 172.21.15.5:0/895528310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81aa670 con 0x55d921c96400 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7cb6ffd700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921ca5190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80092a0 tx=0x7f7ca8005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd24d14683d8cae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.776+0000 7f7c9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca802d040 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ca00a3930 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7ca800fcf0 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca8017040 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 10 client.16914 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d921a14de0 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca00a3930 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f7ca8010d20 con 0x55d921c96400 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 20 client.16914 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921d05e00 0x55d921d081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 4 client.16914 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.777+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d921be2b30 con 0x55d921d05e00 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7caffff700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921d05e00 0x55d921d081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7caffff700 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921d05e00 0x55d921d081f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c20c5c206ac20a4 server_cookie=6960247fa8797fb0 in_seq=0 out_seq=0 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7c9e7fc700 10 client.16914 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7c9e7fc700 1 -- 172.21.15.5:0/895528310 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d921be2b30 con 0x55d921d05e00 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7c9e7fc700 10 client.16914 handle_command_reply: tid=0 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.778+0000 7f7cd0cd5b80 1 client.16914 shutdown 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 msgr2=0x55d921ca5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921ca5190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80092a0 tx=0x7f7ca8005340 comp rx=0 tx=0).stop 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 shutdown_connections 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55d921d05e00 0x55d921d081f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d921ca1bd0 0x55d921cfe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d921c96400 0x55d921ca5190 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7c9e7fc700 0 client.16914 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 --2- 172.21.15.5:0/895528310 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d921c96d20 0x55d921ca56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 >> 172.21.15.5:0/895528310 conn(0x55d921c0b260 msgr2=0x55d921c9f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 shutdown_connections 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 1 -- 172.21.15.5:0/895528310 wait complete. 2023-09-23T04:51:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cd0cd5b80 20 client.16914 trim_cache size 0 max 16384 2023-09-23T04:51:11.786 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 msgr2=0x7f7ca40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 0x7f7ca40a3bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009ec0 tx=0x7f7cb000b040 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 msgr2=0x7f7cb81a27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.779+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0004760 tx=0x7f7ca0004790 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 shutdown_connections 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7ca40a1710 0x7f7ca40a3bd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cb810c950 0x7f7cb810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7cb81010d0 0x7f7cb81a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 --2- 172.21.15.5:0/1009383325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7cb81061f0 0x7f7cb81a27f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 >> 172.21.15.5:0/1009383325 conn(0x7f7cb80fc990 msgr2=0x7f7cb80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 shutdown_connections 2023-09-23T04:51:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:11.780+0000 7f7cbd3ff700 1 -- 172.21.15.5:0/1009383325 wait complete. 2023-09-23T04:51:11.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:11 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2869376187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:11.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:11 smithi005 ceph-mon[97638]: pgmap v150: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2023-09-23T04:51:11.797 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:51:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:11 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2869376187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:11 smithi118 ceph-mon[134297]: pgmap v150: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2023-09-23T04:51:11.942 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.39 MB/sec execute 107 sec latency 4100.385 ms 2023-09-23T04:51:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:11 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2869376187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:11 smithi069 ceph-mon[133931]: pgmap v150: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2023-09-23T04:51:12.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 -- 172.21.15.5:0/2743446070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301061f0 msgr2=0x7f203010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 --2- 172.21.15.5:0/2743446070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301061f0 0x7f203010dd70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f202800b0a0 tx=0x7f202800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 -- 172.21.15.5:0/2743446070 shutdown_connections 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 --2- 172.21.15.5:0/2743446070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301061f0 0x7f203010dd70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 --2- 172.21.15.5:0/2743446070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301010d0 0x7f20301014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 --2- 172.21.15.5:0/2743446070 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030101a80 0x7f2030105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 -- 172.21.15.5:0/2743446070 >> 172.21.15.5:0/2743446070 conn(0x7f20300fc990 msgr2=0x7f20300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 -- 172.21.15.5:0/2743446070 shutdown_connections 2023-09-23T04:51:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.008+0000 7f20367ff700 1 -- 172.21.15.5:0/2743446070 wait complete. 2023-09-23T04:51:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.009+0000 7f20367ff700 1 Processor -- start 2023-09-23T04:51:12.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.009+0000 7f20367ff700 1 -- start start 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301061f0 0x7f2030073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 0x7f20301a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301a59a0 con 0x7f20301010d0 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301a5b10 con 0x7f20301061f0 2023-09-23T04:51:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f20367ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301a5cd0 con 0x7f2030073ba0 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f202ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f202ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51392/0 (socket says 172.21.15.5:51392) 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f202ffff700 1 -- 172.21.15.5:0/1196926922 learned_addr learned my addr 172.21.15.5:0/1196926922 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.010+0000 7f202f7fe700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301061f0 0x7f2030073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 0x7f20301a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 -- 172.21.15.5:0/1196926922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 msgr2=0x7f20301a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 0x7f20301a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 -- 172.21.15.5:0/1196926922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301061f0 msgr2=0x7f2030073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301061f0 0x7f2030073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 -- 172.21.15.5:0/1196926922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028009d00 con 0x7f20301010d0 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 0x7f20301a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202ffff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f202000e9a0 tx=0x7f202000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202d7fa700 1 -- 172.21.15.5:0/1196926922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020009790 con 0x7f20301010d0 2023-09-23T04:51:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202d7fa700 1 -- 172.21.15.5:0/1196926922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202000cbd0 con 0x7f20301010d0 2023-09-23T04:51:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20301a5fb0 con 0x7f20301010d0 2023-09-23T04:51:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f202d7fa700 1 -- 172.21.15.5:0/1196926922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020018680 con 0x7f20301010d0 2023-09-23T04:51:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.011+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20301a6430 con 0x7f20301010d0 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.014+0000 7f202d7fa700 1 -- 172.21.15.5:0/1196926922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f202000cd40 con 0x7f20301010d0 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 Processor -- start 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 -- start start 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f202d7fa700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 0x7f20180a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ebc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ebefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8e1a460 con 0x55b0d8eb1d20 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8e0bc20 con 0x55b0d8ebcbd0 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.016+0000 7f204a176b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8e09540 con 0x55b0d8eb1400 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202d7fa700 1 -- 172.21.15.5:0/1196926922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2020014070 con 0x7f20301010d0 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 0x7f20180a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ebc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54942/0 (socket says 172.21.15.5:54942) 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 -- 172.21.15.5:0/1676204401 learned_addr learned my addr 172.21.15.5:0/1676204401 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ebefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 -- 172.21.15.5:0/1676204401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8ebcbd0 msgr2=0x55b0d8ebefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ebefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 -- 172.21.15.5:0/1676204401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 msgr2=0x55b0d8ebc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ebc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 -- 172.21.15.5:0/1676204401 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d8e678d0 con 0x55b0d8eb1400 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.017+0000 7f202ffff700 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2020096fc0 tx=0x7f202009a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14647c2bd096bf4a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f20167fc700 1 -- 172.21.15.5:0/1676204401 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202009ac50 con 0x55b0d8eb1400 2023-09-23T04:51:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 0x7f20180a3bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f202800b780 tx=0x7f20280092e0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f20167fc700 1 -- 172.21.15.5:0/1676204401 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202009a550 con 0x55b0d8eb1400 2023-09-23T04:51:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f20167fc700 1 -- 172.21.15.5:0/1676204401 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202009aa10 con 0x55b0d8eb1400 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 -- 172.21.15.5:0/1676204401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 msgr2=0x55b0d8eb17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2020096fc0 tx=0x7f202009a450 comp rx=0 tx=0).stop 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 -- 172.21.15.5:0/1676204401 shutdown_connections 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ebefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb17e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 --2- 172.21.15.5:0/1676204401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ebc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.018+0000 7f204a176b80 1 -- 172.21.15.5:0/1676204401 >> 172.21.15.5:0/1676204401 conn(0x55b0d8e26260 msgr2=0x55b0d8ead790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.019+0000 7f204a176b80 1 -- 172.21.15.5:0/1676204401 shutdown_connections 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.019+0000 7f204a176b80 1 -- 172.21.15.5:0/1676204401 wait complete. 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.019+0000 7f204a176b80 1 -- start start 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.019+0000 7f204a176b80 1 Processor -- start 2023-09-23T04:51:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8e04e30 con 0x55b0d8ebcbd0 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8da1710 con 0x55b0d8eb1400 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8da13b0 con 0x55b0d8eb1d20 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f204a176b80 10 client.0 _ll_register_callbacks cb 0x55b0d8bfe800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f2034d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54956/0 (socket says 172.21.15.5:54956) 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 -- 172.21.15.5:0/3136589260 learned_addr learned my addr 172.21.15.5:0/3136589260 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202ffff700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 -- 172.21.15.5:0/3136589260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 msgr2=0x55b0d8eb98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 -- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 msgr2=0x55b0d8eba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.020+0000 7f202f7fe700 1 -- 172.21.15.5:0/3136589260 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020096040 con 0x55b0d8eb1d20 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f202f7fe700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2024012700 tx=0x7f2024012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8c3503aba91e83 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f200f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024017070 con 0x55b0d8eb1d20 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0d8ea5250 con 0x55b0d8eb1d20 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2024004830 con 0x55b0d8eb1d20 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202401c470 con 0x55b0d8eb1d20 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f204a176b80 10 client.26980 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.021+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b0d8c2fde0 con 0x55b0d8eb1d20 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0d8ea5250 con 0x55b0d8eb1d20 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2024010730 con 0x55b0d8eb1d20 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 10 client.26980 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 20 client.26980 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8ecc6b0 0x55b0d8ecead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 4 client.26980 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0d8dfde10 con 0x55b0d8ecc6b0 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.022+0000 7f202ffff700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8ecc6b0 0x55b0d8ecead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f202ffff700 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8ecc6b0 0x55b0d8ecead0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc0fa7c25f73025 server_cookie=87005214301a8d8 in_seq=0 out_seq=0 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f200f7fe700 10 client.26980 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f200f7fe700 1 -- 172.21.15.5:0/3136589260 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0d8dfde10 con 0x55b0d8ecc6b0 2023-09-23T04:51:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f200f7fe700 10 client.26980 handle_command_reply: tid=0 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f204a176b80 1 client.26980 shutdown 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 msgr2=0x55b0d8eb9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.023+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2024012700 tx=0x7f2024012a10 comp rx=0 tx=0).stop 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 shutdown_connections 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8ecc6b0 0x55b0d8ecead0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8eb98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 --2- 172.21.15.5:0/3136589260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb9e00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f200f7fe700 0 client.26980 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 >> 172.21.15.5:0/3136589260 conn(0x55b0d8e26260 msgr2=0x55b0d8ead320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 shutdown_connections 2023-09-23T04:51:12.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 1 -- 172.21.15.5:0/3136589260 wait complete. 2023-09-23T04:51:12.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.024+0000 7f204a176b80 20 client.26980 trim_cache size 0 max 16384 2023-09-23T04:51:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.041+0000 7f204a176b80 1 Processor -- start 2023-09-23T04:51:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.041+0000 7f204a176b80 1 -- start start 2023-09-23T04:51:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eb70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c85920 con 0x55b0d8eb1d20 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8eb2bf0 con 0x55b0d8eb1400 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f204a176b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8ec3bf0 con 0x55b0d8ebcbd0 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56476/0 (socket says 172.21.15.5:56476) 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51432/0 (socket says 172.21.15.5:51432) 2023-09-23T04:51:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 -- 172.21.15.5:0/132501918 learned_addr learned my addr 172.21.15.5:0/132501918 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f2034d9c700 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eb70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 -- 172.21.15.5:0/132501918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8ebcbd0 msgr2=0x55b0d8eb70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eb70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 -- 172.21.15.5:0/132501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 msgr2=0x55b0d8eb6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202ffff700 1 -- 172.21.15.5:0/132501918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d8ea9480 con 0x55b0d8eb1400 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.042+0000 7f202f7fe700 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.043+0000 7f202ffff700 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f202009a250 tx=0x7f20200ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6a2c786b73d950 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.043+0000 7f2016ffd700 1 -- 172.21.15.5:0/132501918 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200a28e0 con 0x55b0d8eb1400 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.043+0000 7f2016ffd700 1 -- 172.21.15.5:0/132501918 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2020099a40 con 0x55b0d8eb1400 2023-09-23T04:51:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.043+0000 7f2016ffd700 1 -- 172.21.15.5:0/132501918 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200a2670 con 0x55b0d8eb1400 2023-09-23T04:51:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- 172.21.15.5:0/132501918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 msgr2=0x55b0d8ec36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f202009a250 tx=0x7f20200ba040 comp rx=0 tx=0).stop 2023-09-23T04:51:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- 172.21.15.5:0/132501918 shutdown_connections 2023-09-23T04:51:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8eb70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec36b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 --2- 172.21.15.5:0/132501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8eb1d20 0x55b0d8eb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- 172.21.15.5:0/132501918 >> 172.21.15.5:0/132501918 conn(0x55b0d8e26260 msgr2=0x55b0d8eb4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- 172.21.15.5:0/132501918 shutdown_connections 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- 172.21.15.5:0/132501918 wait complete. 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.044+0000 7f204a176b80 1 -- start start 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 Processor -- start 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ec1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8f1f4e0 con 0x55b0d8ebcbd0 2023-09-23T04:51:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8f1f620 con 0x55b0d8eb1400 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8f1f7e0 con 0x55b0d8eb1d20 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.045+0000 7f204a176b80 10 client.0 _ll_register_callbacks cb 0x7f2024009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f2034d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ec1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f2034d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51438/0 (socket says 172.21.15.5:51438) 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f2034d9c700 1 -- 172.21.15.5:0/1998532538 learned_addr learned my addr 172.21.15.5:0/1998532538 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202ffff700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 -- 172.21.15.5:0/1998532538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 msgr2=0x55b0d8ec16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 -- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 msgr2=0x55b0d8ec2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 -- 172.21.15.5:0/1998532538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0d8c2fde0 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f2034d9c700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f202f7fe700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ec1be0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f20240097c0 tx=0x7f20240058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f23ab7b411dfed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.046+0000 7f200f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024017040 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f202009a550 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f2024000e40 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20240054e0 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 10 client.26986 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f20200b2e70 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2024035020 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f2024003c20 con 0x55b0d8eb1d20 2023-09-23T04:51:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 10 client.26986 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.047+0000 7f204a176b80 20 client.26986 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8f20fb0 0x55b0d8f233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f204a176b80 4 client.26986 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b0d8dfdb30 con 0x55b0d8f20fb0 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f202ffff700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8f20fb0 0x55b0d8f233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f202ffff700 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8f20fb0 0x55b0d8f233d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6d61fe44c703b6 server_cookie=8275d73303319f1b in_seq=0 out_seq=0 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.048+0000 7f200f7fe700 10 client.26986 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f200f7fe700 1 -- 172.21.15.5:0/1998532538 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b0d8dfdb30 con 0x55b0d8f20fb0 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f200f7fe700 10 client.26986 handle_command_reply: tid=0 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 client.26986 shutdown 2023-09-23T04:51:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 msgr2=0x55b0d8ec1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ec1be0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f20240097c0 tx=0x7f20240058d0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 shutdown_connections 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55b0d8f20fb0 0x55b0d8f233d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0d8ebcbd0 0x55b0d8ec2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b0d8eb1400 0x55b0d8ec16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f200f7fe700 0 client.26986 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 --2- 172.21.15.5:0/1998532538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b0d8eb1d20 0x55b0d8ec1be0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 >> 172.21.15.5:0/1998532538 conn(0x55b0d8e26260 msgr2=0x55b0d8eb3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 shutdown_connections 2023-09-23T04:51:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 1 -- 172.21.15.5:0/1998532538 wait complete. 2023-09-23T04:51:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.049+0000 7f204a176b80 20 client.26986 trim_cache size 0 max 16384 2023-09-23T04:51:12.055 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 msgr2=0x7f20180a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 0x7f20180a3bd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f202800b780 tx=0x7f20280092e0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 msgr2=0x7f2030073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f202000e9a0 tx=0x7f202000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 shutdown_connections 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f20180a1710 0x7f20180a3bd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2030073ba0 0x7f20301a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20301010d0 0x7f2030073120 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 --2- 172.21.15.5:0/1196926922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20301061f0 0x7f2030073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 >> 172.21.15.5:0/1196926922 conn(0x7f20300fc990 msgr2=0x7f20300fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 shutdown_connections 2023-09-23T04:51:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:12.050+0000 7f20367ff700 1 -- 172.21.15.5:0/1196926922 wait complete. 2023-09-23T04:51:12.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.33 MB/sec execute 108 sec latency 5100.516 ms 2023-09-23T04:51:13.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:51:13.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:51:13.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:12 smithi005 ceph-mon[97638]: pgmap v151: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-23T04:51:13.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:12 smithi118 ceph-mon[134297]: pgmap v151: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-23T04:51:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:12 smithi069 ceph-mon[133931]: pgmap v151: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-09-23T04:51:13.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 -- 172.21.15.5:0/2195407075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 msgr2=0x7f28b0111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 --2- 172.21.15.5:0/2195407075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b0111930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2898009990 tx=0x7f2898009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 -- 172.21.15.5:0/2195407075 shutdown_connections 2023-09-23T04:51:13.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 --2- 172.21.15.5:0/2195407075 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b0111e70 0x7f28b0114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 --2- 172.21.15.5:0/2195407075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b00731c0 0x7f28b00735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 --2- 172.21.15.5:0/2195407075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b0111930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 -- 172.21.15.5:0/2195407075 >> 172.21.15.5:0/2195407075 conn(0x7f28b00fc990 msgr2=0x7f28b00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 -- 172.21.15.5:0/2195407075 shutdown_connections 2023-09-23T04:51:13.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.290+0000 7f28b53ff700 1 -- 172.21.15.5:0/2195407075 wait complete. 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.291+0000 7f28b53ff700 1 Processor -- start 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.291+0000 7f28b53ff700 1 -- start start 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b00731c0 0x7f28b019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 0x7f28b01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01a25e0 con 0x7f28b0073ae0 2023-09-23T04:51:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01a2750 con 0x7f28b0111e70 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28b53ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01a2910 con 0x7f28b00731c0 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51442/0 (socket says 172.21.15.5:51442) 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28a7fff700 1 -- 172.21.15.5:0/4220649818 learned_addr learned my addr 172.21.15.5:0/4220649818 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b00731c0 0x7f28b019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28af7fe700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 0x7f28b01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28a7fff700 1 -- 172.21.15.5:0/4220649818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b00731c0 msgr2=0x7f28b019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.292+0000 7f28a7fff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b00731c0 0x7f28b019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28a7fff700 1 -- 172.21.15.5:0/4220649818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 msgr2=0x7f28b01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28a7fff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 0x7f28b01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28a7fff700 1 -- 172.21.15.5:0/4220649818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980095f0 con 0x7f28b0073ae0 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28af7fe700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 0x7f28b01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28a7fff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2898005910 tx=0x7f2898004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28acff9700 1 -- 172.21.15.5:0/4220649818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289801d070 con 0x7f28b0073ae0 2023-09-23T04:51:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b01a2b90 con 0x7f28b0073ae0 2023-09-23T04:51:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28acff9700 1 -- 172.21.15.5:0/4220649818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2898022470 con 0x7f28b0073ae0 2023-09-23T04:51:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28acff9700 1 -- 172.21.15.5:0/4220649818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898017650 con 0x7f28b0073ae0 2023-09-23T04:51:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.293+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b01aa430 con 0x7f28b0073ae0 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.296+0000 7f28acff9700 1 -- 172.21.15.5:0/4220649818 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f28980225e0 con 0x7f28b0073ae0 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 Processor -- start 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28acff9700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 0x7f289c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 -- start start 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28acff9700 1 -- 172.21.15.5:0/4220649818 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f28980ce930 con 0x7f28b0073ae0 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 0x7f289c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 0x56004a2e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a2454e0 con 0x56004a2e7c50 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a236ca0 con 0x56004a2dcda0 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.298+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a2345c0 con 0x56004a2dc480 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51456/0 (socket says 172.21.15.5:51456) 2023-09-23T04:51:13.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 -- 172.21.15.5:0/2148792133 learned_addr learned my addr 172.21.15.5:0/2148792133 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28a7fff700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 0x56004a2e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28aeffd700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 0x7f289c0a3bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009c70 tx=0x7f28a0011720 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 -- 172.21.15.5:0/2148792133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 msgr2=0x56004a2dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 -- 172.21.15.5:0/2148792133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 msgr2=0x56004a2e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 0x56004a2e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28af7fe700 1 -- 172.21.15.5:0/2148792133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004a292950 con 0x56004a2e7c50 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28aeffd700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.299+0000 7f28a7fff700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 0x56004a2e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28af7fe700 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28a8003e40 tx=0x7f28a8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ea5db7a5b7c8d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28a57fa700 1 -- 172.21.15.5:0/2148792133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80092d0 con 0x56004a2e7c50 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28a57fa700 1 -- 172.21.15.5:0/2148792133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a800f040 con 0x56004a2e7c50 2023-09-23T04:51:13.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28a57fa700 1 -- 172.21.15.5:0/2148792133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a800e750 con 0x56004a2e7c50 2023-09-23T04:51:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/2148792133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 msgr2=0x56004a2ea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.300+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28a8003e40 tx=0x7f28a8009920 comp rx=0 tx=0).stop 2023-09-23T04:51:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/2148792133 shutdown_connections 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2e7c50 0x56004a2ea040 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/2148792133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2dcda0 0x56004a2e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/2148792133 >> 172.21.15.5:0/2148792133 conn(0x56004a2512e0 msgr2=0x56004a2d8810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/2148792133 shutdown_connections 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/2148792133 wait complete. 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 -- start start 2023-09-23T04:51:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.301+0000 7f28c8ce2b80 1 Processor -- start 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a22feb0 con 0x56004a2dcda0 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a1cc790 con 0x56004a2e7c50 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a1cc430 con 0x56004a2dc480 2023-09-23T04:51:13.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28c8ce2b80 10 client.0 _ll_register_callbacks cb 0x56004a029880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51470/0 (socket says 172.21.15.5:51470) 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28aeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28a7fff700 1 -- 172.21.15.5:0/590148000 learned_addr learned my addr 172.21.15.5:0/590148000 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.302+0000 7f28af7fe700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 -- 172.21.15.5:0/590148000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 msgr2=0x56004a2e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 -- 172.21.15.5:0/590148000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 msgr2=0x56004a2e53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 -- 172.21.15.5:0/590148000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b01aa430 con 0x56004a2dcda0 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28a7fff700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28980979f0 tx=0x7f289809b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191c98c0646f048e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28967fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289809abe0 con 0x56004a2dcda0 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f28980b2030 con 0x56004a2dcda0 2023-09-23T04:51:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56004a2d02d0 con 0x56004a2dcda0 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f289809bbf0 con 0x56004a2dcda0 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28c8ce2b80 10 client.16944 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f28a8003b20 con 0x56004a2dcda0 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.303+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56004a2d02d0 con 0x56004a2dcda0 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f28980a3480 con 0x56004a2dcda0 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 10 client.16944 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:13.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 20 client.16944 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a2f7380 0x56004a2f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 4 client.16944 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56004a228e90 con 0x56004a2f7380 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.304+0000 7f28aeffd700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a2f7380 0x56004a2f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28aeffd700 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a2f7380 0x56004a2f97a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b52c7ad621d36fc server_cookie=ad05213c23800e02 in_seq=0 out_seq=0 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28967fc700 10 client.16944 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28967fc700 1 -- 172.21.15.5:0/590148000 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56004a228e90 con 0x56004a2f7380 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28967fc700 10 client.16944 handle_command_reply: tid=0 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28c8ce2b80 1 client.16944 shutdown 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 msgr2=0x56004a2e4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28980979f0 tx=0x7f289809b120 comp rx=0 tx=0).stop 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 shutdown_connections 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.305+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a2f7380 0x56004a2f97a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28967fc700 0 client.16944 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/590148000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e4e80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 >> 172.21.15.5:0/590148000 conn(0x56004a2512e0 msgr2=0x56004a2d8400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 shutdown_connections 2023-09-23T04:51:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/590148000 wait complete. 2023-09-23T04:51:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.306+0000 7f28c8ce2b80 20 client.16944 trim_cache size 0 max 16384 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 Processor -- start 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 -- start start 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a0b09a0 con 0x56004a2dcda0 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a2dd450 con 0x56004a2e7c50 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.324+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a2f3570 con 0x56004a2dc480 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55014/0 (socket says 172.21.15.5:55014) 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 -- 172.21.15.5:0/4210100157 learned_addr learned my addr 172.21.15.5:0/4210100157 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28af7fe700 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 -- 172.21.15.5:0/4210100157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 msgr2=0x56004a2e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 -- 172.21.15.5:0/4210100157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 msgr2=0x56004a2e1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 -- 172.21.15.5:0/4210100157 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004a2e44b0 con 0x56004a2dc480 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28a7fff700 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.325+0000 7f28aeffd700 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56004a2fa0b0 tx=0x7f28a001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c57bc7b96d25bc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28a5ffb700 1 -- 172.21.15.5:0/4210100157 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0023040 con 0x56004a2dc480 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28a5ffb700 1 -- 172.21.15.5:0/4210100157 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a0028ce0 con 0x56004a2dc480 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28a5ffb700 1 -- 172.21.15.5:0/4210100157 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a001d6a0 con 0x56004a2dc480 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/4210100157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 msgr2=0x56004a2e5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56004a2fa0b0 tx=0x7f28a001c000 comp rx=0 tx=0).stop 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/4210100157 shutdown_connections 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a2e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2e5010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/4210100157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/4210100157 >> 172.21.15.5:0/4210100157 conn(0x56004a2512e0 msgr2=0x56004a2e4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/4210100157 shutdown_connections 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.326+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/4210100157 wait complete. 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 -- start start 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 Processor -- start 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a344340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a3448f0 con 0x56004a2dcda0 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a344a60 con 0x56004a2e7c50 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004a344c20 con 0x56004a2dc480 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.327+0000 7f28c8ce2b80 10 client.0 _ll_register_callbacks cb 0x56004a029880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28aeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51490/0 (socket says 172.21.15.5:51490) 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28aeffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2eda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:55022/0 (socket says 172.21.15.5:55022) 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 -- 172.21.15.5:0/1151225426 learned_addr learned my addr 172.21.15.5:0/1151225426 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28af7fe700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a344340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 -- 172.21.15.5:0/1151225426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 msgr2=0x56004a2eda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2eda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 -- 172.21.15.5:0/1151225426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 msgr2=0x56004a344340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a344340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 -- 172.21.15.5:0/1151225426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a8003b20 con 0x56004a2dcda0 2023-09-23T04:51:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28af7fe700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a344340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28a7fff700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f289809bb50 tx=0x7f28980a3570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee4abe432137525 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28967fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289809bbf0 con 0x56004a2dcda0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28a800f040 con 0x56004a2dcda0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.328+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f28980b2070 con 0x56004a2dcda0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f289809f330 con 0x56004a2dcda0 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 10 client.16950 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f28a001e040 con 0x56004a2dcda0 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a800f040 con 0x56004a2dcda0 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f28980972e0 con 0x56004a2dcda0 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 20 client.16950 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a34d250 0x56004a34f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 4 client.16950 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:51:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56004a228bb0 con 0x56004a34d250 2023-09-23T04:51:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.329+0000 7f28aeffd700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a34d250 0x56004a34f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.330+0000 7f28aeffd700 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a34d250 0x56004a34f670 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a486729a5bab9f1 server_cookie=e32c4f23d751c5eb in_seq=0 out_seq=0 2023-09-23T04:51:13.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.330+0000 7f28967fc700 10 client.16950 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.331+0000 7f28967fc700 1 -- 172.21.15.5:0/1151225426 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56004a228bb0 con 0x56004a34d250 2023-09-23T04:51:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.331+0000 7f28967fc700 10 client.16950 handle_command_reply: tid=0 2023-09-23T04:51:13.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.331+0000 7f28c8ce2b80 1 client.16950 shutdown 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 msgr2=0x56004a2edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f289809bb50 tx=0x7f28980a3570 comp rx=0 tx=0).stop 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 shutdown_connections 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56004a34d250 0x56004a34f670 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56004a2e7c50 0x56004a344340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56004a2dc480 0x56004a2eda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 --2- 172.21.15.5:0/1151225426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56004a2dcda0 0x56004a2edf70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 >> 172.21.15.5:0/1151225426 conn(0x56004a2512e0 msgr2=0x56004a2e5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28967fc700 0 client.16950 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 shutdown_connections 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 1 -- 172.21.15.5:0/1151225426 wait complete. 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.332+0000 7f28c8ce2b80 20 client.16950 trim_cache size 0 max 16384 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:13.340 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:51:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8a167350-2a27-4f8c-8744-63c9101ddb51", 2023-09-23T04:51:13.341 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:51:13.341 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 msgr2=0x7f289c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 0x7f289c0a3bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009c70 tx=0x7f28a0011720 comp rx=0 tx=0).stop 2023-09-23T04:51:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 msgr2=0x7f28b019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2898005910 tx=0x7f2898004710 comp rx=0 tx=0).stop 2023-09-23T04:51:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 shutdown_connections 2023-09-23T04:51:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f289c0a1710 0x7f289c0a3bd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28b0111e70 0x7f28b01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28b00731c0 0x7f28b019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 --2- 172.21.15.5:0/4220649818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b0073ae0 0x7f28b019db40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 >> 172.21.15.5:0/4220649818 conn(0x7f28b00fc990 msgr2=0x7f28b00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.333+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 shutdown_connections 2023-09-23T04:51:13.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.334+0000 7f28b53ff700 1 -- 172.21.15.5:0/4220649818 wait complete. 2023-09-23T04:51:13.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/1475407683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 msgr2=0x7f9cf810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/1475407683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf810dd70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000b0a0 tx=0x7f9cf000b3b0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/1475407683 shutdown_connections 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/1475407683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf810dd70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/1475407683 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf81010d0 0x7f9cf81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/1475407683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf8101a80 0x7f9cf8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/1475407683 >> 172.21.15.5:0/1475407683 conn(0x7f9cf80fc990 msgr2=0x7f9cf80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/1475407683 shutdown_connections 2023-09-23T04:51:13.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.563+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/1475407683 wait complete. 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.564+0000 7f9cfe3ff700 1 Processor -- start 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 -- start start 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf81010d0 0x7f9cf81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf819dc50 0x7f9cf819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819e6d0 con 0x7f9cf81061f0 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819e840 con 0x7f9cf81010d0 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfe3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf819ea00 con 0x7f9cf819dc50 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51500/0 (socket says 172.21.15.5:51500) 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9ceffff700 1 -- 172.21.15.5:0/2991455886 learned_addr learned my addr 172.21.15.5:0/2991455886 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf81010d0 0x7f9cf81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.565+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf819dc50 0x7f9cf819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 -- 172.21.15.5:0/2991455886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf819dc50 msgr2=0x7f9cf819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf819dc50 0x7f9cf819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 -- 172.21.15.5:0/2991455886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf81010d0 msgr2=0x7f9cf81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf81010d0 0x7f9cf81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 -- 172.21.15.5:0/2991455886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009d00 con 0x7f9cf81061f0 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000e970 tx=0x7f9ce000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9cf5ffb700 1 -- 172.21.15.5:0/2991455886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0009790 con 0x7f9cf81061f0 2023-09-23T04:51:13.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9cf5ffb700 1 -- 172.21.15.5:0/2991455886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce000c7e0 con 0x7f9cf81061f0 2023-09-23T04:51:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf81a5f60 con 0x7f9cf81061f0 2023-09-23T04:51:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9cf5ffb700 1 -- 172.21.15.5:0/2991455886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00054e0 con 0x7f9cf81061f0 2023-09-23T04:51:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.566+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81a63e0 con 0x7f9cf81061f0 2023-09-23T04:51:13.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.569+0000 7f9cf5ffb700 1 -- 172.21.15.5:0/2991455886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f9ce000c950 con 0x7f9cf81061f0 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9d11c3bb80 1 Processor -- start 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9cf5ffb700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 0x7f9ce40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9cf5ffb700 1 -- 172.21.15.5:0/2991455886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9ce0014070 con 0x7f9cf81061f0 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9d11c3bb80 1 -- start start 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 0x7f9ce40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.571+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x561617a00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161795c460 con 0x5616179f3d20 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161794dc20 con 0x5616179f3400 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56161794b540 con 0x5616179febd0 2023-09-23T04:51:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9cf7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51510/0 (socket says 172.21.15.5:51510) 2023-09-23T04:51:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 -- 172.21.15.5:0/153559519 learned_addr learned my addr 172.21.15.5:0/153559519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x561617a00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 0x7f9ce40a3bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00062a0 tx=0x7f9cf0006210 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 -- 172.21.15.5:0/153559519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 msgr2=0x561617a00fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x561617a00fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 -- 172.21.15.5:0/153559519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 msgr2=0x5616179f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9ceffff700 1 -- 172.21.15.5:0/153559519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616179a98d0 con 0x5616179f3d20 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.572+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9ceffff700 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0096ec0 tx=0x7f9ce0099e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45a50ca5976c6d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9cee7fc700 1 -- 172.21.15.5:0/153559519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce009a450 con 0x5616179f3d20 2023-09-23T04:51:13.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9cee7fc700 1 -- 172.21.15.5:0/153559519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce009e5a0 con 0x5616179f3d20 2023-09-23T04:51:13.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9cee7fc700 1 -- 172.21.15.5:0/153559519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce009e3d0 con 0x5616179f3d20 2023-09-23T04:51:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/153559519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 msgr2=0x5616179fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.573+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0096ec0 tx=0x7f9ce0099e30 comp rx=0 tx=0).stop 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/153559519 shutdown_connections 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x561617a00fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/153559519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fe690 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/153559519 >> 172.21.15.5:0/153559519 conn(0x561617968260 msgr2=0x5616179ef790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/153559519 shutdown_connections 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/153559519 wait complete. 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 -- start start 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.574+0000 7f9d11c3bb80 1 Processor -- start 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x5616179fb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x5616179fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561617946e30 con 0x5616179febd0 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616178e3710 con 0x5616179f3d20 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616178e33b0 con 0x5616179f3400 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9d11c3bb80 10 client.0 _ll_register_callbacks cb 0x561617740800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9cfc99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9cfc99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51520/0 (socket says 172.21.15.5:51520) 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9cfc99c700 1 -- 172.21.15.5:0/2104761924 learned_addr learned my addr 172.21.15.5:0/2104761924 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x5616179fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.575+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x5616179fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 -- 172.21.15.5:0/2104761924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 msgr2=0x5616179fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x5616179fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 -- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 msgr2=0x5616179fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 -- 172.21.15.5:0/2104761924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80098e0 con 0x5616179f3d20 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x5616179fbe50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0096ec0 tx=0x7f9ce009ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c11a0be22866c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9cd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce009a450 con 0x5616179f3d20 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616179e7250 con 0x5616179f3d20 2023-09-23T04:51:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9ce00b1020 con 0x5616179f3d20 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce0005660 con 0x5616179f3d20 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9d11c3bb80 10 client.26963 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.576+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f9cf81a63e0 con 0x5616179f3d20 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce00a6080 con 0x5616179f3d20 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9ce0099940 con 0x5616179f3d20 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 10 client.26963 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 20 client.26963 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a0e3c0 0x561617a107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 4 client.26963 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56161793fe10 con 0x561617a0e3c0 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.577+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a0e3c0 0x561617a107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.578+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a0e3c0 0x561617a107e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b205b378607f4e21 server_cookie=37b4dd3d277f28b8 in_seq=0 out_seq=0 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.578+0000 7f9cd77fe700 10 client.26963 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.578+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2104761924 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56161793fe10 con 0x561617a0e3c0 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.578+0000 7f9cd77fe700 10 client.26963 handle_command_reply: tid=0 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.578+0000 7f9d11c3bb80 1 client.26963 shutdown 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 msgr2=0x5616179fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x5616179fbe50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0096ec0 tx=0x7f9ce009ac30 comp rx=0 tx=0).stop 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 shutdown_connections 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a0e3c0 0x561617a107e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x5616179fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x5616179fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2104761924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x5616179fbe50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9cd77fe700 0 client.26963 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 >> 172.21.15.5:0/2104761924 conn(0x561617968260 msgr2=0x5616179ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 shutdown_connections 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2104761924 wait complete. 2023-09-23T04:51:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.579+0000 7f9d11c3bb80 20 client.26963 trim_cache size 0 max 16384 2023-09-23T04:51:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.596+0000 7f9d11c3bb80 1 Processor -- start 2023-09-23T04:51:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.596+0000 7f9d11c3bb80 1 -- start start 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x5616179f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616177c7920 con 0x5616179f3d20 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616179e98c0 con 0x5616179f3400 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616179f67b0 con 0x5616179febd0 2023-09-23T04:51:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51522/0 (socket says 172.21.15.5:51522) 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 -- 172.21.15.5:0/3423663592 learned_addr learned my addr 172.21.15.5:0/3423663592 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9cfc99c700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x5616179f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 -- 172.21.15.5:0/3423663592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 msgr2=0x5616179f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x5616179f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 -- 172.21.15.5:0/3423663592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 msgr2=0x5616179fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9ceffff700 1 -- 172.21.15.5:0/3423663592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561617771de0 con 0x5616179f3d20 2023-09-23T04:51:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.597+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9ceffff700 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000cf90 tx=0x7f9ce000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5099516440f94dd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9ceeffd700 1 -- 172.21.15.5:0/3423663592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0096040 con 0x5616179f3d20 2023-09-23T04:51:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9ceeffd700 1 -- 172.21.15.5:0/3423663592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5616179e7250 con 0x5616179f3d20 2023-09-23T04:51:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9ceeffd700 1 -- 172.21.15.5:0/3423663592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0099940 con 0x5616179f3d20 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/3423663592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 msgr2=0x5616179fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000cf90 tx=0x7f9ce000cc50 comp rx=0 tx=0).stop 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/3423663592 shutdown_connections 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179febd0 0x5616179f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.598+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3400 0x5616179fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/3423663592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179f3d20 0x5616179fbfe0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/3423663592 >> 172.21.15.5:0/3423663592 conn(0x561617968260 msgr2=0x5616179fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/3423663592 shutdown_connections 2023-09-23T04:51:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/3423663592 wait complete. 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 -- start start 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.599+0000 7f9d11c3bb80 1 Processor -- start 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x561617a04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x561617a04e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x561617a611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561617a05790 con 0x5616179febd0 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561617a616f0 con 0x5616179f3d20 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561617a61830 con 0x5616179f3400 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9d11c3bb80 10 client.0 _ll_register_callbacks cb 0x7f9ce8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9cfc99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x561617a611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9cfc99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x561617a611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51530/0 (socket says 172.21.15.5:51530) 2023-09-23T04:51:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9cfc99c700 1 -- 172.21.15.5:0/2448480004 learned_addr learned my addr 172.21.15.5:0/2448480004 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.600+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x561617a04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x561617a04e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 -- 172.21.15.5:0/2448480004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 msgr2=0x561617a04e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x561617a04e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 -- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 msgr2=0x561617a611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x561617a611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 -- 172.21.15.5:0/2448480004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce007b070 con 0x5616179f3400 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cf7fff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x561617a04940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561617a10dc0 tx=0x7f9ce80099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293206128191e667 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8017070 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561617a0b040 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.601+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f9ce801f040 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce801f200 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 10 client.27022 fetch_fsmap learned FSMap version 33 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56161775ac60 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561617a0b040 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 33) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9ce80104c0 con 0x5616179f3400 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 10 client.27022 fetch_fsmap finished waiting for FSMap version 33 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 20 client.27022 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:13.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a62f80 0x561617a653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 4 client.27022 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f9ce009e5a0 con 0x561617a62f80 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.602+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a62f80 0x561617a653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9ceffff700 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a62f80 0x561617a653a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3725bbd3c0068326 server_cookie=7eba9db3a0a67c45 in_seq=0 out_seq=0 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9cd77fe700 10 client.27022 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9cd77fe700 1 -- 172.21.15.5:0/2448480004 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7f9ce009e5a0 con 0x561617a62f80 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9cd77fe700 10 client.27022 handle_command_reply: tid=0 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 client.27022 shutdown 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 msgr2=0x561617a04940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x561617a04940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561617a10dc0 tx=0x7f9ce80099d0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 shutdown_connections 2023-09-23T04:51:13.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561617a62f80 0x561617a653a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616179febd0 0x561617a611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5616179f3400 0x561617a04940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9cd77fe700 0 client.27022 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 --2- 172.21.15.5:0/2448480004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5616179f3d20 0x561617a04e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.603+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 >> 172.21.15.5:0/2448480004 conn(0x561617968260 msgr2=0x5616179fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 shutdown_connections 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9d11c3bb80 1 -- 172.21.15.5:0/2448480004 wait complete. 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9d11c3bb80 20 client.27022 trim_cache size 0 max 16384 2023-09-23T04:51:13.609 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "8a167350-2a27-4f8c-8744-63c9101ddb51": { 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8a167350-2a27-4f8c-8744-63c9101ddb51", 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:51:13.610 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 msgr2=0x7f9ce40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 0x7f9ce40a3bd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00062a0 tx=0x7f9cf0006210 comp rx=0 tx=0).stop 2023-09-23T04:51:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 msgr2=0x7f9cf81997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000e970 tx=0x7f9ce000ec80 comp rx=0 tx=0).stop 2023-09-23T04:51:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 shutdown_connections 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9ce40a1710 0x7f9ce40a3bd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9cf819dc50 0x7f9cf819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf81010d0 0x7f9cf81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 --2- 172.21.15.5:0/2991455886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9cf81061f0 0x7f9cf81997f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.604+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 >> 172.21.15.5:0/2991455886 conn(0x7f9cf80fc990 msgr2=0x7f9cf80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.605+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 shutdown_connections 2023-09-23T04:51:13.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:13.605+0000 7f9cfe3ff700 1 -- 172.21.15.5:0/2991455886 wait complete. 2023-09-23T04:51:13.622 INFO:tasks.cephfs.filesystem:scrub status for tag:8a167350-2a27-4f8c-8744-63c9101ddb51 - {'path': '/', 'tag': '8a167350-2a27-4f8c-8744-63c9101ddb51', 'options': 'recursive,force'} 2023-09-23T04:51:13.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.27 MB/sec execute 109 sec latency 6100.668 ms 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:14.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:13 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:13 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:51:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:14.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:14.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:13 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:14.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.21 MB/sec execute 110 sec latency 7100.803 ms 2023-09-23T04:51:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:14 smithi005 ceph-mon[97638]: pgmap v152: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1020 KiB/s wr, 130 op/s 2023-09-23T04:51:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:14 smithi118 ceph-mon[134297]: pgmap v152: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1020 KiB/s wr, 130 op/s 2023-09-23T04:51:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:15.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:14 smithi069 ceph-mon[133931]: pgmap v152: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1020 KiB/s wr, 130 op/s 2023-09-23T04:51:15.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.16 MB/sec execute 111 sec latency 8100.935 ms 2023-09-23T04:51:16.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.10 MB/sec execute 112 sec latency 9101.094 ms 2023-09-23T04:51:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:16 smithi005 ceph-mon[97638]: pgmap v153: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 835 KiB/s wr, 105 op/s 2023-09-23T04:51:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:16 smithi118 ceph-mon[134297]: pgmap v153: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 835 KiB/s wr, 105 op/s 2023-09-23T04:51:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:16 smithi069 ceph-mon[133931]: pgmap v153: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 835 KiB/s wr, 105 op/s 2023-09-23T04:51:17.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 6.05 MB/sec execute 113 sec latency 10101.236 ms 2023-09-23T04:51:18.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 5.99 MB/sec execute 114 sec latency 11101.414 ms 2023-09-23T04:51:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:18 smithi005 ceph-mon[97638]: pgmap v154: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 924 KiB/s wr, 115 op/s 2023-09-23T04:51:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:18 smithi118 ceph-mon[134297]: pgmap v154: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 924 KiB/s wr, 115 op/s 2023-09-23T04:51:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:18 smithi069 ceph-mon[133931]: pgmap v154: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 924 KiB/s wr, 115 op/s 2023-09-23T04:51:19.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309205 5.94 MB/sec execute 115 sec latency 12101.582 ms 2023-09-23T04:51:20.740 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:20 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:20.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:20 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:20.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:20 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:20.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 310262 5.94 MB/sec execute 116 sec latency 12505.722 ms 2023-09-23T04:51:21.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:21 smithi005 ceph-mon[97638]: pgmap v155: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 263 KiB/s wr, 24 op/s 2023-09-23T04:51:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:21 smithi118 ceph-mon[134297]: pgmap v155: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 263 KiB/s wr, 24 op/s 2023-09-23T04:51:21.943 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312400 6.00 MB/sec execute 117 sec latency 14.892 ms 2023-09-23T04:51:21.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:21 smithi069 ceph-mon[133931]: pgmap v155: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 263 KiB/s wr, 24 op/s 2023-09-23T04:51:22.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 313648 6.00 MB/sec execute 118 sec latency 15.330 ms 2023-09-23T04:51:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:22 smithi005 ceph-mon[97638]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 569 KiB/s wr, 71 op/s 2023-09-23T04:51:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:22 smithi118 ceph-mon[134297]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 569 KiB/s wr, 71 op/s 2023-09-23T04:51:23.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:22 smithi069 ceph-mon[133931]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 569 KiB/s wr, 71 op/s 2023-09-23T04:51:23.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 315684 6.05 MB/sec execute 119 sec latency 19.251 ms 2023-09-23T04:51:24.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 316763 6.03 MB/sec execute 120 sec latency 10.867 ms 2023-09-23T04:51:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:24 smithi005 ceph-mon[97638]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 404 KiB/s wr, 60 op/s 2023-09-23T04:51:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:24 smithi118 ceph-mon[134297]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 404 KiB/s wr, 60 op/s 2023-09-23T04:51:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:24 smithi069 ceph-mon[133931]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 404 KiB/s wr, 60 op/s 2023-09-23T04:51:25.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 318870 6.07 MB/sec execute 121 sec latency 27.786 ms 2023-09-23T04:51:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:25 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:26.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:25 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:25 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:26.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:25 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:25 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:25 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:26.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 319913 6.07 MB/sec execute 122 sec latency 34.783 ms 2023-09-23T04:51:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:26 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:26 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:26 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:26 smithi005 ceph-mon[97638]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 403 KiB/s wr, 58 op/s 2023-09-23T04:51:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:26 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:26 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:26 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:26 smithi118 ceph-mon[134297]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 403 KiB/s wr, 58 op/s 2023-09-23T04:51:27.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:26 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:26 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:26 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:26 smithi069 ceph-mon[133931]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 403 KiB/s wr, 58 op/s 2023-09-23T04:51:27.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 321120 6.07 MB/sec execute 123 sec latency 21.500 ms 2023-09-23T04:51:28.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323020 6.12 MB/sec execute 124 sec latency 12.453 ms 2023-09-23T04:51:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:28 smithi005 ceph-mon[97638]: pgmap v159: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1023 KiB/s wr, 139 op/s 2023-09-23T04:51:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:28 smithi118 ceph-mon[134297]: pgmap v159: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1023 KiB/s wr, 139 op/s 2023-09-23T04:51:29.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:28 smithi069 ceph-mon[133931]: pgmap v159: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 1023 KiB/s wr, 139 op/s 2023-09-23T04:51:29.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 324180 6.12 MB/sec execute 125 sec latency 17.973 ms 2023-09-23T04:51:30.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:30 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:51:30.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:30 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:51:30.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:30 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:51:30.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 325804 6.13 MB/sec execute 126 sec latency 18.711 ms 2023-09-23T04:51:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:31 smithi005 ceph-mon[97638]: pgmap v160: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 935 KiB/s wr, 128 op/s 2023-09-23T04:51:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:51:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:51:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:51:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:31 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:31 smithi118 ceph-mon[134297]: pgmap v160: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 935 KiB/s wr, 128 op/s 2023-09-23T04:51:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:51:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:51:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:51:31.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:31 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:31.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 326931 6.13 MB/sec execute 127 sec latency 17.051 ms 2023-09-23T04:51:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:31 smithi069 ceph-mon[133931]: pgmap v160: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 935 KiB/s wr, 128 op/s 2023-09-23T04:51:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:51:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:51:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:51:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:31 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:32.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 327977 6.13 MB/sec execute 128 sec latency 130.995 ms 2023-09-23T04:51:33.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:32 smithi005 ceph-mon[97638]: pgmap v161: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:51:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:32 smithi118 ceph-mon[134297]: pgmap v161: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:51:33.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:32 smithi069 ceph-mon[133931]: pgmap v161: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:51:33.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 329674 6.15 MB/sec execute 129 sec latency 60.366 ms 2023-09-23T04:51:34.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 330753 6.14 MB/sec execute 130 sec latency 19.600 ms 2023-09-23T04:51:35.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:34 smithi005 ceph-mon[97638]: pgmap v162: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2023-09-23T04:51:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:34 smithi118 ceph-mon[134297]: pgmap v162: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2023-09-23T04:51:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:34 smithi069 ceph-mon[133931]: pgmap v162: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2023-09-23T04:51:35.944 INFO:tasks.workunit.client.0.smithi005.stdout: 1 332376 6.15 MB/sec execute 131 sec latency 13.370 ms 2023-09-23T04:51:36.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 333881 6.19 MB/sec execute 132 sec latency 11.693 ms 2023-09-23T04:51:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:36 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:36 smithi005 ceph-mon[97638]: pgmap v163: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2023-09-23T04:51:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:36 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:36 smithi118 ceph-mon[134297]: pgmap v163: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2023-09-23T04:51:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:36 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:36 smithi069 ceph-mon[133931]: pgmap v163: 129 pgs: 129 active+clean; 217 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2023-09-23T04:51:37.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 335120 6.18 MB/sec execute 133 sec latency 76.316 ms 2023-09-23T04:51:38.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 335120 6.14 MB/sec execute 134 sec latency 1076.419 ms 2023-09-23T04:51:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:38 smithi005 ceph-mon[97638]: pgmap v164: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 327 op/s 2023-09-23T04:51:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:38 smithi118 ceph-mon[134297]: pgmap v164: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 327 op/s 2023-09-23T04:51:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:38 smithi069 ceph-mon[133931]: pgmap v164: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 327 op/s 2023-09-23T04:51:39.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 335120 6.09 MB/sec execute 135 sec latency 2076.563 ms 2023-09-23T04:51:40.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 336973 6.14 MB/sec execute 136 sec latency 2210.760 ms 2023-09-23T04:51:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:41 smithi005 ceph-mon[97638]: pgmap v165: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2023-09-23T04:51:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:41 smithi118 ceph-mon[134297]: pgmap v165: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2023-09-23T04:51:41.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338035 6.12 MB/sec execute 137 sec latency 20.075 ms 2023-09-23T04:51:41.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:41 smithi069 ceph-mon[133931]: pgmap v165: 129 pgs: 129 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2023-09-23T04:51:42.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338670 6.10 MB/sec execute 138 sec latency 435.847 ms 2023-09-23T04:51:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:42 smithi005 ceph-mon[97638]: pgmap v166: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 287 op/s 2023-09-23T04:51:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:42 smithi118 ceph-mon[134297]: pgmap v166: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 287 op/s 2023-09-23T04:51:43.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:42 smithi069 ceph-mon[133931]: pgmap v166: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 287 op/s 2023-09-23T04:51:43.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.843+0000 7f8d541ff700 1 -- 172.21.15.5:0/848169760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c1032d0 msgr2=0x7f8d4c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.843+0000 7f8d541ff700 1 --2- 172.21.15.5:0/848169760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1036b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009990 tx=0x7f8d48009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 -- 172.21.15.5:0/848169760 shutdown_connections 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 --2- 172.21.15.5:0/848169760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 0x7f8d4c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 --2- 172.21.15.5:0/848169760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1036b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 --2- 172.21.15.5:0/848169760 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c103c80 0x7f8d4c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 -- 172.21.15.5:0/848169760 >> 172.21.15.5:0/848169760 conn(0x7f8d4c0feb50 msgr2=0x7f8d4c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 -- 172.21.15.5:0/848169760 shutdown_connections 2023-09-23T04:51:43.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.844+0000 7f8d541ff700 1 -- 172.21.15.5:0/848169760 wait complete. 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 Processor -- start 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 -- start start 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 0x7f8d4c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19e5d0 con 0x7f8d4c19db50 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19e740 con 0x7f8d4c1083f0 2023-09-23T04:51:43.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.845+0000 7f8d541ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19e900 con 0x7f8d4c1032d0 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34200/0 (socket says 172.21.15.5:34200) 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 -- 172.21.15.5:0/434671089 learned_addr learned my addr 172.21.15.5:0/434671089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5179a700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 0x7f8d4c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 -- 172.21.15.5:0/434671089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c1032d0 msgr2=0x7f8d4c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 -- 172.21.15.5:0/434671089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 msgr2=0x7f8d4c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 0x7f8d4c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 -- 172.21.15.5:0/434671089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480095f0 con 0x7f8d4c19db50 2023-09-23T04:51:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d5279c700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d44007ee0 tx=0x7f8d4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d42ffd700 1 -- 172.21.15.5:0/434671089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44017070 con 0x7f8d4c19db50 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.846+0000 7f8d42ffd700 1 -- 172.21.15.5:0/434671089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4400f040 con 0x7f8d4c19db50 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.847+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c1a5e60 con 0x7f8d4c19db50 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.847+0000 7f8d42ffd700 1 -- 172.21.15.5:0/434671089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44005580 con 0x7f8d4c19db50 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.847+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1a6340 con 0x7f8d4c19db50 2023-09-23T04:51:43.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.850+0000 7f8d42ffd700 1 -- 172.21.15.5:0/434671089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f8d4400db20 con 0x7f8d4c19db50 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.851+0000 7f8d67be4b80 1 Processor -- start 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.851+0000 7f8d42ffd700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 0x7f8d380a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.851+0000 7f8d67be4b80 1 -- start start 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.851+0000 7f8d42ffd700 1 -- 172.21.15.5:0/434671089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8d440ccdb0 con 0x7f8d4c19db50 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d5179a700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 0x7f8d380a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5228460 con 0x5638b52bf400 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5219c20 con 0x5638b52cabd0 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5217540 con 0x5638b52bfd20 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34212/0 (socket says 172.21.15.5:34212) 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d51f9b700 1 -- 172.21.15.5:0/250263038 learned_addr learned my addr 172.21.15.5:0/250263038 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d5279c700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d5179a700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.852+0000 7f8d51f9b700 1 -- 172.21.15.5:0/250263038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 msgr2=0x5638b52ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d51f9b700 1 -- 172.21.15.5:0/250263038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 msgr2=0x5638b52ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d51f9b700 1 -- 172.21.15.5:0/250263038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b52758d0 con 0x5638b52bf400 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d5179a700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d5279c700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d5179a700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 0x7f8d380a3c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c19a7c0 tx=0x7f8d3c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015000 tx=0x7f8d480058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8fece8dc00da4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d37fff700 1 -- 172.21.15.5:0/250263038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4801d030 con 0x5638b52bf400 2023-09-23T04:51:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d37fff700 1 -- 172.21.15.5:0/250263038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4801e040 con 0x5638b52bf400 2023-09-23T04:51:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.853+0000 7f8d37fff700 1 -- 172.21.15.5:0/250263038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480049d0 con 0x5638b52bf400 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 -- 172.21.15.5:0/250263038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 msgr2=0x5638b52bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015000 tx=0x7f8d480058e0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 -- 172.21.15.5:0/250263038 shutdown_connections 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52bf7e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/250263038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 -- 172.21.15.5:0/250263038 >> 172.21.15.5:0/250263038 conn(0x5638b5234260 msgr2=0x5638b52bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 -- 172.21.15.5:0/250263038 shutdown_connections 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.854+0000 7f8d67be4b80 1 -- 172.21.15.5:0/250263038 wait complete. 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.855+0000 7f8d67be4b80 1 -- start start 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.855+0000 7f8d67be4b80 1 Processor -- start 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5212e30 con 0x5638b52cabd0 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b51af710 con 0x5638b52bf400 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b51af3b0 con 0x5638b52bfd20 2023-09-23T04:51:43.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d67be4b80 10 client.0 _ll_register_callbacks cb 0x5638b500c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41624/0 (socket says 172.21.15.5:41624) 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d51f9b700 1 -- 172.21.15.5:0/1944427533 learned_addr learned my addr 172.21.15.5:0/1944427533 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5179a700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5279c700 1 -- 172.21.15.5:0/1944427533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 msgr2=0x5638b52c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5279c700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5279c700 1 -- 172.21.15.5:0/1944427533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 msgr2=0x5638b52ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.856+0000 7f8d5279c700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d5279c700 1 -- 172.21.15.5:0/1944427533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1a6340 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d5179a700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d5279c700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c7ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ca0c0 tx=0x7f8d44095cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949e7eab10a4826b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d357fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44022250 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8d440223b0 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638b52b3250 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d44099b40 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d67be4b80 10 client.16980 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f8d4c1107e0 con 0x5638b52cabd0 2023-09-23T04:51:43.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.857+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638b52b3250 con 0x5638b52cabd0 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8d440a17e0 con 0x5638b52cabd0 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 20 client.16980 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b52da640 0x5638b52dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 4 client.16980 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638b520be10 con 0x5638b52da640 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b52da640 0x5638b52dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d357fa700 10 client.16980 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.858+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b52da640 0x5638b52dca60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f158ceda37a90c server_cookie=9b5d9de650df030a in_seq=0 out_seq=0 2023-09-23T04:51:43.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d357fa700 1 -- 172.21.15.5:0/1944427533 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638b520be10 con 0x5638b52da640 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d357fa700 10 client.16980 handle_command_reply: tid=0 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 client.16980 shutdown 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 msgr2=0x5638b52c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c7ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8d440ca0c0 tx=0x7f8d44095cd0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 shutdown_connections 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b52da640 0x5638b52dca60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c7ff0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.859+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/1944427533 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 >> 172.21.15.5:0/1944427533 conn(0x5638b5234260 msgr2=0x5638b52bb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d357fa700 0 client.16980 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:43.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 shutdown_connections 2023-09-23T04:51:43.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d67be4b80 1 -- 172.21.15.5:0/1944427533 wait complete. 2023-09-23T04:51:43.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.860+0000 7f8d67be4b80 20 client.16980 trim_cache size 0 max 16384 2023-09-23T04:51:43.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.877+0000 7f8d67be4b80 1 Processor -- start 2023-09-23T04:51:43.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.877+0000 7f8d67be4b80 1 -- start start 2023-09-23T04:51:43.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5093920 con 0x5638b52cabd0 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b52d1c00 con 0x5638b52bf400 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b52c5e90 con 0x5638b52bfd20 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34232/0 (socket says 172.21.15.5:34232) 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 -- 172.21.15.5:0/4249621239 learned_addr learned my addr 172.21.15.5:0/4249621239 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5179a700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 -- 172.21.15.5:0/4249621239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 msgr2=0x5638b52c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 -- 172.21.15.5:0/4249621239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 msgr2=0x5638b52d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d5279c700 1 -- 172.21.15.5:0/4249621239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b503dde0 con 0x5638b52cabd0 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.878+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:43.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.879+0000 7f8d5279c700 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d440991d0 tx=0x7f8d44099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb2bfe28de01484 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.879+0000 7f8d40ff9700 1 -- 172.21.15.5:0/4249621239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4409da30 con 0x5638b52cabd0 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.879+0000 7f8d40ff9700 1 -- 172.21.15.5:0/4249621239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d440b9230 con 0x5638b52cabd0 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.879+0000 7f8d40ff9700 1 -- 172.21.15.5:0/4249621239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44022250 con 0x5638b52cabd0 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- 172.21.15.5:0/4249621239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 msgr2=0x5638b52c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8d440991d0 tx=0x7f8d44099a60 comp rx=0 tx=0).stop 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- 172.21.15.5:0/4249621239 shutdown_connections 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52cabd0 0x5638b52c3170 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52bf400 0x5638b52d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/4249621239 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- 172.21.15.5:0/4249621239 >> 172.21.15.5:0/4249621239 conn(0x5638b5234260 msgr2=0x5638b52d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- 172.21.15.5:0/4249621239 shutdown_connections 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- 172.21.15.5:0/4249621239 wait complete. 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 -- start start 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.880+0000 7f8d67be4b80 1 Processor -- start 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b52d0160 con 0x5638b52bf400 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b52d02a0 con 0x5638b52cabd0 2023-09-23T04:51:43.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b52d0460 con 0x5638b52bfd20 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d67be4b80 10 client.0 _ll_register_callbacks cb 0x7f8d4409a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d51f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34246/0 (socket says 172.21.15.5:34246) 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d51f9b700 1 -- 172.21.15.5:0/932410432 learned_addr learned my addr 172.21.15.5:0/932410432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d5279c700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.881+0000 7f8d5179a700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 -- 172.21.15.5:0/932410432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 msgr2=0x5638b52cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 -- 172.21.15.5:0/932410432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 msgr2=0x5638b52cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 -- 172.21.15.5:0/932410432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1107e0 con 0x5638b52bf400 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d5279c700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d5179a700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d51f9b700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8d48006010 tx=0x7f8d48005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc8069a2334bf60 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d357fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4801d040 con 0x5638b52bf400 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d4801e040 con 0x5638b52bf400 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f8d48032e10 con 0x5638b52bf400 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d48032240 con 0x5638b52bf400 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d67be4b80 10 client.16986 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.882+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5638b5026c60 con 0x5638b52bf400 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4801e040 con 0x5638b52bf400 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8d480047d0 con 0x5638b52bf400 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 10 client.16986 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 20 client.16986 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b532f620 0x5638b5331a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 4 client.16986 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638b520bb30 con 0x5638b532f620 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.883+0000 7f8d5179a700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b532f620 0x5638b5331a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d5179a700 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b532f620 0x5638b5331a40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272ca14c29712f73 server_cookie=f01f3b67e4712c81 in_seq=0 out_seq=0 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d357fa700 10 client.16986 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d357fa700 1 -- 172.21.15.5:0/932410432 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638b520bb30 con 0x5638b532f620 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d357fa700 10 client.16986 handle_command_reply: tid=0 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 client.16986 shutdown 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 msgr2=0x5638b52d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8d48006010 tx=0x7f8d48005ee0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 shutdown_connections 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5638b532f620 0x5638b5331a40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638b52cabd0 0x5638b52cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b52bf400 0x5638b52d1360 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d357fa700 0 client.16986 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 --2- 172.21.15.5:0/932410432 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5638b52bfd20 0x5638b52cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.884+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 >> 172.21.15.5:0/932410432 conn(0x5638b5234260 msgr2=0x5638b52c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 shutdown_connections 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d67be4b80 1 -- 172.21.15.5:0/932410432 wait complete. 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d67be4b80 20 client.16986 trim_cache size 0 max 16384 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:43.892 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:51:43.893 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:51:43.893 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 msgr2=0x7f8d380a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 0x7f8d380a3c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c19a7c0 tx=0x7f8d3c009260 comp rx=0 tx=0).stop 2023-09-23T04:51:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 msgr2=0x7f8d4c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d44007ee0 tx=0x7f8d4400d3b0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 shutdown_connections 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8d380a1790 0x7f8d380a3c50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d4c19db50 0x7f8d4c19dfd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8d4c1032d0 0x7f8d4c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 --2- 172.21.15.5:0/434671089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d4c1083f0 0x7f8d4c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 >> 172.21.15.5:0/434671089 conn(0x7f8d4c0feb50 msgr2=0x7f8d4c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.885+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 shutdown_connections 2023-09-23T04:51:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:43.886+0000 7f8d541ff700 1 -- 172.21.15.5:0/434671089 wait complete. 2023-09-23T04:51:43.904 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:51:43.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:51:43.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338670 6.05 MB/sec execute 139 sec latency 1435.920 ms 2023-09-23T04:51:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 -- 172.21.15.5:0/409217751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901032a0 msgr2=0x7f5d90103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 --2- 172.21.15.5:0/409217751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901032a0 0x7f5d90103680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009990 tx=0x7f5d8c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 -- 172.21.15.5:0/409217751 shutdown_connections 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 --2- 172.21.15.5:0/409217751 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d901083c0 0x7f5d9010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 --2- 172.21.15.5:0/409217751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901032a0 0x7f5d90103680 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 --2- 172.21.15.5:0/409217751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d90103c50 0x7f5d90107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 -- 172.21.15.5:0/409217751 >> 172.21.15.5:0/409217751 conn(0x7f5d900feb20 msgr2=0x7f5d90100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 -- 172.21.15.5:0/409217751 shutdown_connections 2023-09-23T04:51:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.119+0000 7f5d97fff700 1 -- 172.21.15.5:0/409217751 wait complete. 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.120+0000 7f5d97fff700 1 Processor -- start 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 -- start start 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 0x7f5d901996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d9019db10 0x7f5d9019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019e590 con 0x7f5d901032a0 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019e700 con 0x7f5d901083c0 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d97fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019e8c0 con 0x7f5d9019db10 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d95d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d95d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34258/0 (socket says 172.21.15.5:34258) 2023-09-23T04:51:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d95d9b700 1 -- 172.21.15.5:0/3896083539 learned_addr learned my addr 172.21.15.5:0/3896083539 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d9559a700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 0x7f5d901996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.121+0000 7f5d9659c700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d9019db10 0x7f5d9019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 -- 172.21.15.5:0/3896083539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d9019db10 msgr2=0x7f5d9019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d9019db10 0x7f5d9019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 -- 172.21.15.5:0/3896083539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 msgr2=0x7f5d901996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 0x7f5d901996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 -- 172.21.15.5:0/3896083539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0095f0 con 0x7f5d901032a0 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d9559a700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 0x7f5d901996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d95d9b700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005910 tx=0x7f5d8c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c01d070 con 0x7f5d901032a0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8c021410 con 0x7f5d901032a0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d901a5dc0 con 0x7f5d901032a0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.122+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c005720 con 0x7f5d901032a0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.123+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d901a61e0 con 0x7f5d901032a0 2023-09-23T04:51:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.124+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d900680f0 con 0x7f5d901032a0 2023-09-23T04:51:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.126+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f5d8c021580 con 0x7f5d901032a0 2023-09-23T04:51:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.127+0000 7f5d82ffd700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 0x7f5d7c0a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.127+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f5d8c0ce930 con 0x7f5d901032a0 2023-09-23T04:51:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.127+0000 7f5d9559a700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 0x7f5d7c0a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.128+0000 7f5d9559a700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 0x7f5d7c0a3c10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d9019a780 tx=0x7f5d84009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.131+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f5d8c097830 con 0x7f5d901032a0 2023-09-23T04:51:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.404+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d9019a4c0 con 0x7f5d901032a0 2023-09-23T04:51:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.405+0000 7f5d82ffd700 1 -- 172.21.15.5:0/3896083539 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f5d8c09ae50 con 0x7f5d901032a0 2023-09-23T04:51:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:51:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:51:35.313575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:51:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 msgr2=0x7f5d7c0a3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 0x7f5d7c0a3c10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d9019a780 tx=0x7f5d84009260 comp rx=0 tx=0).stop 2023-09-23T04:51:44.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 msgr2=0x7f5d90199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005910 tx=0x7f5d8c009580 comp rx=0 tx=0).stop 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 shutdown_connections 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f5d7c0a1750 0x7f5d7c0a3c10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f5d9019db10 0x7f5d9019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d901032a0 0x7f5d90199170 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 --2- 172.21.15.5:0/3896083539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d901083c0 0x7f5d901996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 >> 172.21.15.5:0/3896083539 conn(0x7f5d900feb20 msgr2=0x7f5d90106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.407+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 shutdown_connections 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.408+0000 7f5d97fff700 1 -- 172.21.15.5:0/3896083539 wait complete. 2023-09-23T04:51:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2023-09-23T04:51:44.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:51:44.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 -- 172.21.15.5:0/619326981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 msgr2=0x7ffac810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/619326981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac810d160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffab00098f0 tx=0x7ffab0009c00 comp rx=0 tx=0).stop 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 -- 172.21.15.5:0/619326981 shutdown_connections 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/619326981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/619326981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 0x7ffac8073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/619326981 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac810d160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 -- 172.21.15.5:0/619326981 >> 172.21.15.5:0/619326981 conn(0x7ffac80fc990 msgr2=0x7ffac80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 -- 172.21.15.5:0/619326981 shutdown_connections 2023-09-23T04:51:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.673+0000 7ffacd3ff700 1 -- 172.21.15.5:0/619326981 wait complete. 2023-09-23T04:51:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.675+0000 7ffacd3ff700 1 Processor -- start 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.675+0000 7ffacd3ff700 1 -- start start 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.675+0000 7ffacd3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 0x7ffac81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.675+0000 7ffacd3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffacd3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffacd3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac819e1c0 con 0x7ffac810d6a0 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffacd3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac819e330 con 0x7ffac8073250 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffacd3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac819e4f0 con 0x7ffac8073b70 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffabffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffabffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33100/0 (socket says 172.21.15.5:33100) 2023-09-23T04:51:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffabffff700 1 -- 172.21.15.5:0/1050012759 learned_addr learned my addr 172.21.15.5:0/1050012759 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 0x7ffac81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 -- 172.21.15.5:0/1050012759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 msgr2=0x7ffac8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 -- 172.21.15.5:0/1050012759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 msgr2=0x7ffac81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 0x7ffac81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 -- 172.21.15.5:0/1050012759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0009d00 con 0x7ffac810d6a0 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.676+0000 7ffac77fe700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac819db80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009cd0 tx=0x7ffac000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.677+0000 7ffac4ff9700 1 -- 172.21.15.5:0/1050012759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0017070 con 0x7ffac810d6a0 2023-09-23T04:51:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.677+0000 7ffac4ff9700 1 -- 172.21.15.5:0/1050012759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac000f040 con 0x7ffac810d6a0 2023-09-23T04:51:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.677+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab0009520 con 0x7ffac810d6a0 2023-09-23T04:51:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.677+0000 7ffac4ff9700 1 -- 172.21.15.5:0/1050012759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000e050 con 0x7ffac810d6a0 2023-09-23T04:51:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.677+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81a5e30 con 0x7ffac810d6a0 2023-09-23T04:51:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.680+0000 7ffac4ff9700 1 -- 172.21.15.5:0/1050012759 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ffac000cd10 con 0x7ffac810d6a0 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.681+0000 7ffac4ff9700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 0x7ffab40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.681+0000 7ffae0cc1b80 1 Processor -- start 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffac4ff9700 1 -- 172.21.15.5:0/1050012759 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffac00ccff0 con 0x7ffac810d6a0 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 -- start start 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 0x7ffab40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf82e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf83bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf797460 con 0x55c2cf82ed20 2023-09-23T04:51:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf788c20 con 0x55c2cf82e400 2023-09-23T04:51:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf786540 con 0x55c2cf839bd0 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffabffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffac6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf82e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffabffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34278/0 (socket says 172.21.15.5:34278) 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffabffff700 1 -- 172.21.15.5:0/917415390 learned_addr learned my addr 172.21.15.5:0/917415390 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.682+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 0x7ffab40a3bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009cb0 tx=0x7ffab8009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac77fe700 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf83bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 -- 172.21.15.5:0/917415390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 msgr2=0x55c2cf83bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf83bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 -- 172.21.15.5:0/917415390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 msgr2=0x55c2cf839690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 -- 172.21.15.5:0/917415390 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2cf7e48d0 con 0x55c2cf82e400 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffabffff700 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf82e7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffab8012d20 tx=0x7ffab8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5defb872d2dd36a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffabd7fa700 1 -- 172.21.15.5:0/917415390 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab801cd80 con 0x55c2cf82e400 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffabd7fa700 1 -- 172.21.15.5:0/917415390 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab8012e10 con 0x55c2cf82e400 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.683+0000 7ffabd7fa700 1 -- 172.21.15.5:0/917415390 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8005610 con 0x55c2cf82e400 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/917415390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 msgr2=0x55c2cf82e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf82e7e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffab8012d20 tx=0x7ffab8014000 comp rx=0 tx=0).stop 2023-09-23T04:51:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/917415390 shutdown_connections 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf83bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf82e7e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/917415390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf839690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/917415390 >> 172.21.15.5:0/917415390 conn(0x55c2cf7a3260 msgr2=0x55c2cf82a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/917415390 shutdown_connections 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.684+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/917415390 wait complete. 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.685+0000 7ffae0cc1b80 1 -- start start 2023-09-23T04:51:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.685+0000 7ffae0cc1b80 1 Processor -- start 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf839610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf781e30 con 0x55c2cf82ed20 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf71e710 con 0x55c2cf82e400 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf71e3b0 con 0x55c2cf839bd0 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffae0cc1b80 10 client.0 _ll_register_callbacks cb 0x55c2cf57b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34292/0 (socket says 172.21.15.5:34292) 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 -- 172.21.15.5:0/3895535624 learned_addr learned my addr 172.21.15.5:0/3895535624 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf839610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffac77fe700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 -- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 msgr2=0x55c2cf836fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 -- 172.21.15.5:0/3895535624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 msgr2=0x55c2cf839610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf839610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffabffff700 1 -- 172.21.15.5:0/3895535624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81107d0 con 0x55c2cf82ed20 2023-09-23T04:51:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.686+0000 7ffac77fe700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffabffff700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b790 tx=0x7ffab000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20f8a18cc2c2e99 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffaae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab000bc20 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2cf822250 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffab0005240 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab0005400 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffae0cc1b80 10 client.17004 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.687+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7ffab8017050 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2cf822250 con 0x55c2cf82ed20 2023-09-23T04:51:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ffab0015020 con 0x55c2cf82ed20 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 10 client.17004 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 10 client.17004 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 20 client.17004 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf8497d0 0x55c2cf84bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 4 client.17004 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2cf77ae10 con 0x55c2cf8497d0 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.688+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf8497d0 0x55c2cf84bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.689+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf8497d0 0x55c2cf84bbf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4780d45017e39f11 server_cookie=e143c5b44bc5fb3b in_seq=0 out_seq=0 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.689+0000 7ffaae7fc700 10 client.17004 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:44.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffaae7fc700 1 -- 172.21.15.5:0/3895535624 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2cf77ae10 con 0x55c2cf8497d0 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffaae7fc700 10 client.17004 handle_command_reply: tid=0 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 client.17004 shutdown 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 msgr2=0x55c2cf836890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b790 tx=0x7ffab000b7c0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 shutdown_connections 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf839610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf8497d0 0x55c2cf84bbf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/3895535624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82ed20 0x55c2cf836890 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 >> 172.21.15.5:0/3895535624 conn(0x55c2cf7a3260 msgr2=0x55c2cf82a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffaae7fc700 0 client.17004 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 shutdown_connections 2023-09-23T04:51:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.690+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/3895535624 wait complete. 2023-09-23T04:51:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.691+0000 7ffae0cc1b80 20 client.17004 trim_cache size 0 max 16384 2023-09-23T04:51:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.704+0000 7ffae0cc1b80 1 Processor -- start 2023-09-23T04:51:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.704+0000 7ffae0cc1b80 1 -- start start 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 0x55c2cf8320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf602920 con 0x55c2cf82e400 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf8379a0 con 0x55c2cf82ed20 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf834df0 con 0x55c2cf839bd0 2023-09-23T04:51:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34306/0 (socket says 172.21.15.5:34306) 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 -- 172.21.15.5:0/11838136 learned_addr learned my addr 172.21.15.5:0/11838136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffabffff700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 0x55c2cf8320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac77fe700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 -- 172.21.15.5:0/11838136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 msgr2=0x55c2cf836050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 -- 172.21.15.5:0/11838136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 msgr2=0x55c2cf8320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 0x55c2cf8320d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac6ffd700 1 -- 172.21.15.5:0/11838136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2cf5acde0 con 0x55c2cf82e400 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffac77fe700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.705+0000 7ffabffff700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 0x55c2cf8320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.706+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffab801c9d0 tx=0x7ffab8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32e0c0f3b7ddde8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.706+0000 7ffabdffb700 1 -- 172.21.15.5:0/11838136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab801cc40 con 0x55c2cf82e400 2023-09-23T04:51:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.706+0000 7ffabdffb700 1 -- 172.21.15.5:0/11838136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab8037160 con 0x55c2cf82e400 2023-09-23T04:51:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.706+0000 7ffabdffb700 1 -- 172.21.15.5:0/11838136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80046b0 con 0x55c2cf82e400 2023-09-23T04:51:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/11838136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 msgr2=0x55c2cf831b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffab801c9d0 tx=0x7ffab8009710 comp rx=0 tx=0).stop 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/11838136 shutdown_connections 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf839bd0 0x55c2cf836050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf82e400 0x55c2cf831b90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/11838136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82ed20 0x55c2cf8320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/11838136 >> 172.21.15.5:0/11838136 conn(0x55c2cf7a3260 msgr2=0x55c2cf844e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/11838136 shutdown_connections 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/11838136 wait complete. 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 -- start start 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.707+0000 7ffae0cc1b80 1 Processor -- start 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf82ed20 0x55c2cf83ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 0x55c2cf89c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf89cb00 con 0x55c2cf839bd0 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf89cc40 con 0x55c2cf82e400 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2cf89ce00 con 0x55c2cf82ed20 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffae0cc1b80 10 client.0 _ll_register_callbacks cb 0x7ffab000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 0x55c2cf89c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffac6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffac6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41712/0 (socket says 172.21.15.5:41712) 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffac6ffd700 1 -- 172.21.15.5:0/2169850562 learned_addr learned my addr 172.21.15.5:0/2169850562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.708+0000 7ffabffff700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf82ed20 0x55c2cf83ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 -- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf82ed20 msgr2=0x55c2cf83ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf82ed20 0x55c2cf83ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 -- 172.21.15.5:0/2169850562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 msgr2=0x55c2cf89c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 0x55c2cf89c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 -- 172.21.15.5:0/2169850562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab802a040 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac77fe700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 0x55c2cf89c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffac6ffd700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009dc0 tx=0x7ffab8005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e2930f50c2eb39 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffaae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80372d0 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffab8012e10 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.709+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7ffab8037430 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab8005760 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 10 client.27011 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c2cf595c60 con 0x55c2cf82e400 2023-09-23T04:51:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffab8012e10 con 0x55c2cf82e400 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ffab8034c00 con 0x55c2cf82e400 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 10 client.27011 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 10 client.27011 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 20 client.27011 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf89e140 0x55c2cf8a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 4 client.27011 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2cf77ab30 con 0x55c2cf89e140 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.710+0000 7ffabffff700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf89e140 0x55c2cf8a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffabffff700 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf89e140 0x55c2cf8a0530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df4f4c4620f9e45 server_cookie=cd1bcf990548b01b in_seq=0 out_seq=0 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffaae7fc700 10 client.27011 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffaae7fc700 1 -- 172.21.15.5:0/2169850562 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2cf77ab30 con 0x55c2cf89e140 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffaae7fc700 10 client.27011 handle_command_reply: tid=0 2023-09-23T04:51:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 client.27011 shutdown 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 msgr2=0x55c2cf83e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009dc0 tx=0x7ffab8005540 comp rx=0 tx=0).stop 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 shutdown_connections 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x55c2cf89e140 0x55c2cf8a0530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2cf839bd0 0x55c2cf89c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2cf82e400 0x55c2cf83e8f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffaae7fc700 0 client.27011 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 --2- 172.21.15.5:0/2169850562 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c2cf82ed20 0x55c2cf83ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 >> 172.21.15.5:0/2169850562 conn(0x55c2cf7a3260 msgr2=0x55c2cf831b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 shutdown_connections 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.711+0000 7ffae0cc1b80 1 -- 172.21.15.5:0/2169850562 wait complete. 2023-09-23T04:51:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffae0cc1b80 20 client.27011 trim_cache size 0 max 16384 2023-09-23T04:51:44.718 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 msgr2=0x7ffab40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 0x7ffab40a3bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009cb0 tx=0x7ffab8009260 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 msgr2=0x7ffac819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac819db80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009cd0 tx=0x7ffac000c620 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 shutdown_connections 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ffab40a1710 0x7ffab40a3bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffac810d6a0 0x7ffac819db80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffac8073250 0x7ffac81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 --2- 172.21.15.5:0/1050012759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ffac8073b70 0x7ffac8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 >> 172.21.15.5:0/1050012759 conn(0x7ffac80fc990 msgr2=0x7ffac810f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 shutdown_connections 2023-09-23T04:51:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.712+0000 7ffacd3ff700 1 -- 172.21.15.5:0/1050012759 wait complete. 2023-09-23T04:51:44.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:44 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3896083539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:44.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:51:44.740 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:44 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3896083539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:44.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:44 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3896083539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:51:44.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 -- 172.21.15.5:0/4109364005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda00101a80 msgr2=0x7fda00105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 --2- 172.21.15.5:0/4109364005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda00101a80 0x7fda00105ac0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009990 tx=0x7fd9f4009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 -- 172.21.15.5:0/4109364005 shutdown_connections 2023-09-23T04:51:44.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 --2- 172.21.15.5:0/4109364005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda001061f0 0x7fda0010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 --2- 172.21.15.5:0/4109364005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 0x7fda001014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 --2- 172.21.15.5:0/4109364005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda00101a80 0x7fda00105ac0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 -- 172.21.15.5:0/4109364005 >> 172.21.15.5:0/4109364005 conn(0x7fda000fc990 msgr2=0x7fda000fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 -- 172.21.15.5:0/4109364005 shutdown_connections 2023-09-23T04:51:44.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.938+0000 7fda04bff700 1 -- 172.21.15.5:0/4109364005 wait complete. 2023-09-23T04:51:44.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 Processor -- start 2023-09-23T04:51:44.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 -- start start 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 0x7fda001991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 0x7fda0019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0019e560 con 0x7fda001061f0 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0019e6d0 con 0x7fda001010d0 2023-09-23T04:51:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.939+0000 7fda04bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0019e890 con 0x7fda0019dae0 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34332/0 (socket says 172.21.15.5:34332) 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/4278379808 learned_addr learned my addr 172.21.15.5:0/4278379808 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 0x7fda0019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 0x7fda001991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/4278379808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 msgr2=0x7fda0019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 0x7fda0019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/4278379808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 msgr2=0x7fda001991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 0x7fda001991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/4278379808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0009520 con 0x7fda001061f0 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 0x7fda0019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.940+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.941+0000 7fd9ef7fe700 1 -- 172.21.15.5:0/4278379808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f401d070 con 0x7fda001061f0 2023-09-23T04:51:44.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.941+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f40095f0 con 0x7fda001061f0 2023-09-23T04:51:44.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.941+0000 7fd9ef7fe700 1 -- 172.21.15.5:0/4278379808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f4021410 con 0x7fda001061f0 2023-09-23T04:51:44.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.941+0000 7fd9ef7fe700 1 -- 172.21.15.5:0/4278379808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4017650 con 0x7fda001061f0 2023-09-23T04:51:44.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.941+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001a6020 con 0x7fda001061f0 2023-09-23T04:51:44.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.944+0000 7fd9ef7fe700 1 -- 172.21.15.5:0/4278379808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd9f4021580 con 0x7fda001061f0 2023-09-23T04:51:44.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338670 6.01 MB/sec execute 140 sec latency 2435.968 ms 2023-09-23T04:51:44.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.945+0000 7fd9ef7fe700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 0x7fd9e80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.945+0000 7fda18606b80 1 Processor -- start 2023-09-23T04:51:44.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.945+0000 7fd9ef7fe700 1 -- 172.21.15.5:0/4278379808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd9f40ce960 con 0x7fda001061f0 2023-09-23T04:51:44.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.945+0000 7fda18606b80 1 -- start start 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 0x7fd9e80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f443460 con 0x55c90f4da400 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f434c20 con 0x55c90f4dad20 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fda18606b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f432540 con 0x55c90f4e5bd0 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41746/0 (socket says 172.21.15.5:41746) 2023-09-23T04:51:44.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3130560762 learned_addr learned my addr 172.21.15.5:0/3130560762 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.946+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3130560762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4e5bd0 msgr2=0x55c90f4e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3130560762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 msgr2=0x55c90f4da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3130560762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90f4908d0 con 0x55c90f4dad20 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 0x7fd9e80a3bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0013680 tx=0x7fd9f0013a40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4097f90 tx=0x7fd9f409b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09a7f7c9be851d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9ecff9700 1 -- 172.21.15.5:0/3130560762 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4097040 con 0x55c90f4dad20 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9ecff9700 1 -- 172.21.15.5:0/3130560762 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f409bc20 con 0x55c90f4dad20 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.947+0000 7fd9ecff9700 1 -- 172.21.15.5:0/3130560762 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f409b780 con 0x55c90f4dad20 2023-09-23T04:51:44.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 -- 172.21.15.5:0/3130560762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 msgr2=0x55c90f4e5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4097f90 tx=0x7fd9f409b370 comp rx=0 tx=0).stop 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 -- 172.21.15.5:0/3130560762 shutdown_connections 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 --2- 172.21.15.5:0/3130560762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4e5690 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 -- 172.21.15.5:0/3130560762 >> 172.21.15.5:0/3130560762 conn(0x55c90f44f260 msgr2=0x55c90f4d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 -- 172.21.15.5:0/3130560762 shutdown_connections 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.948+0000 7fda18606b80 1 -- 172.21.15.5:0/3130560762 wait complete. 2023-09-23T04:51:44.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 -- start start 2023-09-23T04:51:44.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 Processor -- start 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f42de30 con 0x55c90f4da400 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f3ca710 con 0x55c90f4e5bd0 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f3ca3b0 con 0x55c90f4dad20 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.949+0000 7fda18606b80 10 client.0 _ll_register_callbacks cb 0x55c90f227800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34356/0 (socket says 172.21.15.5:34356) 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41758/0 (socket says 172.21.15.5:41758) 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fe59c700 1 -- 172.21.15.5:0/4150874324 learned_addr learned my addr 172.21.15.5:0/4150874324 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fdd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33160/0 (socket says 172.21.15.5:33160) 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 -- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 msgr2=0x55c90f4e64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 -- 172.21.15.5:0/4150874324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 msgr2=0x55c90f4e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 -- 172.21.15.5:0/4150874324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001a6020 con 0x55c90f4e5bd0 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80042f0 tx=0x7fd9f8004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d5c0722947b0bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.950+0000 7fd9e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8004bb0 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c90f4ce250 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd9f8004d10 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f8007500 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 10 client.27023 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c90f258de0 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c90f4ce250 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd9f800e640 con 0x55c90f4e5bd0 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 10 client.27023 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 10 client.27023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 20 client.27023 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f4f5340 0x55c90f4f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 4 client.27023 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.951+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c90f426e10 con 0x55c90f4f5340 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.952+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f4f5340 0x55c90f4f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.952+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f4f5340 0x55c90f4f7760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5132389a2b7c8466 server_cookie=d108d0e8ef770542 in_seq=0 out_seq=0 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.952+0000 7fd9e5ffb700 10 client.27023 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/4150874324 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c90f426e10 con 0x55c90f4f5340 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fd9e5ffb700 10 client.27023 handle_command_reply: tid=0 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 client.27023 shutdown 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 msgr2=0x55c90f4e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80042f0 tx=0x7fd9f8004320 comp rx=0 tx=0).stop 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 shutdown_connections 2023-09-23T04:51:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4e5bd0 0x55c90f4e6a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f4f5340 0x55c90f4f7760 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4da400 0x55c90f4e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 --2- 172.21.15.5:0/4150874324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fd9e5ffb700 0 client.27023 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 >> 172.21.15.5:0/4150874324 conn(0x55c90f44f260 msgr2=0x55c90f4d6380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 shutdown_connections 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.953+0000 7fda18606b80 1 -- 172.21.15.5:0/4150874324 wait complete. 2023-09-23T04:51:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.954+0000 7fda18606b80 20 client.27023 trim_cache size 0 max 16384 2023-09-23T04:51:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 Processor -- start 2023-09-23T04:51:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 -- start start 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4da400 0x55c90f4e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4df5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f2ae920 con 0x55c90f4e5bd0 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f4dfb20 con 0x55c90f4dad20 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fda18606b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f4db360 con 0x55c90f4da400 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.967+0000 7fd9fed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4da400 0x55c90f4e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34360/0 (socket says 172.21.15.5:34360) 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fed9d700 1 -- 172.21.15.5:0/3397632260 learned_addr learned my addr 172.21.15.5:0/3397632260 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4df5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3397632260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4da400 msgr2=0x55c90f4e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4da400 0x55c90f4e9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3397632260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 msgr2=0x55c90f4f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 -- 172.21.15.5:0/3397632260 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90f4e5880 con 0x55c90f4dad20 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4df5e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40a32f0 tx=0x7fd9f4021c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8efa17c7ce789604 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9ed7fa700 1 -- 172.21.15.5:0/3397632260 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4097040 con 0x55c90f4dad20 2023-09-23T04:51:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9ed7fa700 1 -- 172.21.15.5:0/3397632260 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f40a3820 con 0x55c90f4dad20 2023-09-23T04:51:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.968+0000 7fd9ed7fa700 1 -- 172.21.15.5:0/3397632260 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f409b9e0 con 0x55c90f4dad20 2023-09-23T04:51:44.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- 172.21.15.5:0/3397632260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 msgr2=0x55c90f4df5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4df5e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40a32f0 tx=0x7fd9f4021c60 comp rx=0 tx=0).stop 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- 172.21.15.5:0/3397632260 shutdown_connections 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f4f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4da400 0x55c90f4e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 --2- 172.21.15.5:0/3397632260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4dad20 0x55c90f4df5e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- 172.21.15.5:0/3397632260 >> 172.21.15.5:0/3397632260 conn(0x55c90f44f260 msgr2=0x55c90f4fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- 172.21.15.5:0/3397632260 shutdown_connections 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- 172.21.15.5:0/3397632260 wait complete. 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.969+0000 7fda18606b80 1 -- start start 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 Processor -- start 2023-09-23T04:51:44.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f548170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f4e84f0 con 0x55c90f4e5bd0 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f4e8660 con 0x55c90f4da400 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90f5486b0 con 0x55c90f4dad20 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fda18606b80 10 client.0 _ll_register_callbacks cb 0x7fd9f8003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fd9fe59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41768/0 (socket says 172.21.15.5:41768) 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fd9fe59c700 1 -- 172.21.15.5:0/538423423 learned_addr learned my addr 172.21.15.5:0/538423423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.970+0000 7fd9fed9d700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f548170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 -- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 msgr2=0x55c90f4e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 -- 172.21.15.5:0/538423423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 msgr2=0x55c90f548170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f548170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 -- 172.21.15.5:0/538423423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c90f258de0 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9fe59c700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55c90f4f8070 tx=0x7fd9f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c5762cc28e8c89 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0026040 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f409bc20 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.971+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd9f0029290 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f0018040 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 10 client.27029 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c90f4908d0 con 0x55c90f4da400 2023-09-23T04:51:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f409bc20 con 0x55c90f4da400 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd9f000f480 con 0x55c90f4da400 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 10 client.27029 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 10 client.27029 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 20 client.27029 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.972+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f549e90 0x55c90f54c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fda18606b80 4 client.27029 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c90f426b30 con 0x55c90f549e90 2023-09-23T04:51:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f549e90 0x55c90f54c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:44.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fd9fdd9b700 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f549e90 0x55c90f54c2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f87c21f1d94476f9 server_cookie=d4ed934c1f0fe2de in_seq=0 out_seq=0 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fd9e5ffb700 10 client.27029 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fd9e5ffb700 1 -- 172.21.15.5:0/538423423 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c90f426b30 con 0x55c90f549e90 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fd9e5ffb700 10 client.27029 handle_command_reply: tid=0 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.973+0000 7fda18606b80 1 client.27029 shutdown 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 msgr2=0x55c90f4e7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55c90f4f8070 tx=0x7fd9f00047c0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 shutdown_connections 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55c90f549e90 0x55c90f54c2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c90f4e5bd0 0x55c90f548170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c90f4da400 0x55c90f4e7810 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fd9e5ffb700 0 client.27029 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:51:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 --2- 172.21.15.5:0/538423423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c90f4dad20 0x55c90f4e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 >> 172.21.15.5:0/538423423 conn(0x55c90f44f260 msgr2=0x55c90f4e0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 shutdown_connections 2023-09-23T04:51:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 1 -- 172.21.15.5:0/538423423 wait complete. 2023-09-23T04:51:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda18606b80 20 client.27029 trim_cache size 0 max 16384 2023-09-23T04:51:44.980 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 msgr2=0x7fd9e80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 0x7fd9e80a3bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0013680 tx=0x7fd9f0013a40 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 msgr2=0x7fda001996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.974+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4000c00 tx=0x7fd9f4005740 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 shutdown_connections 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9e80a1710 0x7fd9e80a3bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fda0019dae0 0x7fda0019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda001010d0 0x7fda001991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 --2- 172.21.15.5:0/4278379808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda001061f0 0x7fda001996f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 >> 172.21.15.5:0/4278379808 conn(0x7fda000fc990 msgr2=0x7fda000fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 shutdown_connections 2023-09-23T04:51:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:44.975+0000 7fda04bff700 1 -- 172.21.15.5:0/4278379808 wait complete. 2023-09-23T04:51:44.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:51:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 -- 172.21.15.5:0/1007764885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09141029e0 msgr2=0x7f091410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 --2- 172.21.15.5:0/1007764885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09141029e0 0x7f091410aec0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009990 tx=0x7f08fc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 -- 172.21.15.5:0/1007764885 shutdown_connections 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 --2- 172.21.15.5:0/1007764885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f091410b490 0x7f091410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 --2- 172.21.15.5:0/1007764885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141020c0 0x7f09141024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 --2- 172.21.15.5:0/1007764885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09141029e0 0x7f091410aec0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 -- 172.21.15.5:0/1007764885 >> 172.21.15.5:0/1007764885 conn(0x7f0914076f30 msgr2=0x7f0914077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 -- 172.21.15.5:0/1007764885 shutdown_connections 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.200+0000 7f0918bff700 1 -- 172.21.15.5:0/1007764885 wait complete. 2023-09-23T04:51:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.202+0000 7f0918bff700 1 Processor -- start 2023-09-23T04:51:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.202+0000 7f0918bff700 1 -- start start 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.202+0000 7f0918bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.202+0000 7f0918bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141029e0 0x7f09141975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.202+0000 7f0918bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f091410b490 0x7f091419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f0918bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091419c050 con 0x7f091410b490 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f0918bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091419c1c0 con 0x7f09141029e0 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f0918bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091419c380 con 0x7f09141020c0 2023-09-23T04:51:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33188/0 (socket says 172.21.15.5:33188) 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f091410b490 0x7f091419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 -- 172.21.15.5:0/2873394013 learned_addr learned my addr 172.21.15.5:0/2873394013 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f090bfff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141029e0 0x7f09141975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 -- 172.21.15.5:0/2873394013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141029e0 msgr2=0x7f09141975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141029e0 0x7f09141975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 -- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f091410b490 msgr2=0x7f091419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f091410b490 0x7f091419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 -- 172.21.15.5:0/2873394013 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904009520 con 0x7f09141020c0 2023-09-23T04:51:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.203+0000 7f091259c700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f090400eb30 tx=0x7f090400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.204+0000 7f090b7fe700 1 -- 172.21.15.5:0/2873394013 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904009880 con 0x7f09141020c0 2023-09-23T04:51:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.204+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc0095f0 con 0x7f09141020c0 2023-09-23T04:51:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.204+0000 7f090b7fe700 1 -- 172.21.15.5:0/2873394013 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09040044e0 con 0x7f09141020c0 2023-09-23T04:51:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.204+0000 7f090b7fe700 1 -- 172.21.15.5:0/2873394013 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003d80 con 0x7f09141020c0 2023-09-23T04:51:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.204+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091419c850 con 0x7f09141020c0 2023-09-23T04:51:45.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.207+0000 7f090b7fe700 1 -- 172.21.15.5:0/2873394013 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f0904003f20 con 0x7f09141020c0 2023-09-23T04:51:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.208+0000 7f092c51bb80 1 Processor -- start 2023-09-23T04:51:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.208+0000 7f092c51bb80 1 -- start start 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1edfd20 0x5620c1eea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1e48460 con 0x5620c1eeabd0 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1e39c20 con 0x5620c1edfd20 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f092c51bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1e37540 con 0x5620c1edf400 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f090bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33190/0 (socket says 172.21.15.5:33190) 2023-09-23T04:51:45.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 -- 172.21.15.5:0/52714687 learned_addr learned my addr 172.21.15.5:0/52714687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f090bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34384/0 (socket says 172.21.15.5:34384) 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f091259c700 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1edfd20 0x5620c1eea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 -- 172.21.15.5:0/52714687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1edfd20 msgr2=0x5620c1eea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1edfd20 0x5620c1eea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 -- 172.21.15.5:0/52714687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 msgr2=0x5620c1eecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f0912d9d700 1 -- 172.21.15.5:0/52714687 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c1e958d0 con 0x5620c1edf400 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.209+0000 7f090bfff700 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f0912d9d700 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f090c003fa0 tx=0x7f090c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0feab0dfb641ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f090b7fe700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 0x7f09000a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f0908ff9700 1 -- 172.21.15.5:0/52714687 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c019070 con 0x5620c1edf400 2023-09-23T04:51:45.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f0908ff9700 1 -- 172.21.15.5:0/52714687 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090c00f040 con 0x5620c1edf400 2023-09-23T04:51:45.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f0908ff9700 1 -- 172.21.15.5:0/52714687 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c007a00 con 0x5620c1edf400 2023-09-23T04:51:45.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f090b7fe700 1 -- 172.21.15.5:0/2873394013 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f0904014070 con 0x7f09141020c0 2023-09-23T04:51:45.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.210+0000 7f091259c700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 0x7f09000a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- 172.21.15.5:0/52714687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 msgr2=0x5620c1edf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f090c003fa0 tx=0x7f090c009ba0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f091259c700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 0x7f09000a3bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09040a25e0 tx=0x7f09040d14a0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- 172.21.15.5:0/52714687 shutdown_connections 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1eeabd0 0x5620c1eecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1edf7e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 --2- 172.21.15.5:0/52714687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1edfd20 0x5620c1eea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- 172.21.15.5:0/52714687 >> 172.21.15.5:0/52714687 conn(0x5620c1e54260 msgr2=0x5620c1edb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- 172.21.15.5:0/52714687 shutdown_connections 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- 172.21.15.5:0/52714687 wait complete. 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.211+0000 7f092c51bb80 1 -- start start 2023-09-23T04:51:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 Processor -- start 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1e32e30 con 0x5620c1edfd20 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1dcf710 con 0x5620c1eeabd0 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1dcf3b0 con 0x5620c1edf400 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f092c51bb80 10 client.0 _ll_register_callbacks cb 0x5620c1c2c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.212+0000 7f090bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33196/0 (socket says 172.21.15.5:33196) 2023-09-23T04:51:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f0912d9d700 1 -- 172.21.15.5:0/1355641904 learned_addr learned my addr 172.21.15.5:0/1355641904 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34390/0 (socket says 172.21.15.5:34390) 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f0912d9d700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 -- 172.21.15.5:0/1355641904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 msgr2=0x5620c1eea5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 -- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 msgr2=0x5620c1ee7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 -- 172.21.15.5:0/1355641904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091410e5d0 con 0x5620c1edfd20 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f0912d9d700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f090bfff700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b2e0 tx=0x7f08fc00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8e46aa594d2089 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.213+0000 7f08f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc01d070 con 0x5620c1edfd20 2023-09-23T04:51:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620c1ed3250 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f08fc004500 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08fc0046c0 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 10 client.17040 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f090c003bd0 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620c1ed3250 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f08fc00f7d0 con 0x5620c1edfd20 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 10 client.17040 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.214+0000 7f092c51bb80 20 client.17040 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1efa600 0x5620c1efca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f092c51bb80 4 client.17040 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620c1e2be10 con 0x5620c1efa600 2023-09-23T04:51:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f0912d9d700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1efa600 0x5620c1efca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f0912d9d700 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1efa600 0x5620c1efca20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60cfd701d71b967b server_cookie=9d51e3ae61339cb in_seq=0 out_seq=0 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.215+0000 7f08f9ffb700 10 client.17040 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f08f9ffb700 1 -- 172.21.15.5:0/1355641904 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620c1e2be10 con 0x5620c1efa600 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f08f9ffb700 10 client.17040 handle_command_reply: tid=0 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 client.17040 shutdown 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 msgr2=0x5620c1ee7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b2e0 tx=0x7f08fc00b610 comp rx=0 tx=0).stop 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 shutdown_connections 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1efa600 0x5620c1efca20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edf400 0x5620c1eea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1355641904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edfd20 0x5620c1ee7870 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f08f9ffb700 0 client.17040 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.216+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 >> 172.21.15.5:0/1355641904 conn(0x5620c1e54260 msgr2=0x5620c1edb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.217+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 shutdown_connections 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.217+0000 7f092c51bb80 1 -- 172.21.15.5:0/1355641904 wait complete. 2023-09-23T04:51:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.217+0000 7f092c51bb80 20 client.17040 trim_cache size 0 max 16384 2023-09-23T04:51:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.230+0000 7f092c51bb80 1 Processor -- start 2023-09-23T04:51:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 -- start start 2023-09-23T04:51:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ef1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1eea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1cb3920 con 0x5620c1edf400 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1ef1cd0 con 0x5620c1eeabd0 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f092c51bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1efd100 con 0x5620c1edfd20 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ef1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ef1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34398/0 (socket says 172.21.15.5:34398) 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f0912d9d700 1 -- 172.21.15.5:0/1107997215 learned_addr learned my addr 172.21.15.5:0/1107997215 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f090bfff700 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1eea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.231+0000 7f0912d9d700 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 -- 172.21.15.5:0/1107997215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 msgr2=0x5620c1ee2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 -- 172.21.15.5:0/1107997215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 msgr2=0x5620c1ef1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ef1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 -- 172.21.15.5:0/1107997215 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c1c5dde0 con 0x5620c1edfd20 2023-09-23T04:51:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f090bfff700 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1eea120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00fb00 tx=0x7f08fc005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0bd6f1966c1fdd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f09097fa700 1 -- 172.21.15.5:0/1107997215 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc01d070 con 0x5620c1edfd20 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f09097fa700 1 -- 172.21.15.5:0/1107997215 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc01e040 con 0x5620c1edfd20 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.232+0000 7f09097fa700 1 -- 172.21.15.5:0/1107997215 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0248a0 con 0x5620c1edfd20 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- 172.21.15.5:0/1107997215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 msgr2=0x5620c1eea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1eea120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00fb00 tx=0x7f08fc005320 comp rx=0 tx=0).stop 2023-09-23T04:51:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- 172.21.15.5:0/1107997215 shutdown_connections 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ef1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 --2- 172.21.15.5:0/1107997215 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1eea120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- 172.21.15.5:0/1107997215 >> 172.21.15.5:0/1107997215 conn(0x5620c1e54260 msgr2=0x5620c1f01340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- 172.21.15.5:0/1107997215 shutdown_connections 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- 172.21.15.5:0/1107997215 wait complete. 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.233+0000 7f092c51bb80 1 -- start start 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 Processor -- start 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1ee4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1ee4ce0 con 0x5620c1edf400 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1ee4e50 con 0x5620c1eeabd0 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c1ee5010 con 0x5620c1edfd20 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f092c51bb80 10 client.0 _ll_register_callbacks cb 0x5620c1c2c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f0912d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34414/0 (socket says 172.21.15.5:34414) 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.234+0000 7f0912d9d700 1 -- 172.21.15.5:0/2785704350 learned_addr learned my addr 172.21.15.5:0/2785704350 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f090bfff700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1ee4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 -- 172.21.15.5:0/2785704350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 msgr2=0x5620c1ee4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1ee4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 -- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 msgr2=0x5620c1ee45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 -- 172.21.15.5:0/2785704350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c003bd0 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f0912d9d700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5620c1efd2f0 tx=0x7f090c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba0762ad8dae9e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f08f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c019040 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f090c00f040 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.235+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f090c02ba20 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f090c02b590 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 10 client.17046 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5620c1c46c60 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090c037020 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f090c022570 con 0x5620c1edf400 2023-09-23T04:51:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 10 client.17046 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 20 client.17046 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1f4f010 0x5620c1f51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 4 client.17046 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620c1e2bb30 con 0x5620c1f4f010 2023-09-23T04:51:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.236+0000 7f090bfff700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1f4f010 0x5620c1f51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f090bfff700 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1f4f010 0x5620c1f51430 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20a7db8755dc2ee server_cookie=39ed0d130c373f35 in_seq=0 out_seq=0 2023-09-23T04:51:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f08f9ffb700 10 client.17046 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f08f9ffb700 1 -- 172.21.15.5:0/2785704350 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620c1e2bb30 con 0x5620c1f4f010 2023-09-23T04:51:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f08f9ffb700 10 client.17046 handle_command_reply: tid=0 2023-09-23T04:51:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 client.17046 shutdown 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 msgr2=0x5620c1ee3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5620c1efd2f0 tx=0x7f090c00b040 comp rx=0 tx=0).stop 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 shutdown_connections 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x5620c1f4f010 0x5620c1f51430 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5620c1eeabd0 0x5620c1ee45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620c1edf400 0x5620c1ee3b10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f08f9ffb700 0 client.17046 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 --2- 172.21.15.5:0/2785704350 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5620c1edfd20 0x5620c1ee4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.237+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 >> 172.21.15.5:0/2785704350 conn(0x5620c1e54260 msgr2=0x5620c1ee2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 shutdown_connections 2023-09-23T04:51:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f092c51bb80 1 -- 172.21.15.5:0/2785704350 wait complete. 2023-09-23T04:51:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f092c51bb80 20 client.17046 trim_cache size 0 max 16384 2023-09-23T04:51:45.244 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 msgr2=0x7f09000a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 0x7f09000a3bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09040a25e0 tx=0x7f09040d14a0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 msgr2=0x7f0914197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f090400eb30 tx=0x7f090400ee40 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 shutdown_connections 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f09000a1710 0x7f09000a3bd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f091410b490 0x7f091419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f09141020c0 0x7f0914197070 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 --2- 172.21.15.5:0/2873394013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09141029e0 0x7f09141975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 >> 172.21.15.5:0/2873394013 conn(0x7f0914076f30 msgr2=0x7f09140fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.238+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 shutdown_connections 2023-09-23T04:51:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.239+0000 7f0918bff700 1 -- 172.21.15.5:0/2873394013 wait complete. 2023-09-23T04:51:45.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 -- 172.21.15.5:0/4031983590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c0731c0 msgr2=0x7f7d2c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 --2- 172.21.15.5:0/4031983590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c0735a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009990 tx=0x7f7d1c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 -- 172.21.15.5:0/4031983590 shutdown_connections 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 --2- 172.21.15.5:0/4031983590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c111e70 0x7f7d2c114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 --2- 172.21.15.5:0/4031983590 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c0735a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 --2- 172.21.15.5:0/4031983590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 0x7f7d2c111930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 -- 172.21.15.5:0/4031983590 >> 172.21.15.5:0/4031983590 conn(0x7f7d2c0fc990 msgr2=0x7f7d2c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 -- 172.21.15.5:0/4031983590 shutdown_connections 2023-09-23T04:51:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.471+0000 7f7d329ff700 1 -- 172.21.15.5:0/4031983590 wait complete. 2023-09-23T04:51:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 Processor -- start 2023-09-23T04:51:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 -- start start 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 0x7f7d2c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 0x7f7d2c1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1a2580 con 0x7f7d2c073ae0 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1a26f0 con 0x7f7d2c0731c0 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.473+0000 7f7d329ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1a28b0 con 0x7f7d2c111e70 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d30f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 0x7f7d2c1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 0x7f7d2c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41868/0 (socket says 172.21.15.5:41868) 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1111004107 learned_addr learned my addr 172.21.15.5:0/1111004107 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d30f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 0x7f7d2c1a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33230/0 (socket says 172.21.15.5:33230) 2023-09-23T04:51:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1111004107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 msgr2=0x7f7d2c1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 0x7f7d2c1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 msgr2=0x7f7d2c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 0x7f7d2c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1111004107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0095f0 con 0x7f7d2c0731c0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.474+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005950 tx=0x7f7d1c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.475+0000 7f7d297fa700 1 -- 172.21.15.5:0/1111004107 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01d070 con 0x7f7d2c0731c0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.475+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c1a2b30 con 0x7f7d2c0731c0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.475+0000 7f7d297fa700 1 -- 172.21.15.5:0/1111004107 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c005e10 con 0x7f7d2c0731c0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.475+0000 7f7d297fa700 1 -- 172.21.15.5:0/1111004107 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0173f0 con 0x7f7d2c0731c0 2023-09-23T04:51:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.475+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c1aa3d0 con 0x7f7d2c0731c0 2023-09-23T04:51:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.478+0000 7f7d297fa700 1 -- 172.21.15.5:0/1111004107 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7d1c0226e0 con 0x7f7d2c0731c0 2023-09-23T04:51:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 Processor -- start 2023-09-23T04:51:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d297fa700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 0x7f7d140a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 -- start start 2023-09-23T04:51:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d297fa700 1 -- 172.21.15.5:0/1111004107 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7d1c0cea50 con 0x7f7d2c0731c0 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe56011690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56013fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55f6f460 con 0x55fe56006d20 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55f60c20 con 0x55fe56006400 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55f5e540 con 0x55fe56011bd0 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.481+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 0x7f7d140a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41872/0 (socket says 172.21.15.5:41872) 2023-09-23T04:51:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4178226985 learned_addr learned my addr 172.21.15.5:0/4178226985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56013fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe56011690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4178226985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 msgr2=0x55fe56013fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56013fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4178226985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 msgr2=0x55fe56011690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe56011690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4178226985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe55fbc8d0 con 0x55fe56006400 2023-09-23T04:51:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.482+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0981a0 tx=0x7f7d1c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93d0d83e993d311 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 0x7f7d140a3bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c068e70 tx=0x7f7d24003e10 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d127fc700 1 -- 172.21.15.5:0/4178226985 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c09b730 con 0x55fe56006400 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d127fc700 1 -- 172.21.15.5:0/4178226985 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c09f880 con 0x55fe56006400 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d127fc700 1 -- 172.21.15.5:0/4178226985 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c09ae80 con 0x55fe56006400 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4178226985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 msgr2=0x55fe560067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.483+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0981a0 tx=0x7f7d1c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4178226985 shutdown_connections 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56013fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe560067e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4178226985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe56011690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4178226985 >> 172.21.15.5:0/4178226985 conn(0x55fe55f7b260 msgr2=0x55fe56002790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4178226985 shutdown_connections 2023-09-23T04:51:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4178226985 wait complete. 2023-09-23T04:51:45.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.484+0000 7f7d4639bb80 1 -- start start 2023-09-23T04:51:45.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 Processor -- start 2023-09-23T04:51:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 0x55fe560124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 0x55fe5601d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55f59e30 con 0x55fe56011bd0 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ef6710 con 0x55fe56006400 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.485+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ef63b0 con 0x55fe56006d20 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d4639bb80 10 client.0 _ll_register_callbacks cb 0x55fe55d53800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 0x55fe560124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41876/0 (socket says 172.21.15.5:41876) 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 0x55fe560124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33242/0 (socket says 172.21.15.5:33242) 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4065720670 learned_addr learned my addr 172.21.15.5:0/4065720670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 0x55fe5601d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 msgr2=0x55fe560124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 0x55fe560124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4065720670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 msgr2=0x55fe5601d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 0x55fe5601d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 -- 172.21.15.5:0/4065720670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c1aa3d0 con 0x55fe56006400 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 0x55fe5601d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0a3500 tx=0x7f7d1c0a3a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa375aef278caa87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.486+0000 7f7d0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c09ae20 con 0x55fe56006400 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7d1c09b990 con 0x55fe56006400 2023-09-23T04:51:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe55ffa250 con 0x55fe56006400 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1c09bb50 con 0x55fe56006400 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d4639bb80 10 client.27059 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe55d84de0 con 0x55fe56006400 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c0a7020 con 0x55fe56006400 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.487+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f7d1c097640 con 0x55fe56006400 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 10 client.27059 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 10 client.27059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 20 client.27059 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56020e50 0x55fe56023240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 4 client.27059 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe55f52e10 con 0x55fe56020e50 2023-09-23T04:51:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56020e50 0x55fe56023240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56020e50 0x55fe56023240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f7286c9321681d8 server_cookie=d3939b4db126f66a in_seq=0 out_seq=0 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.488+0000 7f7d0b7fe700 10 client.27059 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.489+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/4065720670 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe55f52e10 con 0x55fe56020e50 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.489+0000 7f7d0b7fe700 10 client.27059 handle_command_reply: tid=0 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.489+0000 7f7d4639bb80 1 client.27059 shutdown 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.489+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 msgr2=0x55fe56011fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.489+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0a3500 tx=0x7f7d1c0a3a30 comp rx=0 tx=0).stop 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 shutdown_connections 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56011bd0 0x55fe5601d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56020e50 0x55fe56023240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe56011fb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/4065720670 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56006d20 0x55fe560124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d0b7fe700 0 client.27059 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 >> 172.21.15.5:0/4065720670 conn(0x55fe55f7b260 msgr2=0x55fe56002320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 shutdown_connections 2023-09-23T04:51:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 1 -- 172.21.15.5:0/4065720670 wait complete. 2023-09-23T04:51:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.490+0000 7f7d4639bb80 20 client.27059 trim_cache size 0 max 16384 2023-09-23T04:51:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 Processor -- start 2023-09-23T04:51:45.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 -- start start 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe5600c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe5600c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55dda920 con 0x55fe56006d20 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55ff9e20 con 0x55fe56006400 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5600f2f0 con 0x55fe56011bd0 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.505+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41880/0 (socket says 172.21.15.5:41880) 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1119094768 learned_addr learned my addr 172.21.15.5:0/1119094768 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d30f9c700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe5600c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe5600c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1119094768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 msgr2=0x55fe5600c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe5600c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1119094768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 msgr2=0x55fe5600c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe5600c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 -- 172.21.15.5:0/1119094768 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe55ffe480 con 0x55fe56006400 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe5600c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c09b430 tx=0x7f7d1c0979d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833b338f3a81667d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d12ffd700 1 -- 172.21.15.5:0/1119094768 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0a37a0 con 0x55fe56006400 2023-09-23T04:51:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d12ffd700 1 -- 172.21.15.5:0/1119094768 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c0a7070 con 0x55fe56006400 2023-09-23T04:51:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.506+0000 7f7d12ffd700 1 -- 172.21.15.5:0/1119094768 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c09ae80 con 0x55fe56006400 2023-09-23T04:51:45.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 -- 172.21.15.5:0/1119094768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 msgr2=0x55fe5600b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c09b430 tx=0x7f7d1c0979d0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 -- 172.21.15.5:0/1119094768 shutdown_connections 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe5600c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006400 0x55fe5600b520 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/1119094768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006d20 0x55fe5600c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 -- 172.21.15.5:0/1119094768 >> 172.21.15.5:0/1119094768 conn(0x55fe55f7b260 msgr2=0x55fe5601d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 -- 172.21.15.5:0/1119094768 shutdown_connections 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.507+0000 7f7d4639bb80 1 -- 172.21.15.5:0/1119094768 wait complete. 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 -- start start 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 Processor -- start 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 0x55fe56015050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56015cc0 con 0x55fe56006400 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe56074160 con 0x55fe56006d20 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe560742f0 con 0x55fe56011bd0 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.508+0000 7f7d4639bb80 10 client.0 _ll_register_callbacks cb 0x7f7d24009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 0x55fe56015050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d30f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d30f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33256/0 (socket says 172.21.15.5:33256) 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 0x55fe56015050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34476/0 (socket says 172.21.15.5:34476) 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2bfff700 1 -- 172.21.15.5:0/446212810 learned_addr learned my addr 172.21.15.5:0/446212810 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41888/0 (socket says 172.21.15.5:41888) 2023-09-23T04:51:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 -- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 msgr2=0x55fe56073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 -- 172.21.15.5:0/446212810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 msgr2=0x55fe56015050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 0x55fe56015050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 -- 172.21.15.5:0/446212810 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe55d84de0 con 0x55fe56006d20 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d2b7fe700 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55fe56023b20 tx=0x7f7d2001fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffb7984cde39546 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.509+0000 7f7d0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20016040 con 0x55fe56006d20 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f7d2001f2e0 con 0x55fe56006d20 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d1c09f880 con 0x55fe56006d20 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d2001f4a0 con 0x55fe56006d20 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 10 client.27065 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe55d6dc60 con 0x55fe56006d20 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c09f880 con 0x55fe56006d20 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f7d20010670 con 0x55fe56006d20 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 10 client.27065 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 10 client.27065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.510+0000 7f7d4639bb80 20 client.27065 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56075a40 0x55fe56077e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d4639bb80 4 client.27065 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe55f52b30 con 0x55fe56075a40 2023-09-23T04:51:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56075a40 0x55fe56077e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d2bfff700 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56075a40 0x55fe56077e60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85605cfedd76854 server_cookie=a7ff1be81b0a8fd in_seq=0 out_seq=0 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d0b7fe700 10 client.27065 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d0b7fe700 1 -- 172.21.15.5:0/446212810 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe55f52b30 con 0x55fe56075a40 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d0b7fe700 10 client.27065 handle_command_reply: tid=0 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.511+0000 7f7d4639bb80 1 client.27065 shutdown 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 msgr2=0x55fe56015590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55fe56023b20 tx=0x7f7d2001fea0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 shutdown_connections 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55fe56011bd0 0x55fe56073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe56006400 0x55fe56015050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55fe56075a40 0x55fe56077e60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 --2- 172.21.15.5:0/446212810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe56006d20 0x55fe56015590 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d0b7fe700 0 client.27065 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 >> 172.21.15.5:0/446212810 conn(0x55fe55f7b260 msgr2=0x55fe56010600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 shutdown_connections 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 1 -- 172.21.15.5:0/446212810 wait complete. 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.512+0000 7f7d4639bb80 20 client.27065 trim_cache size 0 max 16384 2023-09-23T04:51:45.518 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 msgr2=0x7f7d140a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 0x7f7d140a3bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c068e70 tx=0x7f7d24003e10 comp rx=0 tx=0).stop 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 msgr2=0x7f7d2c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005950 tx=0x7f7d1c005740 comp rx=0 tx=0).stop 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 shutdown_connections 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7d140a1710 0x7f7d140a3bd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7d2c111e70 0x7f7d2c1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d2c0731c0 0x7f7d2c19d5a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 --2- 172.21.15.5:0/1111004107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d2c073ae0 0x7f7d2c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 >> 172.21.15.5:0/1111004107 conn(0x7f7d2c0fc990 msgr2=0x7f7d2c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 shutdown_connections 2023-09-23T04:51:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.513+0000 7f7d329ff700 1 -- 172.21.15.5:0/1111004107 wait complete. 2023-09-23T04:51:45.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:45 smithi005 ceph-mon[97638]: pgmap v167: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2023-09-23T04:51:45.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:51:45.740 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:45 smithi069 ceph-mon[133931]: pgmap v167: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2023-09-23T04:51:45.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 -- 172.21.15.5:0/663006048 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc614101a80 msgr2=0x7fc614105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 --2- 172.21.15.5:0/663006048 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc614101a80 0x7fc614105ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc608009990 tx=0x7fc608009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 -- 172.21.15.5:0/663006048 shutdown_connections 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 --2- 172.21.15.5:0/663006048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141061f0 0x7fc61410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 --2- 172.21.15.5:0/663006048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6141010d0 0x7fc6141014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 --2- 172.21.15.5:0/663006048 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc614101a80 0x7fc614105ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.759+0000 7fc619fff700 1 -- 172.21.15.5:0/663006048 >> 172.21.15.5:0/663006048 conn(0x7fc6140fc990 msgr2=0x7fc6140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.760+0000 7fc619fff700 1 -- 172.21.15.5:0/663006048 shutdown_connections 2023-09-23T04:51:45.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.760+0000 7fc619fff700 1 -- 172.21.15.5:0/663006048 wait complete. 2023-09-23T04:51:45.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.761+0000 7fc619fff700 1 Processor -- start 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.761+0000 7fc619fff700 1 -- start start 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.761+0000 7fc619fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.761+0000 7fc619fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.761+0000 7fc619fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61419db70 0x7fc61419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc619fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419e5f0 con 0x7fc6141010d0 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc619fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419e760 con 0x7fc61419db70 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc619fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419e920 con 0x7fc6141061f0 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34496/0 (socket says 172.21.15.5:34496) 2023-09-23T04:51:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33270/0 (socket says 172.21.15.5:33270) 2023-09-23T04:51:45.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc6137fe700 1 -- 172.21.15.5:0/2635087879 learned_addr learned my addr 172.21.15.5:0/2635087879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc613fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61419db70 0x7fc61419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 -- 172.21.15.5:0/2635087879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61419db70 msgr2=0x7fc61419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61419db70 0x7fc61419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 -- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 msgr2=0x7fc6141991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc612ffd700 1 -- 172.21.15.5:0/2635087879 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080095f0 con 0x7fc6141061f0 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.762+0000 7fc6137fe700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc612ffd700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc610ff9700 1 -- 172.21.15.5:0/2635087879 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60801d070 con 0x7fc6141061f0 2023-09-23T04:51:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc610ff9700 1 -- 172.21.15.5:0/2635087879 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc608021410 con 0x7fc6141061f0 2023-09-23T04:51:45.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6141a5e20 con 0x7fc6141061f0 2023-09-23T04:51:45.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc610ff9700 1 -- 172.21.15.5:0/2635087879 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608017650 con 0x7fc6141061f0 2023-09-23T04:51:45.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.763+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6141a6240 con 0x7fc6141061f0 2023-09-23T04:51:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.766+0000 7fc610ff9700 1 -- 172.21.15.5:0/2635087879 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fc608021580 con 0x7fc6141061f0 2023-09-23T04:51:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.768+0000 7fc610ff9700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 0x7fc5fc0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.768+0000 7fc610ff9700 1 -- 172.21.15.5:0/2635087879 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc6080ce960 con 0x7fc6141061f0 2023-09-23T04:51:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.768+0000 7fc6137fe700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 0x7fc5fc0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 Processor -- start 2023-09-23T04:51:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 -- start start 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc6137fe700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 0x7fc5fc0a3c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc604011550 tx=0x7fc604009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 0x55c6df8727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 0x55c6df87d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df87ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df7db460 con 0x55c6df872400 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df7ccc20 con 0x55c6df872d20 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df7ca540 con 0x55c6df87dbd0 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 0x55c6df8727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 0x55c6df87d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc613fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df87ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 0x55c6df8727e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34498/0 (socket says 172.21.15.5:34498) 2023-09-23T04:51:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 0x55c6df87d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41906/0 (socket says 172.21.15.5:41906) 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.769+0000 7fc6137fe700 1 -- 172.21.15.5:0/1214863538 learned_addr learned my addr 172.21.15.5:0/1214863538 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 -- 172.21.15.5:0/1214863538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 msgr2=0x55c6df87d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 0x55c6df87d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 -- 172.21.15.5:0/1214863538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 msgr2=0x55c6df8727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 0x55c6df8727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 -- 172.21.15.5:0/1214863538 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6df8288d0 con 0x55c6df87dbd0 2023-09-23T04:51:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc613fff700 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df87ffc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc60c011580 tx=0x7fc60c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3484687b6fbe1f5e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc5f9ffb700 1 -- 172.21.15.5:0/1214863538 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00e040 con 0x55c6df87dbd0 2023-09-23T04:51:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.770+0000 7fc5f9ffb700 1 -- 172.21.15.5:0/1214863538 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60c00f040 con 0x55c6df87dbd0 2023-09-23T04:51:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc5f9ffb700 1 -- 172.21.15.5:0/1214863538 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0044f0 con 0x55c6df87dbd0 2023-09-23T04:51:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1214863538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 msgr2=0x55c6df87ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df87ffc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc60c011580 tx=0x7fc60c009920 comp rx=0 tx=0).stop 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1214863538 shutdown_connections 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df87ffc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872400 0x55c6df8727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1214863538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872d20 0x55c6df87d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1214863538 >> 172.21.15.5:0/1214863538 conn(0x55c6df7e7260 msgr2=0x55c6df86e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1214863538 shutdown_connections 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.771+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1214863538 wait complete. 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.772+0000 7fc62d87cb80 1 -- start start 2023-09-23T04:51:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.772+0000 7fc62d87cb80 1 Processor -- start 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df87b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df7c5e30 con 0x55c6df872d20 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df762710 con 0x55c6df87dbd0 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df7623b0 con 0x55c6df872400 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.773+0000 7fc62d87cb80 10 client.0 _ll_register_callbacks cb 0x55c6df5bf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33280/0 (socket says 172.21.15.5:33280) 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc6137fe700 1 -- 172.21.15.5:0/3692400917 learned_addr learned my addr 172.21.15.5:0/3692400917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc613fff700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df87b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 -- 172.21.15.5:0/3692400917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 msgr2=0x55c6df87a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 -- 172.21.15.5:0/3692400917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 msgr2=0x55c6df87b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df87b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 -- 172.21.15.5:0/3692400917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c011230 con 0x55c6df872d20 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc6137fe700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc612ffd700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87ae40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6080cbc70 tx=0x7fc608099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b5603fdeaf4095 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.774+0000 7fc5f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608097040 con 0x55c6df872d20 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6df866250 con 0x55c6df872d20 2023-09-23T04:51:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc60809bc20 con 0x55c6df872d20 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60809b520 con 0x55c6df872d20 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc62d87cb80 10 client.17073 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c6df5f0de0 con 0x55c6df872d20 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6df866250 con 0x55c6df872d20 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.775+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc6080a3390 con 0x55c6df872d20 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 10 client.17073 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 20 client.17073 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df88d370 0x55c6df88f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 4 client.17073 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6df7bee10 con 0x55c6df88d370 2023-09-23T04:51:45.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.776+0000 7fc6137fe700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df88d370 0x55c6df88f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.777+0000 7fc6137fe700 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df88d370 0x55c6df88f790 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c393b933247067 server_cookie=71d6f4c9e769a90 in_seq=0 out_seq=0 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.777+0000 7fc5f2ffd700 10 client.17073 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.777+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/3692400917 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6df7bee10 con 0x55c6df88d370 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.777+0000 7fc5f2ffd700 10 client.17073 handle_command_reply: tid=0 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.777+0000 7fc62d87cb80 1 client.17073 shutdown 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 msgr2=0x55c6df87ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87ae40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6080cbc70 tx=0x7fc608099ad0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 shutdown_connections 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df88d370 0x55c6df88f790 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df87b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df87a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/3692400917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87ae40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc5f2ffd700 0 client.17073 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:45.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 >> 172.21.15.5:0/3692400917 conn(0x55c6df7e7260 msgr2=0x55c6df86e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 shutdown_connections 2023-09-23T04:51:45.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 1 -- 172.21.15.5:0/3692400917 wait complete. 2023-09-23T04:51:45.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.778+0000 7fc62d87cb80 20 client.17073 trim_cache size 0 max 16384 2023-09-23T04:51:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.792+0000 7fc62d87cb80 1 Processor -- start 2023-09-23T04:51:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 -- start start 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df877b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df646920 con 0x55c6df872d20 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df876770 con 0x55c6df87dbd0 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df875710 con 0x55c6df872400 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc6137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33284/0 (socket says 172.21.15.5:33284) 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc6137fe700 1 -- 172.21.15.5:0/1064630641 learned_addr learned my addr 172.21.15.5:0/1064630641 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc612ffd700 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.793+0000 7fc613fff700 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df877b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 -- 172.21.15.5:0/1064630641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 msgr2=0x55c6df877b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df877b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 -- 172.21.15.5:0/1064630641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 msgr2=0x55c6df87b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 -- 172.21.15.5:0/1064630641 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6df5d9c60 con 0x55c6df872400 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc604011ca0 tx=0x7fc604014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed56202a2d5e8fb9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc5fa7fc700 1 -- 172.21.15.5:0/1064630641 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604022070 con 0x55c6df872400 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc5fa7fc700 1 -- 172.21.15.5:0/1064630641 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc604023040 con 0x55c6df872400 2023-09-23T04:51:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.794+0000 7fc5fa7fc700 1 -- 172.21.15.5:0/1064630641 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604004cf0 con 0x55c6df872400 2023-09-23T04:51:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1064630641 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 msgr2=0x55c6df8797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc604011ca0 tx=0x7fc604014000 comp rx=0 tx=0).stop 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1064630641 shutdown_connections 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df87dbd0 0x55c6df877b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df872400 0x55c6df8797c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1064630641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df87b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1064630641 >> 172.21.15.5:0/1064630641 conn(0x55c6df7e7260 msgr2=0x55c6df8790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1064630641 shutdown_connections 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1064630641 wait complete. 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.795+0000 7fc62d87cb80 1 -- start start 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.796+0000 7fc62d87cb80 1 Processor -- start 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 0x55c6df882920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df8e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df8835f0 con 0x55c6df872d20 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df8e06e0 con 0x55c6df872400 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6df8e0850 con 0x55c6df87dbd0 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc62d87cb80 10 client.0 _ll_register_callbacks cb 0x7fc6080860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34532/0 (socket says 172.21.15.5:34532) 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc612ffd700 1 -- 172.21.15.5:0/1623292411 learned_addr learned my addr 172.21.15.5:0/1623292411 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc613fff700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df8e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 0x55c6df882920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 -- 172.21.15.5:0/1623292411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 msgr2=0x55c6df8e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df8e01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 -- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 msgr2=0x55c6df882e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.797+0000 7fc6137fe700 1 -- 172.21.15.5:0/1623292411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc604017070 con 0x55c6df872400 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc612ffd700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc6137fe700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 0x55c6df882920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc604011ca0 tx=0x7fc604004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c624d478221e43 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc5f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604022040 con 0x55c6df872400 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc60c00f040 con 0x55c6df872400 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fc60402b2b0 con 0x55c6df872400 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60402b470 con 0x55c6df872400 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc62d87cb80 10 client.27083 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.798+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c6df617250 con 0x55c6df872400 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60c00f040 con 0x55c6df872400 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc6040347d0 con 0x55c6df872400 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 10 client.27083 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 10 client.27083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 20 client.27083 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df8e2040 0x55c6df8e4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 4 client.27083 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:51:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6df7beb30 con 0x55c6df8e2040 2023-09-23T04:51:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.799+0000 7fc612ffd700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df8e2040 0x55c6df8e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc612ffd700 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df8e2040 0x55c6df8e4460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472c3a67b4ad2805 server_cookie=ed09f6ab9e1ed55a in_seq=0 out_seq=0 2023-09-23T04:51:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc5f2ffd700 10 client.27083 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc5f2ffd700 1 -- 172.21.15.5:0/1623292411 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6df7beb30 con 0x55c6df8e2040 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc5f2ffd700 10 client.27083 handle_command_reply: tid=0 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc62d87cb80 1 client.27083 shutdown 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 msgr2=0x55c6df882920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.800+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 0x55c6df882920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc604011ca0 tx=0x7fc604004750 comp rx=0 tx=0).stop 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 shutdown_connections 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6df8e2040 0x55c6df8e4460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6df87dbd0 0x55c6df8e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc5f2ffd700 0 client.27083 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6df872400 0x55c6df882920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 --2- 172.21.15.5:0/1623292411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6df872d20 0x55c6df882e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 >> 172.21.15.5:0/1623292411 conn(0x55c6df7e7260 msgr2=0x55c6df879940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 shutdown_connections 2023-09-23T04:51:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 1 -- 172.21.15.5:0/1623292411 wait complete. 2023-09-23T04:51:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.801+0000 7fc62d87cb80 20 client.27083 trim_cache size 0 max 16384 2023-09-23T04:51:45.806 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:51:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 msgr2=0x7fc5fc0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 0x7fc5fc0a3c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc604011550 tx=0x7fc604009260 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 msgr2=0x7fc614199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608005740 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 shutdown_connections 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc5fc0a1790 0x7fc5fc0a3c50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc61419db70 0x7fc61419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6141010d0 0x7fc6141991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 --2- 172.21.15.5:0/2635087879 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc6141061f0 0x7fc614199710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 >> 172.21.15.5:0/2635087879 conn(0x7fc6140fc990 msgr2=0x7fc6140fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 shutdown_connections 2023-09-23T04:51:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:45.802+0000 7fc619fff700 1 -- 172.21.15.5:0/2635087879 wait complete. 2023-09-23T04:51:45.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:45 smithi118 ceph-mon[134297]: pgmap v167: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 198 op/s 2023-09-23T04:51:45.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 339182 5.97 MB/sec execute 141 sec latency 2968.901 ms 2023-09-23T04:51:46.823 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:51:46.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:51:46.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 340836 6.02 MB/sec execute 142 sec latency 11.546 ms 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 -- 172.21.15.5:0/3566498876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 msgr2=0x7f7578069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 --2- 172.21.15.5:0/3566498876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578069480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7568009990 tx=0x7f7568009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 -- 172.21.15.5:0/3566498876 shutdown_connections 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 --2- 172.21.15.5:0/3566498876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7578106210 0x7f757810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 --2- 172.21.15.5:0/3566498876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578069480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 --2- 172.21.15.5:0/3566498876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578069a50 0x7f7578105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 -- 172.21.15.5:0/3566498876 >> 172.21.15.5:0/3566498876 conn(0x7f7578077ca0 msgr2=0x7f75780780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 -- 172.21.15.5:0/3566498876 shutdown_connections 2023-09-23T04:51:47.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.051+0000 7f757e7ff700 1 -- 172.21.15.5:0/3566498876 wait complete. 2023-09-23T04:51:47.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.052+0000 7f757e7ff700 1 Processor -- start 2023-09-23T04:51:47.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.052+0000 7f757e7ff700 1 -- start start 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578106210 0x7f75781974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757819b900 0x7f757819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757819c380 con 0x7f7578106210 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757819c4f0 con 0x7f75780690a0 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757e7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757819c6b0 con 0x7f757819b900 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f7577fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f7577fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41944/0 (socket says 172.21.15.5:41944) 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f7577fff700 1 -- 172.21.15.5:0/613713394 learned_addr learned my addr 172.21.15.5:0/613713394 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f757cd9c700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757819b900 0x7f757819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f75777fe700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578106210 0x7f75781974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.053+0000 7f7577fff700 1 -- 172.21.15.5:0/613713394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757819b900 msgr2=0x7f757819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f7577fff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757819b900 0x7f757819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f7577fff700 1 -- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578106210 msgr2=0x7f75781974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f7577fff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578106210 0x7f75781974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f7577fff700 1 -- 172.21.15.5:0/613713394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680095f0 con 0x7f75780690a0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f7577fff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7568005910 tx=0x7f7568005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f75757fa700 1 -- 172.21.15.5:0/613713394 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756801d070 con 0x7f75780690a0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f75757fa700 1 -- 172.21.15.5:0/613713394 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7568022470 con 0x7f75780690a0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f75757fa700 1 -- 172.21.15.5:0/613713394 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568017650 con 0x7f75780690a0 2023-09-23T04:51:47.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75780fff20 con 0x7f75780690a0 2023-09-23T04:51:47.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.054+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781003e0 con 0x7f75780690a0 2023-09-23T04:51:47.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.058+0000 7f75757fa700 1 -- 172.21.15.5:0/613713394 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f75680225e0 con 0x7f75780690a0 2023-09-23T04:51:47.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.059+0000 7f75757fa700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 0x7f75600a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.059+0000 7f7592220b80 1 Processor -- start 2023-09-23T04:51:47.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.059+0000 7f75757fa700 1 -- 172.21.15.5:0/613713394 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f75680ce870 con 0x7f75780690a0 2023-09-23T04:51:47.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.059+0000 7f75777fe700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 0x7f75600a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.059+0000 7f7592220b80 1 -- start start 2023-09-23T04:51:47.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4115710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4118040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add40734e0 con 0x55add410a480 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add4064ca0 con 0x55add410ada0 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7592220b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add40625c0 con 0x55add4115c50 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7577fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7577fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34558/0 (socket says 172.21.15.5:34558) 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f7577fff700 1 -- 172.21.15.5:0/1696254372 learned_addr learned my addr 172.21.15.5:0/1696254372 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f75777fe700 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4115710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f757cd9c700 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4118040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.060+0000 7f75777fe700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 0x7f75600a3c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7578198570 tx=0x7f756c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 -- 172.21.15.5:0/1696254372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 msgr2=0x55add4118040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4118040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 -- 172.21.15.5:0/1696254372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 msgr2=0x55add4115710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4115710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 -- 172.21.15.5:0/1696254372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55add40c0950 con 0x55add410a480 2023-09-23T04:51:47.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f7577fff700 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75680a3380 tx=0x7f7568009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c2bea8e5dce80f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f755e7fc700 1 -- 172.21.15.5:0/1696254372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756809af10 con 0x55add410a480 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f755e7fc700 1 -- 172.21.15.5:0/1696254372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756809bcb0 con 0x55add410a480 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.061+0000 7f755e7fc700 1 -- 172.21.15.5:0/1696254372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680ce6b0 con 0x55add410a480 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 -- 172.21.15.5:0/1696254372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 msgr2=0x55add410a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75680a3380 tx=0x7f7568009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 -- 172.21.15.5:0/1696254372 shutdown_connections 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4118040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add410a860 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 --2- 172.21.15.5:0/1696254372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4115710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 -- 172.21.15.5:0/1696254372 >> 172.21.15.5:0/1696254372 conn(0x55add407f2e0 msgr2=0x55add4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 -- 172.21.15.5:0/1696254372 shutdown_connections 2023-09-23T04:51:47.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.062+0000 7f7592220b80 1 -- 172.21.15.5:0/1696254372 wait complete. 2023-09-23T04:51:47.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.063+0000 7f7592220b80 1 -- start start 2023-09-23T04:51:47.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.063+0000 7f7592220b80 1 Processor -- start 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4116570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add4116ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add405deb0 con 0x55add4115c50 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add3ffa790 con 0x55add410a480 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add3ffa430 con 0x55add410ada0 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7592220b80 10 client.0 _ll_register_callbacks cb 0x55add3e57880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7577fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f75777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7577fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41970/0 (socket says 172.21.15.5:41970) 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f7577fff700 1 -- 172.21.15.5:0/3749853959 learned_addr learned my addr 172.21.15.5:0/3749853959 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.064+0000 7f757cd9c700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add4116ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 -- 172.21.15.5:0/3749853959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 msgr2=0x55add4116570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4116570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 -- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 msgr2=0x55add4116ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add4116ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 -- 172.21.15.5:0/3749853959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781003e0 con 0x55add410a480 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f75777fe700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7577fff700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7568006010 tx=0x7f7568022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9b70c4d9a53108 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f75577fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756809af10 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55add40fe2d0 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f75680b3e40 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756809b5b0 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7592220b80 10 client.27095 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.065+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55add3e88e60 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55add40fe2d0 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f756809ab80 con 0x55add410a480 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 20 client.27095 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add4125000 0x55add4127420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 4 client.27095 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:47.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55add4056e90 con 0x55add4125000 2023-09-23T04:51:47.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.066+0000 7f75777fe700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add4125000 0x55add4127420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.067+0000 7f75777fe700 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add4125000 0x55add4127420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6d575c21536a47 server_cookie=4ad3a5626ed08953 in_seq=0 out_seq=0 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.067+0000 7f75577fe700 10 client.27095 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.067+0000 7f75577fe700 1 -- 172.21.15.5:0/3749853959 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55add4056e90 con 0x55add4125000 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.067+0000 7f75577fe700 10 client.27095 handle_command_reply: tid=0 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.067+0000 7f7592220b80 1 client.27095 shutdown 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 msgr2=0x55add4116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7568006010 tx=0x7f7568022af0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 shutdown_connections 2023-09-23T04:51:47.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add4125000 0x55add4127420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add4116ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add4116030 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 --2- 172.21.15.5:0/3749853959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 >> 172.21.15.5:0/3749853959 conn(0x55add407f2e0 msgr2=0x55add41063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f75577fe700 0 client.27095 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 shutdown_connections 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 1 -- 172.21.15.5:0/3749853959 wait complete. 2023-09-23T04:51:47.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.068+0000 7f7592220b80 20 client.27095 trim_cache size 0 max 16384 2023-09-23T04:51:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.087+0000 7f7592220b80 1 Processor -- start 2023-09-23T04:51:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.087+0000 7f7592220b80 1 -- start start 2023-09-23T04:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add411a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add3ede9a0 con 0x55add4115c50 2023-09-23T04:51:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add411a5a0 con 0x55add410a480 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7592220b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add4127a80 con 0x55add410ada0 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f75777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f75777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33328/0 (socket says 172.21.15.5:33328) 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f75777fe700 1 -- 172.21.15.5:0/3832894885 learned_addr learned my addr 172.21.15.5:0/3832894885 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f757cd9c700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.088+0000 7f7577fff700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add411a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 -- 172.21.15.5:0/3832894885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 msgr2=0x55add411a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add411a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 -- 172.21.15.5:0/3832894885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 msgr2=0x55add410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 -- 172.21.15.5:0/3832894885 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55add3e71ce0 con 0x55add410ada0 2023-09-23T04:51:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f7577fff700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f75777fe700 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f756c0094f0 tx=0x7f756c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852f1a889a33d91d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f755effd700 1 -- 172.21.15.5:0/3832894885 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c022030 con 0x55add410ada0 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f755effd700 1 -- 172.21.15.5:0/3832894885 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756c023040 con 0x55add410ada0 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.089+0000 7f755effd700 1 -- 172.21.15.5:0/3832894885 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c01d410 con 0x55add410ada0 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- 172.21.15.5:0/3832894885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 msgr2=0x55add4113c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f756c0094f0 tx=0x7f756c014000 comp rx=0 tx=0).stop 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- 172.21.15.5:0/3832894885 shutdown_connections 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add4115c50 0x55add410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410a480 0x55add411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 --2- 172.21.15.5:0/3832894885 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add410ada0 0x55add4113c50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- 172.21.15.5:0/3832894885 >> 172.21.15.5:0/3832894885 conn(0x55add407f2e0 msgr2=0x55add4113540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- 172.21.15.5:0/3832894885 shutdown_connections 2023-09-23T04:51:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- 172.21.15.5:0/3832894885 wait complete. 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.090+0000 7f7592220b80 1 -- start start 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 Processor -- start 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add41186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4119140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add4178f70 con 0x55add410a480 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add41790e0 con 0x55add410ada0 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55add41792a0 con 0x55add4115c50 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7592220b80 10 client.0 _ll_register_callbacks cb 0x7f756c009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f75777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f7577fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add41186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f757cd9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4119140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f75777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41992/0 (socket says 172.21.15.5:41992) 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.091+0000 7f75777fe700 1 -- 172.21.15.5:0/447667206 learned_addr learned my addr 172.21.15.5:0/447667206 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 -- 172.21.15.5:0/447667206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 msgr2=0x55add4119140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4119140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 -- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 msgr2=0x55add41186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add41186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 -- 172.21.15.5:0/447667206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55add3e88e60 con 0x55add410ada0 2023-09-23T04:51:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f757cd9c700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4119140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75777fe700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f756c02be40 tx=0x7f756c02be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d96502cf8a8397 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75577fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c022070 con 0x55add410ada0 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f756809bcb0 con 0x55add410ada0 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.092+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f756c02b540 con 0x55add410ada0 2023-09-23T04:51:47.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756c003880 con 0x55add410ada0 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 10 client.27101 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f756c017070 con 0x55add410ada0 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756809bcb0 con 0x55add410ada0 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f756c032070 con 0x55add410ada0 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 10 client.27101 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 20 client.27101 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add417a940 0x55add417cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 4 client.27101 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.093+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55add4056bb0 con 0x55add417a940 2023-09-23T04:51:47.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.094+0000 7f7577fff700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add417a940 0x55add417cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.094+0000 7f7577fff700 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add417a940 0x55add417cd60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f012180ea78e2814 server_cookie=cced4b20aa0e03e in_seq=0 out_seq=0 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.094+0000 7f75577fe700 10 client.27101 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.095+0000 7f75577fe700 1 -- 172.21.15.5:0/447667206 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55add4056bb0 con 0x55add417a940 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.095+0000 7f75577fe700 10 client.27101 handle_command_reply: tid=0 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.095+0000 7f7592220b80 1 client.27101 shutdown 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 msgr2=0x55add4118be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f756c02be40 tx=0x7f756c02be70 comp rx=0 tx=0).stop 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 shutdown_connections 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55add417a940 0x55add417cd60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55add4115c50 0x55add4119140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55add410a480 0x55add41186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 --2- 172.21.15.5:0/447667206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55add410ada0 0x55add4118be0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f75577fe700 0 client.27101 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 >> 172.21.15.5:0/447667206 conn(0x55add407f2e0 msgr2=0x55add4113dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 shutdown_connections 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 1 -- 172.21.15.5:0/447667206 wait complete. 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.096+0000 7f7592220b80 20 client.27101 trim_cache size 0 max 16384 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:51:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "013f7259-7341-4980-a3a4-5d0e141bf74a", 2023-09-23T04:51:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:51:47.101 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:47.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 msgr2=0x7f75600a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 0x7f75600a3c50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7578198570 tx=0x7f756c009260 comp rx=0 tx=0).stop 2023-09-23T04:51:47.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 msgr2=0x7f7578196f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7568005910 tx=0x7f7568005720 comp rx=0 tx=0).stop 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 shutdown_connections 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75600a1790 0x7f75600a3c50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757819b900 0x7f757819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75780690a0 0x7f7578196f60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 --2- 172.21.15.5:0/613713394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7578106210 0x7f75781974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 >> 172.21.15.5:0/613713394 conn(0x7f7578077ca0 msgr2=0x7f7578104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 shutdown_connections 2023-09-23T04:51:47.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.097+0000 7f757e7ff700 1 -- 172.21.15.5:0/613713394 wait complete. 2023-09-23T04:51:47.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:46 smithi005 ceph-mon[97638]: pgmap v168: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2023-09-23T04:51:47.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:51:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:46 smithi118 ceph-mon[134297]: pgmap v168: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2023-09-23T04:51:47.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:46 smithi069 ceph-mon[133931]: pgmap v168: 129 pgs: 129 active+clean; 204 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2023-09-23T04:51:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 -- 172.21.15.5:0/2738783264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 msgr2=0x7fd03c111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 --2- 172.21.15.5:0/2738783264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c111930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030009ca0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 -- 172.21.15.5:0/2738783264 shutdown_connections 2023-09-23T04:51:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 --2- 172.21.15.5:0/2738783264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c111e70 0x7fd03c114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 --2- 172.21.15.5:0/2738783264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c0731c0 0x7fd03c0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 --2- 172.21.15.5:0/2738783264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c111930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 -- 172.21.15.5:0/2738783264 >> 172.21.15.5:0/2738783264 conn(0x7fd03c0fc990 msgr2=0x7fd03c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 -- 172.21.15.5:0/2738783264 shutdown_connections 2023-09-23T04:51:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.358+0000 7fd040bff700 1 -- 172.21.15.5:0/2738783264 wait complete. 2023-09-23T04:51:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.359+0000 7fd040bff700 1 Processor -- start 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 -- start start 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c111e70 0x7fd03c1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1a2580 con 0x7fd03c073ae0 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1a26f0 con 0x7fd03c111e70 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd040bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1a28b0 con 0x7fd03c0731c0 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33356/0 (socket says 172.21.15.5:33356) 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd03a59c700 1 -- 172.21.15.5:0/655897777 learned_addr learned my addr 172.21.15.5:0/655897777 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd039d9b700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.360+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c111e70 0x7fd03c1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 -- 172.21.15.5:0/655897777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c111e70 msgr2=0x7fd03c1a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c111e70 0x7fd03c1a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 -- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 msgr2=0x7fd03c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 -- 172.21.15.5:0/655897777 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0300095f0 con 0x7fd03c0731c0 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd039d9b700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd03a59c700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd02b7fe700 1 -- 172.21.15.5:0/655897777 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c009790 con 0x7fd03c0731c0 2023-09-23T04:51:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd02b7fe700 1 -- 172.21.15.5:0/655897777 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd02c00cbd0 con 0x7fd03c0731c0 2023-09-23T04:51:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd02b7fe700 1 -- 172.21.15.5:0/655897777 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c018680 con 0x7fd03c0731c0 2023-09-23T04:51:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c1a2b90 con 0x7fd03c0731c0 2023-09-23T04:51:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.361+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c1aa3d0 con 0x7fd03c0731c0 2023-09-23T04:51:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.365+0000 7fd02b7fe700 1 -- 172.21.15.5:0/655897777 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd02c00cd40 con 0x7fd03c0731c0 2023-09-23T04:51:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.366+0000 7fd05458db80 1 Processor -- start 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 -- start start 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e926b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926bd20 0x5651e9276690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e91d4460 con 0x5651e926b400 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e91c5c20 con 0x5651e926bd20 2023-09-23T04:51:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd05458db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e91c3540 con 0x5651e9276bd0 2023-09-23T04:51:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926bd20 0x5651e9276690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd039d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd02b7fe700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 0x7fd0240a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd03ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e926b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd039d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33358/0 (socket says 172.21.15.5:33358) 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd039d9b700 1 -- 172.21.15.5:0/1058272601 learned_addr learned my addr 172.21.15.5:0/1058272601 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.367+0000 7fd02b7fe700 1 -- 172.21.15.5:0/655897777 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd02c014070 con 0x7fd03c0731c0 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 -- 172.21.15.5:0/1058272601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926bd20 msgr2=0x5651e9276690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926bd20 0x5651e9276690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 -- 172.21.15.5:0/1058272601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 msgr2=0x5651e926b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e926b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 -- 172.21.15.5:0/1058272601 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651e92218d0 con 0x5651e9276bd0 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e926b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 0x7fd0240a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd039d9b700 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b2e0 tx=0x7fd030005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498b062166543c38 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd028ff9700 1 -- 172.21.15.5:0/1058272601 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000beb0 con 0x5651e9276bd0 2023-09-23T04:51:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd028ff9700 1 -- 172.21.15.5:0/1058272601 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03001e040 con 0x5651e9276bd0 2023-09-23T04:51:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.368+0000 7fd028ff9700 1 -- 172.21.15.5:0/1058272601 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030004b60 con 0x5651e9276bd0 2023-09-23T04:51:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 -- 172.21.15.5:0/1058272601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 msgr2=0x5651e9278fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 0x7fd0240a3bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd034009ec0 tx=0x7fd034009040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b2e0 tx=0x7fd030005c00 comp rx=0 tx=0).stop 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 -- 172.21.15.5:0/1058272601 shutdown_connections 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9278fc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e926b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 --2- 172.21.15.5:0/1058272601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926bd20 0x5651e9276690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.369+0000 7fd05458db80 1 -- 172.21.15.5:0/1058272601 >> 172.21.15.5:0/1058272601 conn(0x5651e91e0260 msgr2=0x5651e9267790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.370+0000 7fd05458db80 1 -- 172.21.15.5:0/1058272601 shutdown_connections 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.370+0000 7fd05458db80 1 -- 172.21.15.5:0/1058272601 wait complete. 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.371+0000 7fd05458db80 1 -- start start 2023-09-23T04:51:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.371+0000 7fd05458db80 1 Processor -- start 2023-09-23T04:51:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e9273960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 0x5651e9273ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e92743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e91bee30 con 0x5651e926b400 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e915b710 con 0x5651e9276bd0 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e915b3b0 con 0x5651e926bd20 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd05458db80 10 client.0 _ll_register_callbacks cb 0x5651e8fb8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd039d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 0x5651e9273ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd03ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e92743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd039d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 0x5651e9273ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33370/0 (socket says 172.21.15.5:33370) 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd039d9b700 1 -- 172.21.15.5:0/2346614703 learned_addr learned my addr 172.21.15.5:0/2346614703 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.372+0000 7fd03a59c700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e9273960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 -- 172.21.15.5:0/2346614703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 msgr2=0x5651e9273ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 0x5651e9273ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 -- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 msgr2=0x5651e9273960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e9273960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 -- 172.21.15.5:0/2346614703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c114c10 con 0x5651e9276bd0 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e92743e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd034009fd0 tx=0x7fd034004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c873116ad776b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034007ea0 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd034016bc0 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.373+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651e925f250 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd03400e040 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd05458db80 10 client.27113 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5651e8fe9de0 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651e925f250 con 0x5651e9276bd0 2023-09-23T04:51:47.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd03402b440 con 0x5651e9276bd0 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd05458db80 10 client.27113 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.374+0000 7fd05458db80 10 client.27113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.375+0000 7fd05458db80 20 client.27113 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.375+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e9286810 0x5651e9288c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.375+0000 7fd05458db80 4 client.27113 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.375+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651e91b7e10 con 0x5651e9286810 2023-09-23T04:51:47.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.375+0000 7fd03a59c700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e9286810 0x5651e9288c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.376+0000 7fd021ffb700 10 client.27113 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.376+0000 7fd03a59c700 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e9286810 0x5651e9288c30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50d4fa4c59d8daa server_cookie=978139cf6bf24b6e in_seq=0 out_seq=0 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd021ffb700 1 -- 172.21.15.5:0/2346614703 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5651e91b7e10 con 0x5651e9286810 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd021ffb700 10 client.27113 handle_command_reply: tid=0 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 client.27113 shutdown 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 msgr2=0x5651e92743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e92743e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd034009fd0 tx=0x7fd034004ad0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 shutdown_connections 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e9286810 0x5651e9288c30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e92743e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926b400 0x5651e9273960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 --2- 172.21.15.5:0/2346614703 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926bd20 0x5651e9273ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd021ffb700 0 client.27113 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 >> 172.21.15.5:0/2346614703 conn(0x5651e91e0260 msgr2=0x5651e9267380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 shutdown_connections 2023-09-23T04:51:47.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.377+0000 7fd05458db80 1 -- 172.21.15.5:0/2346614703 wait complete. 2023-09-23T04:51:47.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.378+0000 7fd05458db80 20 client.27113 trim_cache size 0 max 16384 2023-09-23T04:51:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.399+0000 7fd05458db80 1 Processor -- start 2023-09-23T04:51:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.399+0000 7fd05458db80 1 -- start start 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926b400 0x5651e9274030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e9272f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e903f920 con 0x5651e926bd20 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e925f100 con 0x5651e926b400 2023-09-23T04:51:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd05458db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e9283020 con 0x5651e9276bd0 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd039d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e9272f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33378/0 (socket says 172.21.15.5:33378) 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03ad9d700 1 -- 172.21.15.5:0/942058031 learned_addr learned my addr 172.21.15.5:0/942058031 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03a59c700 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926b400 0x5651e9274030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03ad9d700 1 -- 172.21.15.5:0/942058031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926b400 msgr2=0x5651e9274030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.400+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926b400 0x5651e9274030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd03ad9d700 1 -- 172.21.15.5:0/942058031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 msgr2=0x5651e9272f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e9272f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd03ad9d700 1 -- 172.21.15.5:0/942058031 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651e8fd2c60 con 0x5651e9276bd0 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd039d9b700 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e9272f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd034000f80 tx=0x7fd034009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d5f397eb80e2a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd0297fa700 1 -- 172.21.15.5:0/942058031 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400e040 con 0x5651e9276bd0 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd0297fa700 1 -- 172.21.15.5:0/942058031 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03400f040 con 0x5651e9276bd0 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.401+0000 7fd0297fa700 1 -- 172.21.15.5:0/942058031 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034003c60 con 0x5651e9276bd0 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- 172.21.15.5:0/942058031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 msgr2=0x5651e9270c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd034000f80 tx=0x7fd034009510 comp rx=0 tx=0).stop 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- 172.21.15.5:0/942058031 shutdown_connections 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e9276bd0 0x5651e9270c30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e926b400 0x5651e9274030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 --2- 172.21.15.5:0/942058031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e9272f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- 172.21.15.5:0/942058031 >> 172.21.15.5:0/942058031 conn(0x5651e91e0260 msgr2=0x5651e9273920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- 172.21.15.5:0/942058031 shutdown_connections 2023-09-23T04:51:47.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- 172.21.15.5:0/942058031 wait complete. 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.402+0000 7fd05458db80 1 -- start start 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 Processor -- start 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e927b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e927bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e927dc60 con 0x5651e926bd20 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e927c100 con 0x5651e9276bd0 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651e927c2c0 con 0x5651e926b400 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd05458db80 10 client.0 _ll_register_callbacks cb 0x7fd034017810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd03a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33394/0 (socket says 172.21.15.5:33394) 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.403+0000 7fd03a59c700 1 -- 172.21.15.5:0/991491245 learned_addr learned my addr 172.21.15.5:0/991491245 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e927bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd039d9b700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e927b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 -- 172.21.15.5:0/991491245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 msgr2=0x5651e927bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e927bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 -- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 msgr2=0x5651e927b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e927b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 -- 172.21.15.5:0/991491245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651e92218d0 con 0x5651e926b400 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03ad9d700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e927bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd039d9b700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e927b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd03a59c700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd02c096a20 tx=0x7fd02c0cfc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd6d25583bb4eeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:51:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd021ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c096040 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fd02c0b2e40 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.404+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03001e040 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02c09e670 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 10 client.27166 fetch_fsmap learned FSMap version 35 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5651e8fe9de0 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03001e040 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fd02c0a2420 con 0x5651e926b400 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 10 client.27166 fetch_fsmap finished waiting for FSMap version 35 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 20 client.27166 populate_metadata read hostname 'smithi005' 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e92dba00 0x5651e92dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:51:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 4 client.27166 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651e91b7b30 con 0x5651e92dba00 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.405+0000 7fd039d9b700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e92dba00 0x5651e92dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd039d9b700 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e92dba00 0x5651e92dde20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2977f528df2186b0 server_cookie=27b31a90205e09a5 in_seq=0 out_seq=0 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd021ffb700 10 client.27166 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd021ffb700 1 -- 172.21.15.5:0/991491245 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5651e91b7b30 con 0x5651e92dba00 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd021ffb700 10 client.27166 handle_command_reply: tid=0 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd05458db80 1 client.27166 shutdown 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 msgr2=0x5651e927d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.406+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd02c096a20 tx=0x7fd02c0cfc70 comp rx=0 tx=0).stop 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 shutdown_connections 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5651e92dba00 0x5651e92dde20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5651e9276bd0 0x5651e927bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5651e926b400 0x5651e927d390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 --2- 172.21.15.5:0/991491245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651e926bd20 0x5651e927b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd021ffb700 0 client.27166 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 >> 172.21.15.5:0/991491245 conn(0x5651e91e0260 msgr2=0x5651e926db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 shutdown_connections 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 1 -- 172.21.15.5:0/991491245 wait complete. 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.407+0000 7fd05458db80 20 client.27166 trim_cache size 0 max 16384 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:51:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "013f7259-7341-4980-a3a4-5d0e141bf74a": { 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "013f7259-7341-4980-a3a4-5d0e141bf74a", 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:51:47.413 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 msgr2=0x7fd0240a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 0x7fd0240a3bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd034009ec0 tx=0x7fd034009040 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 msgr2=0x7fd03c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00e9a0 tx=0x7fd02c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 shutdown_connections 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd0240a1710 0x7fd0240a3bd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd03c111e70 0x7fd03c1a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd03c0731c0 0x7fd03c19d5a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 --2- 172.21.15.5:0/655897777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd03c073ae0 0x7fd03c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 >> 172.21.15.5:0/655897777 conn(0x7fd03c0fc990 msgr2=0x7fd03c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:51:47.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 shutdown_connections 2023-09-23T04:51:47.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:51:47.408+0000 7fd040bff700 1 -- 172.21.15.5:0/655897777 wait complete. 2023-09-23T04:51:47.430 INFO:tasks.cephfs.filesystem:scrub status for tag:013f7259-7341-4980-a3a4-5d0e141bf74a - {'path': '/', 'tag': '013f7259-7341-4980-a3a4-5d0e141bf74a', 'options': 'recursive,force'} 2023-09-23T04:51:47.945 INFO:tasks.workunit.client.0.smithi005.stdout: 1 341990 6.02 MB/sec execute 143 sec latency 17.451 ms 2023-09-23T04:51:48.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:51:48.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:48.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:47 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:51:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:48.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:47 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:48.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:47 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:48.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 342210 5.98 MB/sec execute 144 sec latency 757.497 ms 2023-09-23T04:51:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:51:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:48 smithi005 ceph-mon[97638]: pgmap v169: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-09-23T04:51:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:51:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:48 smithi118 ceph-mon[134297]: pgmap v169: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:51:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:48 smithi069 ceph-mon[133931]: pgmap v169: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2023-09-23T04:51:49.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 342210 5.94 MB/sec execute 145 sec latency 1757.629 ms 2023-09-23T04:51:50.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 343448 5.94 MB/sec execute 146 sec latency 2151.433 ms 2023-09-23T04:51:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:51 smithi005 ceph-mon[97638]: pgmap v170: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 564 KiB/s wr, 93 op/s 2023-09-23T04:51:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:51 smithi118 ceph-mon[134297]: pgmap v170: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 564 KiB/s wr, 93 op/s 2023-09-23T04:51:51.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 344734 5.95 MB/sec execute 147 sec latency 33.301 ms 2023-09-23T04:51:51.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:51 smithi069 ceph-mon[133931]: pgmap v170: 129 pgs: 129 active+clean; 206 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 564 KiB/s wr, 93 op/s 2023-09-23T04:51:52.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 345759 5.94 MB/sec execute 148 sec latency 196.725 ms 2023-09-23T04:51:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:52 smithi005 ceph-mon[97638]: pgmap v171: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 133 op/s 2023-09-23T04:51:53.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:52 smithi118 ceph-mon[134297]: pgmap v171: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 133 op/s 2023-09-23T04:51:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:52 smithi069 ceph-mon[133931]: pgmap v171: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 KiB/s wr, 133 op/s 2023-09-23T04:51:53.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 345759 5.90 MB/sec execute 149 sec latency 1196.874 ms 2023-09-23T04:51:54.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 345759 5.86 MB/sec execute 150 sec latency 2197.025 ms 2023-09-23T04:51:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:54 smithi005 ceph-mon[97638]: pgmap v172: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 635 KiB/s wr, 102 op/s 2023-09-23T04:51:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:54 smithi118 ceph-mon[134297]: pgmap v172: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 635 KiB/s wr, 102 op/s 2023-09-23T04:51:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:54 smithi069 ceph-mon[133931]: pgmap v172: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 635 KiB/s wr, 102 op/s 2023-09-23T04:51:55.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 347622 5.91 MB/sec execute 151 sec latency 2330.374 ms 2023-09-23T04:51:56.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:55 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:56.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:55 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:55 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:56.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:55 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:56.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:55 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:56.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:55 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:56.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348584 5.88 MB/sec execute 152 sec latency 64.050 ms 2023-09-23T04:51:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:56 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:56 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:56 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:51:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:56 smithi005 ceph-mon[97638]: pgmap v173: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 585 KiB/s wr, 93 op/s 2023-09-23T04:51:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:56 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:56 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:56 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:51:57.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:56 smithi118 ceph-mon[134297]: pgmap v173: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 585 KiB/s wr, 93 op/s 2023-09-23T04:51:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:56 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:56 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:56 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:51:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:56 smithi069 ceph-mon[133931]: pgmap v173: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 585 KiB/s wr, 93 op/s 2023-09-23T04:51:57.946 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348584 5.85 MB/sec execute 153 sec latency 1064.106 ms 2023-09-23T04:51:58.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348584 5.81 MB/sec execute 154 sec latency 2064.250 ms 2023-09-23T04:51:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:58 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:51:58 smithi005 ceph-mon[97638]: pgmap v174: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 849 KiB/s wr, 134 op/s 2023-09-23T04:51:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:58 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:51:58 smithi118 ceph-mon[134297]: pgmap v174: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 849 KiB/s wr, 134 op/s 2023-09-23T04:51:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:58 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:51:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:51:58 smithi069 ceph-mon[133931]: pgmap v174: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 849 KiB/s wr, 134 op/s 2023-09-23T04:51:59.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348584 5.77 MB/sec execute 155 sec latency 3064.409 ms 2023-09-23T04:52:00.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 349302 5.76 MB/sec execute 156 sec latency 3201.530 ms 2023-09-23T04:52:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:01 smithi005 ceph-mon[97638]: pgmap v175: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 573 KiB/s wr, 91 op/s 2023-09-23T04:52:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:01 smithi118 ceph-mon[134297]: pgmap v175: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 573 KiB/s wr, 91 op/s 2023-09-23T04:52:01.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 349386 5.73 MB/sec execute 157 sec latency 1326.647 ms 2023-09-23T04:52:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:01 smithi069 ceph-mon[133931]: pgmap v175: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 573 KiB/s wr, 91 op/s 2023-09-23T04:52:02.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 350101 5.70 MB/sec execute 158 sec latency 52.090 ms 2023-09-23T04:52:03.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:02 smithi005 ceph-mon[97638]: pgmap v176: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 KiB/s wr, 124 op/s 2023-09-23T04:52:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:02 smithi118 ceph-mon[134297]: pgmap v176: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 KiB/s wr, 124 op/s 2023-09-23T04:52:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:02 smithi069 ceph-mon[133931]: pgmap v176: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 KiB/s wr, 124 op/s 2023-09-23T04:52:03.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 351526 5.74 MB/sec execute 159 sec latency 68.682 ms 2023-09-23T04:52:04.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 352646 5.74 MB/sec execute 160 sec latency 30.635 ms 2023-09-23T04:52:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:04 smithi005 ceph-mon[97638]: pgmap v177: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 507 KiB/s wr, 100 op/s 2023-09-23T04:52:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:04 smithi118 ceph-mon[134297]: pgmap v177: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 507 KiB/s wr, 100 op/s 2023-09-23T04:52:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:04 smithi069 ceph-mon[133931]: pgmap v177: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 507 KiB/s wr, 100 op/s 2023-09-23T04:52:05.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 352850 5.70 MB/sec execute 161 sec latency 668.353 ms 2023-09-23T04:52:06.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 352934 5.67 MB/sec execute 162 sec latency 1615.498 ms 2023-09-23T04:52:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:06 smithi005 ceph-mon[97638]: pgmap v178: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 447 KiB/s wr, 91 op/s 2023-09-23T04:52:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:06 smithi118 ceph-mon[134297]: pgmap v178: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 447 KiB/s wr, 91 op/s 2023-09-23T04:52:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:06 smithi069 ceph-mon[133931]: pgmap v178: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 447 KiB/s wr, 91 op/s 2023-09-23T04:52:07.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 354941 5.71 MB/sec execute 163 sec latency 16.319 ms 2023-09-23T04:52:08.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 355933 5.69 MB/sec execute 164 sec latency 64.786 ms 2023-09-23T04:52:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:08 smithi005 ceph-mon[97638]: pgmap v179: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-09-23T04:52:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:08 smithi118 ceph-mon[134297]: pgmap v179: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-09-23T04:52:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:08 smithi069 ceph-mon[133931]: pgmap v179: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2023-09-23T04:52:09.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 356392 5.68 MB/sec execute 165 sec latency 574.110 ms 2023-09-23T04:52:10.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 356392 5.65 MB/sec execute 166 sec latency 1574.187 ms 2023-09-23T04:52:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:11 smithi005 ceph-mon[97638]: pgmap v180: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:52:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:11 smithi118 ceph-mon[134297]: pgmap v180: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:52:11.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 356476 5.61 MB/sec execute 167 sec latency 2528.720 ms 2023-09-23T04:52:11.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:11 smithi069 ceph-mon[133931]: pgmap v180: 129 pgs: 129 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:52:12.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 358457 5.66 MB/sec execute 168 sec latency 22.228 ms 2023-09-23T04:52:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:12 smithi005 ceph-mon[97638]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-09-23T04:52:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:12 smithi118 ceph-mon[134297]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-09-23T04:52:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:12 smithi069 ceph-mon[133931]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-09-23T04:52:13.947 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359500 5.64 MB/sec execute 169 sec latency 15.966 ms 2023-09-23T04:52:14.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359941 5.62 MB/sec execute 170 sec latency 588.166 ms 2023-09-23T04:52:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:14 smithi005 ceph-mon[97638]: pgmap v182: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2023-09-23T04:52:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:14 smithi118 ceph-mon[134297]: pgmap v182: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2023-09-23T04:52:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:14 smithi069 ceph-mon[133931]: pgmap v182: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2023-09-23T04:52:15.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 361775 5.66 MB/sec execute 171 sec latency 713.837 ms 2023-09-23T04:52:16.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 362706 5.64 MB/sec execute 172 sec latency 19.263 ms 2023-09-23T04:52:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:16 smithi005 ceph-mon[97638]: pgmap v183: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2023-09-23T04:52:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:16 smithi118 ceph-mon[134297]: pgmap v183: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2023-09-23T04:52:17.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:16 smithi069 ceph-mon[133931]: pgmap v183: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2023-09-23T04:52:17.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:52:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 -- 172.21.15.5:0/2749398531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 msgr2=0x7f757c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 --2- 172.21.15.5:0/2749398531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c10dda0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f757400b0a0 tx=0x7f757400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 -- 172.21.15.5:0/2749398531 shutdown_connections 2023-09-23T04:52:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 --2- 172.21.15.5:0/2749398531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c10dda0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 --2- 172.21.15.5:0/2749398531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 --2- 172.21.15.5:0/2749398531 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c069a30 0x7f757c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 -- 172.21.15.5:0/2749398531 >> 172.21.15.5:0/2749398531 conn(0x7f757c077ca0 msgr2=0x7f757c0780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 -- 172.21.15.5:0/2749398531 shutdown_connections 2023-09-23T04:52:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.639+0000 7f75827ff700 1 -- 172.21.15.5:0/2749398531 wait complete. 2023-09-23T04:52:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.640+0000 7f75827ff700 1 Processor -- start 2023-09-23T04:52:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.640+0000 7f75827ff700 1 -- start start 2023-09-23T04:52:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c19dc40 0x7f757c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c19e6c0 con 0x7f757c106220 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c19e830 con 0x7f757c069080 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f75827ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c19e9f0 con 0x7f757c19dc40 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58748/0 (socket says 172.21.15.5:58748) 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 -- 172.21.15.5:0/1126840984 learned_addr learned my addr 172.21.15.5:0/1126840984 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757b7fe700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f7580d9c700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c19dc40 0x7f757c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 -- 172.21.15.5:0/1126840984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c19dc40 msgr2=0x7f757c19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c19dc40 0x7f757c19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 -- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 msgr2=0x7f757c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.641+0000 7f757bfff700 1 -- 172.21.15.5:0/1126840984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574009d00 con 0x7f757c069080 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f757bfff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e9a0 tx=0x7f756c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f75797fa700 1 -- 172.21.15.5:0/1126840984 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c009790 con 0x7f757c069080 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f75797fa700 1 -- 172.21.15.5:0/1126840984 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756c010430 con 0x7f757c069080 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c1a5f50 con 0x7f757c069080 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f75797fa700 1 -- 172.21.15.5:0/1126840984 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c018680 con 0x7f757c069080 2023-09-23T04:52:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.642+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c1a63d0 con 0x7f757c069080 2023-09-23T04:52:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.645+0000 7f75797fa700 1 -- 172.21.15.5:0/1126840984 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f756c0105a0 con 0x7f757c069080 2023-09-23T04:52:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.646+0000 7f7596169b80 1 Processor -- start 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 -- start start 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ed47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f75797fa700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 0x7f75640a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4e3d460 con 0x563ab4ed4400 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4e2ec20 con 0x563ab4edfbd0 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7596169b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4e2c540 con 0x563ab4ed4d20 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f75797fa700 1 -- 172.21.15.5:0/1126840984 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f756c014070 con 0x7f757c069080 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f7580d9c700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 0x7f75640a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47504/0 (socket says 172.21.15.5:47504) 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f757bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ed47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.647+0000 7f757b7fe700 1 -- 172.21.15.5:0/2207728004 learned_addr learned my addr 172.21.15.5:0/2207728004 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f7580d9c700 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 -- 172.21.15.5:0/2207728004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 msgr2=0x563ab4ee1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 -- 172.21.15.5:0/2207728004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 msgr2=0x563ab4ed47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ed47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 -- 172.21.15.5:0/2207728004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab4e8a8d0 con 0x563ab4ed4d20 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f757b7fe700 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7570009cb0 tx=0x7f7570011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2424d02b492ad5cd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f7580d9c700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 0x7f75640a3bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75740062a0 tx=0x7f7574006000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f75627fc700 1 -- 172.21.15.5:0/2207728004 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570011d20 con 0x563ab4ed4d20 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f75627fc700 1 -- 172.21.15.5:0/2207728004 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7570011e80 con 0x563ab4ed4d20 2023-09-23T04:52:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.648+0000 7f75627fc700 1 -- 172.21.15.5:0/2207728004 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570005280 con 0x563ab4ed4d20 2023-09-23T04:52:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 -- 172.21.15.5:0/2207728004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 msgr2=0x563ab4edf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7570009cb0 tx=0x7f7570011440 comp rx=0 tx=0).stop 2023-09-23T04:52:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 -- 172.21.15.5:0/2207728004 shutdown_connections 2023-09-23T04:52:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ed47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 --2- 172.21.15.5:0/2207728004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edf690 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 -- 172.21.15.5:0/2207728004 >> 172.21.15.5:0/2207728004 conn(0x563ab4e49260 msgr2=0x563ab4ed0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 -- 172.21.15.5:0/2207728004 shutdown_connections 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.649+0000 7f7596169b80 1 -- 172.21.15.5:0/2207728004 wait complete. 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.650+0000 7f7596169b80 1 -- start start 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.650+0000 7f7596169b80 1 Processor -- start 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.650+0000 7f7596169b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4e27e30 con 0x563ab4ed4400 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4dc4710 con 0x563ab4edfbd0 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4dc43b0 con 0x563ab4ed4d20 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7596169b80 10 client.0 _ll_register_callbacks cb 0x563ab4c21800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7580d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54320/0 (socket says 172.21.15.5:54320) 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 -- 172.21.15.5:0/2727748692 learned_addr learned my addr 172.21.15.5:0/2727748692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47518/0 (socket says 172.21.15.5:47518) 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7580d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58762/0 (socket says 172.21.15.5:58762) 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 -- 172.21.15.5:0/2727748692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 msgr2=0x563ab4edc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 -- 172.21.15.5:0/2727748692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 msgr2=0x563ab4edd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 -- 172.21.15.5:0/2727748692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700098e0 con 0x563ab4ed4400 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f7580d9c700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.651+0000 7f757bfff700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f756c0cac50 tx=0x7f756c0a21f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6904272bfb408aaf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c09e320 con 0x563ab4ed4400 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f756c09a410 con 0x563ab4ed4400 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab4e20e10 con 0x563ab4ed4400 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f756c09e560 con 0x563ab4ed4400 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f7596169b80 10 client.17127 fetch_fsmap learned FSMap version 36 2023-09-23T04:52:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563ab4c52de0 con 0x563ab4ed4400 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ab4e20e10 con 0x563ab4ed4400 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f756c099900 con 0x563ab4ed4400 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f7596169b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 36 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.652+0000 7f7596169b80 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f7596169b80 20 client.17127 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4eef6c0 0x563ab4ef1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f7596169b80 4 client.17127 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab4e20b30 con 0x563ab4eef6c0 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f757b7fe700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4eef6c0 0x563ab4ef1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f757b7fe700 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4eef6c0 0x563ab4ef1ae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa1769ce4653a8 server_cookie=9fd8b63499e439d4 in_seq=0 out_seq=0 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.653+0000 7f755b7fe700 10 client.17127 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f755b7fe700 1 -- 172.21.15.5:0/2727748692 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ab4e20b30 con 0x563ab4eef6c0 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f755b7fe700 10 client.17127 handle_command_reply: tid=0 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 client.17127 shutdown 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 msgr2=0x563ab4edf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f756c0cac50 tx=0x7f756c0a21f0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 shutdown_connections 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4eef6c0 0x563ab4ef1ae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4edd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4edf670 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 --2- 172.21.15.5:0/2727748692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4edc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f755b7fe700 0 client.17127 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 >> 172.21.15.5:0/2727748692 conn(0x563ab4e49260 msgr2=0x563ab4ed0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 shutdown_connections 2023-09-23T04:52:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.654+0000 7f7596169b80 1 -- 172.21.15.5:0/2727748692 wait complete. 2023-09-23T04:52:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.655+0000 7f7596169b80 20 client.17127 trim_cache size 0 max 16384 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 Processor -- start 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 -- start start 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4400 0x563ab4ee1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4edfbd0 0x563ab4ed7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4ca8920 con 0x563ab4edfbd0 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4edae90 con 0x563ab4ed4d20 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7596169b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4ed87a0 con 0x563ab4ed4400 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f7580d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4edfbd0 0x563ab4ed7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58764/0 (socket says 172.21.15.5:58764) 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.673+0000 7f757b7fe700 1 -- 172.21.15.5:0/4191159276 learned_addr learned my addr 172.21.15.5:0/4191159276 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757bfff700 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4400 0x563ab4ee1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 -- 172.21.15.5:0/4191159276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4400 msgr2=0x563ab4ee1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4400 0x563ab4ee1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 -- 172.21.15.5:0/4191159276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4edfbd0 msgr2=0x563ab4ed7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4edfbd0 0x563ab4ed7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 -- 172.21.15.5:0/4191159276 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab4c3bc60 con 0x563ab4ed4d20 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f757b7fe700 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7570009cb0 tx=0x7f7570011840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a654e7c98741ade server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f7562ffd700 1 -- 172.21.15.5:0/4191159276 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700049d0 con 0x563ab4ed4d20 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f7562ffd700 1 -- 172.21.15.5:0/4191159276 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7570026a90 con 0x563ab4ed4d20 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.674+0000 7f7562ffd700 1 -- 172.21.15.5:0/4191159276 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570004050 con 0x563ab4ed4d20 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 -- 172.21.15.5:0/4191159276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 msgr2=0x563ab4edf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7570009cb0 tx=0x7f7570011840 comp rx=0 tx=0).stop 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 -- 172.21.15.5:0/4191159276 shutdown_connections 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4edfbd0 0x563ab4ed7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4400 0x563ab4ee1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 --2- 172.21.15.5:0/4191159276 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4ed4d20 0x563ab4edf210 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 -- 172.21.15.5:0/4191159276 >> 172.21.15.5:0/4191159276 conn(0x563ab4e49260 msgr2=0x563ab4eeb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 -- 172.21.15.5:0/4191159276 shutdown_connections 2023-09-23T04:52:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.675+0000 7f7596169b80 1 -- 172.21.15.5:0/4191159276 wait complete. 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 -- start start 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 Processor -- start 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ee26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4f42630 con 0x563ab4ed4400 2023-09-23T04:52:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4f427a0 con 0x563ab4edfbd0 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab4f42960 con 0x563ab4ed4d20 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.676+0000 7f7596169b80 10 client.0 _ll_register_callbacks cb 0x7f756c099f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ee26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47550/0 (socket says 172.21.15.5:47550) 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 -- 172.21.15.5:0/3915818188 learned_addr learned my addr 172.21.15.5:0/3915818188 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f7580d9c700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 -- 172.21.15.5:0/3915818188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 msgr2=0x563ab4ee3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 -- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 msgr2=0x563ab4ee26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ee26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 -- 172.21.15.5:0/3915818188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757001d040 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f757b7fe700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75700094c0 tx=0x7f7570005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb811c86e4eeb542 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.677+0000 7f755b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570011040 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f75700111a0 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab4ec8250 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7570011b20 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 10 client.27184 fetch_fsmap learned FSMap version 36 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563ab4c79250 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ab4ec8250 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7570003680 con 0x563ab4ed4d20 2023-09-23T04:52:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 10 client.27184 fetch_fsmap finished waiting for FSMap version 36 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 10 client.27184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 20 client.27184 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4f43cc0 0x563ab4f460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 4 client.27184 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.678+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563ab4c73f70 con 0x563ab4f43cc0 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f757bfff700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4f43cc0 0x563ab4f460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f757bfff700 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4f43cc0 0x563ab4f460e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c199a7259779ea server_cookie=44bd43e0b8759fb in_seq=0 out_seq=0 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f755b7fe700 10 client.27184 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f755b7fe700 1 -- 172.21.15.5:0/3915818188 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563ab4c73f70 con 0x563ab4f43cc0 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f755b7fe700 10 client.27184 handle_command_reply: tid=0 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.679+0000 7f7596169b80 1 client.27184 shutdown 2023-09-23T04:52:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 msgr2=0x563ab4ee2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75700094c0 tx=0x7f7570005ce0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 shutdown_connections 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563ab4f43cc0 0x563ab4f460e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ab4edfbd0 0x563ab4ee3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ab4ed4400 0x563ab4ee26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f755b7fe700 0 client.27184 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 --2- 172.21.15.5:0/3915818188 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563ab4ed4d20 0x563ab4ee2c30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 >> 172.21.15.5:0/3915818188 conn(0x563ab4e49260 msgr2=0x563ab4ed7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 shutdown_connections 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 1 -- 172.21.15.5:0/3915818188 wait complete. 2023-09-23T04:52:17.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.680+0000 7f7596169b80 20 client.27184 trim_cache size 0 max 16384 2023-09-23T04:52:17.686 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:52:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:52:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:52:17.686 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 msgr2=0x7f75640a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 0x7f75640a3bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75740062a0 tx=0x7f7574006000 comp rx=0 tx=0).stop 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 msgr2=0x7f757c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e9a0 tx=0x7f756c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 shutdown_connections 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75640a1710 0x7f75640a3bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f757c19dc40 0x7f757c19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f757c069080 0x7f757c1992f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 --2- 172.21.15.5:0/1126840984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c106220 0x7f757c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 >> 172.21.15.5:0/1126840984 conn(0x7f757c077ca0 msgr2=0x7f757c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 shutdown_connections 2023-09-23T04:52:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.681+0000 7f75827ff700 1 -- 172.21.15.5:0/1126840984 wait complete. 2023-09-23T04:52:17.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:52:17.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 -- 172.21.15.5:0/2505704837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9964101090 msgr2=0x7f9964101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 --2- 172.21.15.5:0/2505704837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9964101090 0x7f9964101470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9954009990 tx=0x7f9954009ca0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 -- 172.21.15.5:0/2505704837 shutdown_connections 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 --2- 172.21.15.5:0/2505704837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 0x7f996410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 --2- 172.21.15.5:0/2505704837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9964101090 0x7f9964101470 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 --2- 172.21.15.5:0/2505704837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101a40 0x7f9964105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 -- 172.21.15.5:0/2505704837 >> 172.21.15.5:0/2505704837 conn(0x7f99640fc8f0 msgr2=0x7f99640fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 -- 172.21.15.5:0/2505704837 shutdown_connections 2023-09-23T04:52:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.905+0000 7f9969dff700 1 -- 172.21.15.5:0/2505704837 wait complete. 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 Processor -- start 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 -- start start 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 0x7f99641996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f996419db20 0x7f996419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996419e5a0 con 0x7f99641061b0 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996419e710 con 0x7f996419db20 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.906+0000 7f9969dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996419e8d0 con 0x7f9964101090 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f9962ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 0x7f99641996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47564/0 (socket says 172.21.15.5:47564) 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 -- 172.21.15.5:0/2535100304 learned_addr learned my addr 172.21.15.5:0/2535100304 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f9963fff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f996419db20 0x7f996419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 -- 172.21.15.5:0/2535100304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f996419db20 msgr2=0x7f996419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f996419db20 0x7f996419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 -- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 msgr2=0x7f99641996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 0x7f99641996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 -- 172.21.15.5:0/2535100304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540095f0 con 0x7f9964101090 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.907+0000 7f99637fe700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9954005910 tx=0x7f9954004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.908+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995401d070 con 0x7f9964101090 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.908+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99641a5dd0 con 0x7f9964101090 2023-09-23T04:52:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.908+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9954022470 con 0x7f9964101090 2023-09-23T04:52:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.908+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954017650 con 0x7f9964101090 2023-09-23T04:52:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.908+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99641a6240 con 0x7f9964101090 2023-09-23T04:52:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.909+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99640680f0 con 0x7f9964101090 2023-09-23T04:52:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.911+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f995400fad0 con 0x7f9964101090 2023-09-23T04:52:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.912+0000 7f9960ff9700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 0x7f994c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.912+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f99540ceb00 con 0x7f9964101090 2023-09-23T04:52:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.912+0000 7f9962ffd700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 0x7f994c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.913+0000 7f9962ffd700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 0x7f994c0a3bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f996419a790 tx=0x7f995800b410 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:17.914+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f9954097a00 con 0x7f9964101090 2023-09-23T04:52:17.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363483 5.64 MB/sec execute 173 sec latency 527.264 ms 2023-09-23T04:52:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.174+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9964101a40 con 0x7f9964101090 2023-09-23T04:52:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.175+0000 7f9960ff9700 1 -- 172.21.15.5:0/2535100304 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f995409b020 con 0x7f9964101090 2023-09-23T04:52:18.175 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:52:18.175 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:52:17.227016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 msgr2=0x7f994c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 0x7f994c0a3bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f996419a790 tx=0x7f995800b410 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 msgr2=0x7f9964199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9954005910 tx=0x7f9954004730 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 shutdown_connections 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f994c0a1710 0x7f994c0a3bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f996419db20 0x7f996419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9964101090 0x7f9964199180 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 --2- 172.21.15.5:0/2535100304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99641061b0 0x7f99641996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 >> 172.21.15.5:0/2535100304 conn(0x7f99640fc8f0 msgr2=0x7f996410d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 shutdown_connections 2023-09-23T04:52:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.178+0000 7f9969dff700 1 -- 172.21.15.5:0/2535100304 wait complete. 2023-09-23T04:52:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2023-09-23T04:52:18.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:52:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 -- 172.21.15.5:0/1797174878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa724101a80 msgr2=0x7fa724105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/1797174878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa724101a80 0x7fa724105ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7100098f0 tx=0x7fa710009c00 comp rx=0 tx=0).stop 2023-09-23T04:52:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 -- 172.21.15.5:0/1797174878 shutdown_connections 2023-09-23T04:52:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/1797174878 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241061f0 0x7fa72410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/1797174878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241010d0 0x7fa7241014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/1797174878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa724101a80 0x7fa724105ac0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 -- 172.21.15.5:0/1797174878 >> 172.21.15.5:0/1797174878 conn(0x7fa7240fc990 msgr2=0x7fa7240fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 -- 172.21.15.5:0/1797174878 shutdown_connections 2023-09-23T04:52:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.402+0000 7fa72b1ff700 1 -- 172.21.15.5:0/1797174878 wait complete. 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.403+0000 7fa72b1ff700 1 Processor -- start 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.403+0000 7fa72b1ff700 1 -- start start 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241010d0 0x7fa724199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241061f0 0x7fa7241996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419e5b0 con 0x7fa7241061f0 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419e720 con 0x7fa72419db30 2023-09-23T04:52:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72b1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419e8e0 con 0x7fa7241010d0 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa723fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241061f0 0x7fa7241996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58794/0 (socket says 172.21.15.5:58794) 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 -- 172.21.15.5:0/81975140 learned_addr learned my addr 172.21.15.5:0/81975140 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa728f9b700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241010d0 0x7fa724199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 -- 172.21.15.5:0/81975140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241010d0 msgr2=0x7fa724199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241010d0 0x7fa724199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 -- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241061f0 msgr2=0x7fa7241996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241061f0 0x7fa7241996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.404+0000 7fa72979c700 1 -- 172.21.15.5:0/81975140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa710009520 con 0x7fa72419db30 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa72979c700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ef30 tx=0x7fa71c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa721ffb700 1 -- 172.21.15.5:0/81975140 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c018070 con 0x7fa72419db30 2023-09-23T04:52:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa721ffb700 1 -- 172.21.15.5:0/81975140 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71c00f040 con 0x7fa72419db30 2023-09-23T04:52:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7241a5de0 con 0x7fa72419db30 2023-09-23T04:52:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa721ffb700 1 -- 172.21.15.5:0/81975140 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00ccf0 con 0x7fa72419db30 2023-09-23T04:52:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.405+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7241a62c0 con 0x7fa72419db30 2023-09-23T04:52:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.408+0000 7fa721ffb700 1 -- 172.21.15.5:0/81975140 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa71c003a00 con 0x7fa72419db30 2023-09-23T04:52:18.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa73ea34b80 1 Processor -- start 2023-09-23T04:52:18.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa721ffb700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 0x7fa7140a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa721ffb700 1 -- 172.21.15.5:0/81975140 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa71c0cd1f0 con 0x7fa72419db30 2023-09-23T04:52:18.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa73ea34b80 1 -- start start 2023-09-23T04:52:18.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa728f9b700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 0x7fa7140a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b685b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.410+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b685bbd0 0x5562b685dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b67b9460 con 0x5562b685bbd0 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b67aac20 con 0x5562b6850400 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b67a8540 con 0x5562b6850d20 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 0x7fa7140a3c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa718005fd0 tx=0x7fa718009310 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58798/0 (socket says 172.21.15.5:58798) 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 -- 172.21.15.5:0/1284956218 learned_addr learned my addr 172.21.15.5:0/1284956218 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa72979c700 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b685bbd0 0x5562b685dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa723fff700 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b685b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 -- 172.21.15.5:0/1284956218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 msgr2=0x5562b685b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b685b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 -- 172.21.15.5:0/1284956218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b685bbd0 msgr2=0x5562b685dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b685bbd0 0x5562b685dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.411+0000 7fa728f9b700 1 -- 172.21.15.5:0/1284956218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562b68068d0 con 0x5562b6850400 2023-09-23T04:52:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.412+0000 7fa728f9b700 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa718004140 tx=0x7fa718014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91ed469299ee29c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.412+0000 7fa70effd700 1 -- 172.21.15.5:0/1284956218 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718003b10 con 0x5562b6850400 2023-09-23T04:52:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.412+0000 7fa70effd700 1 -- 172.21.15.5:0/1284956218 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa718003c70 con 0x5562b6850400 2023-09-23T04:52:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.412+0000 7fa70effd700 1 -- 172.21.15.5:0/1284956218 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d610 con 0x5562b6850400 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- 172.21.15.5:0/1284956218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 msgr2=0x5562b68507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa718004140 tx=0x7fa718014000 comp rx=0 tx=0).stop 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- 172.21.15.5:0/1284956218 shutdown_connections 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b685bbd0 0x5562b685dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b68507e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/1284956218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b685b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- 172.21.15.5:0/1284956218 >> 172.21.15.5:0/1284956218 conn(0x5562b67c5260 msgr2=0x5562b684c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- 172.21.15.5:0/1284956218 shutdown_connections 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- 172.21.15.5:0/1284956218 wait complete. 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.413+0000 7fa73ea34b80 1 -- start start 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.414+0000 7fa73ea34b80 1 Processor -- start 2023-09-23T04:52:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.414+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 0x5562b6858900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b685bbd0 0x5562b6859040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b67a3e30 con 0x5562b6850d20 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b6740710 con 0x5562b6850400 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b67403b0 con 0x5562b685bbd0 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa73ea34b80 10 client.0 _ll_register_callbacks cb 0x5562b659d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa723fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 0x5562b6858900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58804/0 (socket says 172.21.15.5:58804) 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 -- 172.21.15.5:0/3293112455 learned_addr learned my addr 172.21.15.5:0/3293112455 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa72979c700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b685bbd0 0x5562b6859040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 -- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b685bbd0 msgr2=0x5562b6859040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b685bbd0 0x5562b6859040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 -- 172.21.15.5:0/3293112455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 msgr2=0x5562b6858900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 0x5562b6858900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa728f9b700 1 -- 172.21.15.5:0/3293112455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa718017070 con 0x5562b6850400 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.415+0000 7fa723fff700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 0x5562b6858900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa728f9b700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa71800a010 tx=0x7fa718009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49e49ac15780c5c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa703fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:52:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71802b070 con 0x5562b6850400 2023-09-23T04:52:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562b6844250 con 0x5562b6850400 2023-09-23T04:52:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa718005050 con 0x5562b6850400 2023-09-23T04:52:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa718003b10 con 0x5562b6850400 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa73ea34b80 10 client.27155 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.416+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5562b65cede0 con 0x5562b6850400 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562b6844250 con 0x5562b6850400 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa718024020 con 0x5562b6850400 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 10 client.27155 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 10 client.27155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 20 client.27155 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b686b510 0x5562b686d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 4 client.27155 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562b679ce10 con 0x5562b686b510 2023-09-23T04:52:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.417+0000 7fa723fff700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b686b510 0x5562b686d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.418+0000 7fa723fff700 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b686b510 0x5562b686d900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=943e46653808ad72 server_cookie=d0fe535b0e2f102b in_seq=0 out_seq=0 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.418+0000 7fa703fff700 10 client.27155 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa703fff700 1 -- 172.21.15.5:0/3293112455 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5562b679ce10 con 0x5562b686b510 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa703fff700 10 client.27155 handle_command_reply: tid=0 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 client.27155 shutdown 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 msgr2=0x5562b685b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa71800a010 tx=0x7fa718009ab0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 shutdown_connections 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b685bbd0 0x5562b6859040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b6850400 0x5562b685b680 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b686b510 0x5562b686d900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/3293112455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850d20 0x5562b6858900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa703fff700 0 client.27155 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:52:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 >> 172.21.15.5:0/3293112455 conn(0x5562b67c5260 msgr2=0x5562b684c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 shutdown_connections 2023-09-23T04:52:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 1 -- 172.21.15.5:0/3293112455 wait complete. 2023-09-23T04:52:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.419+0000 7fa73ea34b80 20 client.27155 trim_cache size 0 max 16384 2023-09-23T04:52:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 Processor -- start 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 -- start start 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b68532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b685b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b6624920 con 0x5562b6850400 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b68537e0 con 0x5562b685bbd0 2023-09-23T04:52:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b6868010 con 0x5562b6850d20 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54408/0 (socket says 172.21.15.5:54408) 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa728f9b700 1 -- 172.21.15.5:0/634612794 learned_addr learned my addr 172.21.15.5:0/634612794 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa72979c700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b685b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.433+0000 7fa723fff700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b68532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 -- 172.21.15.5:0/634612794 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 msgr2=0x5562b68532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b68532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 -- 172.21.15.5:0/634612794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 msgr2=0x5562b685b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b685b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 -- 172.21.15.5:0/634612794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562b65b7c60 con 0x5562b6850400 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa72979c700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b685b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa728f9b700 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa718005950 tx=0x7fa718009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9f4a2772fefc23 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa70f7fe700 1 -- 172.21.15.5:0/634612794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71802b070 con 0x5562b6850400 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa70f7fe700 1 -- 172.21.15.5:0/634612794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71801e8c0 con 0x5562b6850400 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.434+0000 7fa70f7fe700 1 -- 172.21.15.5:0/634612794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180055a0 con 0x5562b6850400 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- 172.21.15.5:0/634612794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 msgr2=0x5562b6858900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa718005950 tx=0x7fa718009380 comp rx=0 tx=0).stop 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- 172.21.15.5:0/634612794 shutdown_connections 2023-09-23T04:52:18.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b685b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6858900 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/634612794 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b68532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- 172.21.15.5:0/634612794 >> 172.21.15.5:0/634612794 conn(0x5562b67c5260 msgr2=0x5562b68581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- 172.21.15.5:0/634612794 shutdown_connections 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- 172.21.15.5:0/634612794 wait complete. 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 -- start start 2023-09-23T04:52:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.435+0000 7fa73ea34b80 1 Processor -- start 2023-09-23T04:52:18.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b6862070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b68be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b68627a0 con 0x5562b6850400 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b68be870 con 0x5562b685bbd0 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562b68be9b0 con 0x5562b6850d20 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa73ea34b80 10 client.0 _ll_register_callbacks cb 0x7fa71c004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa728f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54418/0 (socket says 172.21.15.5:54418) 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa728f9b700 1 -- 172.21.15.5:0/2364348875 learned_addr learned my addr 172.21.15.5:0/2364348875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa72979c700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b68be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.436+0000 7fa723fff700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b6862070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 -- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 msgr2=0x5562b6862070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b6862070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 -- 172.21.15.5:0/2364348875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 msgr2=0x5562b68be330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b68be330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 -- 172.21.15.5:0/2364348875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562b65cede0 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa723fff700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b6862070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa728f9b700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa718004e30 tx=0x7fa718009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19147c20bbabd97 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa703fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71802b040 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa718003c70 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa718043020 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa718005430 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa73ea34b80 10 client.17157 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.437+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5562b65f5250 con 0x5562b6850400 2023-09-23T04:52:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa718003c70 con 0x5562b6850400 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa718024070 con 0x5562b6850400 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 10 client.17157 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 10 client.17157 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 20 client.17157 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b68bfc70 0x5562b68c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 4 client.17157 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5562b679cb30 con 0x5562b68bfc70 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.438+0000 7fa723fff700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b68bfc70 0x5562b68c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa723fff700 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b68bfc70 0x5562b68c2060 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b09fb5b4c849377 server_cookie=ff23d0b4942acb2b in_seq=0 out_seq=0 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa703fff700 10 client.17157 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:52:18.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa703fff700 1 -- 172.21.15.5:0/2364348875 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5562b679cb30 con 0x5562b68bfc70 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa703fff700 10 client.17157 handle_command_reply: tid=0 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 client.17157 shutdown 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 msgr2=0x5562b6861b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa718004e30 tx=0x7fa718009a90 comp rx=0 tx=0).stop 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 shutdown_connections 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5562b685bbd0 0x5562b68be330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.439+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5562b68bfc70 0x5562b68c2060 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562b6850400 0x5562b6861b30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa703fff700 0 client.17157 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 1 --2- 172.21.15.5:0/2364348875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5562b6850d20 0x5562b6862070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 >> 172.21.15.5:0/2364348875 conn(0x5562b67c5260 msgr2=0x5562b684c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 shutdown_connections 2023-09-23T04:52:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 1 -- 172.21.15.5:0/2364348875 wait complete. 2023-09-23T04:52:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa73ea34b80 20 client.17157 trim_cache size 0 max 16384 2023-09-23T04:52:18.445 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:52:18.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 msgr2=0x7fa7140a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 0x7fa7140a3c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa718005fd0 tx=0x7fa718009310 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 msgr2=0x7fa72419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ef30 tx=0x7fa71c00c3a0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 shutdown_connections 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa7140a1790 0x7fa7140a3c50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa72419db30 0x7fa72419dfb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa7241010d0 0x7fa724199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 --2- 172.21.15.5:0/81975140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7241061f0 0x7fa7241996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.440+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 >> 172.21.15.5:0/81975140 conn(0x7fa7240fc990 msgr2=0x7fa7240fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.441+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 shutdown_connections 2023-09-23T04:52:18.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.441+0000 7fa72b1ff700 1 -- 172.21.15.5:0/81975140 wait complete. 2023-09-23T04:52:18.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/877544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481010d0 msgr2=0x7f1e481014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/877544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481010d0 0x7f1e481014b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009990 tx=0x7f1e38009ca0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/877544 shutdown_connections 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/877544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481061f0 0x7f1e4810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/877544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481010d0 0x7f1e481014b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/877544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e48101a80 0x7f1e48105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/877544 >> 172.21.15.5:0/877544 conn(0x7f1e480fc990 msgr2=0x7f1e480fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/877544 shutdown_connections 2023-09-23T04:52:18.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.666+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/877544 wait complete. 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.667+0000 7f1e4d9ff700 1 Processor -- start 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.667+0000 7f1e4d9ff700 1 -- start start 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.667+0000 7f1e4d9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e4d9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e4d9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e4819b980 0x7f1e4819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e4d9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4819c400 con 0x7f1e481061f0 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e4d9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4819c570 con 0x7f1e4819b980 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e4d9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4819c730 con 0x7f1e481010d0 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47644/0 (socket says 172.21.15.5:47644) 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54436/0 (socket says 172.21.15.5:54436) 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 -- 172.21.15.5:0/774827353 learned_addr learned my addr 172.21.15.5:0/774827353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e477fe700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e4819b980 0x7f1e4819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 -- 172.21.15.5:0/774827353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e4819b980 msgr2=0x7f1e4819be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e4819b980 0x7f1e4819be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 -- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 msgr2=0x7f1e48197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 -- 172.21.15.5:0/774827353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380095f0 con 0x7f1e481010d0 2023-09-23T04:52:18.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.668+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005910 tx=0x7f1e38005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.669+0000 7f1e4c9fd700 1 -- 172.21.15.5:0/774827353 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3801d070 con 0x7f1e481010d0 2023-09-23T04:52:18.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.669+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e481a3c30 con 0x7f1e481010d0 2023-09-23T04:52:18.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.669+0000 7f1e4c9fd700 1 -- 172.21.15.5:0/774827353 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e38022470 con 0x7f1e481010d0 2023-09-23T04:52:18.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.669+0000 7f1e4c9fd700 1 -- 172.21.15.5:0/774827353 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38017650 con 0x7f1e481010d0 2023-09-23T04:52:18.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.669+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e481a4050 con 0x7f1e481010d0 2023-09-23T04:52:18.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.673+0000 7f1e4c9fd700 1 -- 172.21.15.5:0/774827353 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f1e380225e0 con 0x7f1e481010d0 2023-09-23T04:52:18.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 Processor -- start 2023-09-23T04:52:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 -- start start 2023-09-23T04:52:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e4c9fd700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 0x7f1e340a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f6476f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f64771fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f646cd460 con 0x558f6476fbd0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f646bec20 con 0x558f64764d20 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f646bc540 con 0x558f64764400 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e4c9fd700 1 -- 172.21.15.5:0/774827353 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1e380cea70 con 0x7f1e481010d0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47652/0 (socket says 172.21.15.5:47652) 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.674+0000 7f1e46ffd700 1 -- 172.21.15.5:0/3672638769 learned_addr learned my addr 172.21.15.5:0/3672638769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e477fe700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 0x7f1e340a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f6476f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e477fe700 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f64771fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 -- 172.21.15.5:0/3672638769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 msgr2=0x558f6476f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f6476f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 -- 172.21.15.5:0/3672638769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 msgr2=0x558f64771fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f64771fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 -- 172.21.15.5:0/3672638769 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f6471a8d0 con 0x558f64764400 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e38097b30 tx=0x7f1e38009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a06e9552f67a692 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e477fe700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 0x7f1e340a3bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009ec0 tx=0x7f1e40009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e3dffb700 1 -- 172.21.15.5:0/3672638769 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809af90 con 0x558f64764400 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e3dffb700 1 -- 172.21.15.5:0/3672638769 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e380650a0 con 0x558f64764400 2023-09-23T04:52:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.675+0000 7f1e3dffb700 1 -- 172.21.15.5:0/3672638769 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3806dab0 con 0x558f64764400 2023-09-23T04:52:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 -- 172.21.15.5:0/3672638769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 msgr2=0x558f647647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e38097b30 tx=0x7f1e38009560 comp rx=0 tx=0).stop 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 -- 172.21.15.5:0/3672638769 shutdown_connections 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f64771fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647647e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/3672638769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f6476f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 -- 172.21.15.5:0/3672638769 >> 172.21.15.5:0/3672638769 conn(0x558f646d9260 msgr2=0x558f64760790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 -- 172.21.15.5:0/3672638769 shutdown_connections 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.676+0000 7f1e612bcb80 1 -- 172.21.15.5:0/3672638769 wait complete. 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 -- start start 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 Processor -- start 2023-09-23T04:52:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f64774900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f6476fbd0 0x558f64774e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f646b7e30 con 0x558f64764400 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f64654710 con 0x558f6476fbd0 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.677+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f646543b0 con 0x558f64764d20 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e612bcb80 10 client.0 _ll_register_callbacks cb 0x558f644b1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54448/0 (socket says 172.21.15.5:54448) 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 -- 172.21.15.5:0/47535364 learned_addr learned my addr 172.21.15.5:0/47535364 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f64774900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e477fe700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f6476fbd0 0x558f64774e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 -- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 msgr2=0x558f64774900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f64774900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 -- 172.21.15.5:0/47535364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f6476fbd0 msgr2=0x558f64774e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f6476fbd0 0x558f64774e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 -- 172.21.15.5:0/47535364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e481a4050 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e380a3400 tx=0x7f1e38022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb61d6a5db225f18 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.678+0000 7f1e26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809af90 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1e3809bd30 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f64758250 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e3806dc80 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 10 client.17169 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558f644e2de0 con 0x558f64764400 2023-09-23T04:52:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f64758250 con 0x558f64764400 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1e380a34a0 con 0x558f64764400 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 10 client.17169 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 10 client.17169 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 20 client.17169 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f64783260 0x558f64785650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 4 client.17169 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.679+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f646b0e10 con 0x558f64783260 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.680+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f64783260 0x558f64785650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.680+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f64783260 0x558f64785650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf760e0be15398c3 server_cookie=6c270554bb037f1b in_seq=0 out_seq=0 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.680+0000 7f1e26ffd700 10 client.17169 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e26ffd700 1 -- 172.21.15.5:0/47535364 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f646b0e10 con 0x558f64783260 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e26ffd700 10 client.17169 handle_command_reply: tid=0 2023-09-23T04:52:18.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 client.17169 shutdown 2023-09-23T04:52:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 msgr2=0x558f647743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e380a3400 tx=0x7f1e38022c90 comp rx=0 tx=0).stop 2023-09-23T04:52:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 shutdown_connections 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f6476fbd0 0x558f64774e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f64783260 0x558f64785650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f64764400 0x558f647743c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/47535364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f64774900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e26ffd700 0 client.17169 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 >> 172.21.15.5:0/47535364 conn(0x558f646d9260 msgr2=0x558f64760380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.681+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 shutdown_connections 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.682+0000 7f1e612bcb80 1 -- 172.21.15.5:0/47535364 wait complete. 2023-09-23T04:52:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.682+0000 7f1e612bcb80 20 client.17169 trim_cache size 0 max 16384 2023-09-23T04:52:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.695+0000 7f1e612bcb80 1 Processor -- start 2023-09-23T04:52:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.695+0000 7f1e612bcb80 1 -- start start 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.695+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f6476a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f6476bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f64538920 con 0x558f6476fbd0 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6476f860 con 0x558f64764400 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f6476abc0 con 0x558f64764d20 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f6476bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f6476bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54458/0 (socket says 172.21.15.5:54458) 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f6476a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58880/0 (socket says 172.21.15.5:58880) 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e477fe700 1 -- 172.21.15.5:0/2690478271 learned_addr learned my addr 172.21.15.5:0/2690478271 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 -- 172.21.15.5:0/2690478271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 msgr2=0x558f6476a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f6476a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 -- 172.21.15.5:0/2690478271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 msgr2=0x558f6476bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f6476bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.696+0000 7f1e46ffd700 1 -- 172.21.15.5:0/2690478271 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f6475c480 con 0x558f64764400 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e38097b30 tx=0x7f1e38009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc79ebd41f24f42 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e3e7fc700 1 -- 172.21.15.5:0/2690478271 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809add0 con 0x558f64764400 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e3e7fc700 1 -- 172.21.15.5:0/2690478271 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e3809af90 con 0x558f64764400 2023-09-23T04:52:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e3e7fc700 1 -- 172.21.15.5:0/2690478271 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809f780 con 0x558f64764400 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 -- 172.21.15.5:0/2690478271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 msgr2=0x558f6476a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e38097b30 tx=0x7f1e38009580 comp rx=0 tx=0).stop 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 -- 172.21.15.5:0/2690478271 shutdown_connections 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f6476bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764400 0x558f6476a0f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/2690478271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764d20 0x558f6476a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 -- 172.21.15.5:0/2690478271 >> 172.21.15.5:0/2690478271 conn(0x558f646d9260 msgr2=0x558f6475ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.697+0000 7f1e612bcb80 1 -- 172.21.15.5:0/2690478271 shutdown_connections 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.698+0000 7f1e612bcb80 1 -- 172.21.15.5:0/2690478271 wait complete. 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.698+0000 7f1e612bcb80 1 -- start start 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.698+0000 7f1e612bcb80 1 Processor -- start 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f64776700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f647d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f647d6580 con 0x558f6476fbd0 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f647d66c0 con 0x558f64764d20 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f647d6830 con 0x558f64764400 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e612bcb80 10 client.0 _ll_register_callbacks cb 0x7f1e30009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f647d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47692/0 (socket says 172.21.15.5:47692) 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f64776700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 -- 172.21.15.5:0/780174127 learned_addr learned my addr 172.21.15.5:0/780174127 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 -- 172.21.15.5:0/780174127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 msgr2=0x558f64776700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f64776700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 -- 172.21.15.5:0/780174127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 msgr2=0x558f647d6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f647d6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e46ffd700 1 -- 172.21.15.5:0/780174127 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f644e2de0 con 0x558f64764400 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e477fe700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f647d6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.699+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f64776700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e46ffd700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800f740 tx=0x7f1e380b3ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40eb8f4fce6e64ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3809ad80 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e380650a0 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f1e3809bd30 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e380a3930 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e612bcb80 10 client.27226 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558f644cbc60 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.700+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e380650a0 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1e3809a590 con 0x558f64764400 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 10 client.27226 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 10 client.27226 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 20 client.27226 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f647d7a60 0x558f647d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 4 client.27226 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f646b0b30 con 0x558f647d7a60 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.701+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f647d7a60 0x558f647d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e3ffff700 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f647d7a60 0x558f647d9e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=949043904448dd34 server_cookie=a7937090e33b75ff in_seq=0 out_seq=0 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e26ffd700 10 client.27226 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e26ffd700 1 -- 172.21.15.5:0/780174127 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f646b0b30 con 0x558f647d7a60 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e26ffd700 10 client.27226 handle_command_reply: tid=0 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 client.27226 shutdown 2023-09-23T04:52:18.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 msgr2=0x558f647761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800f740 tx=0x7f1e380b3ec0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 shutdown_connections 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x558f647d7a60 0x558f647d9e50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f6476fbd0 0x558f647d6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558f64764400 0x558f647761c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 --2- 172.21.15.5:0/780174127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558f64764d20 0x558f64776700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e26ffd700 0 client.27226 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 >> 172.21.15.5:0/780174127 conn(0x558f646d9260 msgr2=0x558f6475f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 shutdown_connections 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 1 -- 172.21.15.5:0/780174127 wait complete. 2023-09-23T04:52:18.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.702+0000 7f1e612bcb80 20 client.27226 trim_cache size 0 max 16384 2023-09-23T04:52:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:52:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 msgr2=0x7f1e340a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 0x7f1e340a3bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009ec0 tx=0x7f1e40009920 comp rx=0 tx=0).stop 2023-09-23T04:52:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 msgr2=0x7f1e48196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1e38005910 tx=0x7f1e38005740 comp rx=0 tx=0).stop 2023-09-23T04:52:18.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 shutdown_connections 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1e340a1710 0x7f1e340a3bd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e4819b980 0x7f1e4819be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1e481010d0 0x7f1e48196fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 --2- 172.21.15.5:0/774827353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e481061f0 0x7f1e48197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 >> 172.21.15.5:0/774827353 conn(0x7f1e480fc990 msgr2=0x7f1e480fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 shutdown_connections 2023-09-23T04:52:18.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.703+0000 7f1e4d9ff700 1 -- 172.21.15.5:0/774827353 wait complete. 2023-09-23T04:52:18.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:52:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 -- 172.21.15.5:0/1213906955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1061f0 msgr2=0x7f001c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 --2- 172.21.15.5:0/1213906955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1061f0 0x7f001c10dd70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f001400b0a0 tx=0x7f001400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 -- 172.21.15.5:0/1213906955 shutdown_connections 2023-09-23T04:52:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 --2- 172.21.15.5:0/1213906955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1061f0 0x7f001c10dd70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 --2- 172.21.15.5:0/1213906955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c1010d0 0x7f001c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 --2- 172.21.15.5:0/1213906955 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c101a80 0x7f001c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 -- 172.21.15.5:0/1213906955 >> 172.21.15.5:0/1213906955 conn(0x7f001c0fc990 msgr2=0x7f001c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 -- 172.21.15.5:0/1213906955 shutdown_connections 2023-09-23T04:52:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.928+0000 7f00223ff700 1 -- 172.21.15.5:0/1213906955 wait complete. 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.930+0000 7f00223ff700 1 Processor -- start 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.930+0000 7f00223ff700 1 -- start start 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.930+0000 7f00223ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.930+0000 7f00223ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1061f0 0x7f001c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.930+0000 7f00223ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f00223ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a59a0 con 0x7f001c1010d0 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f00223ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a5b10 con 0x7f001c073ba0 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f00223ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1a5cd0 con 0x7f001c1061f0 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54472/0 (socket says 172.21.15.5:54472) 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 -- 172.21.15.5:0/2869802792 learned_addr learned my addr 172.21.15.5:0/2869802792 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f0013fff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1061f0 0x7f001c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f002099c700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 -- 172.21.15.5:0/2869802792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1061f0 msgr2=0x7f001c073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1061f0 0x7f001c073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 -- 172.21.15.5:0/2869802792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 msgr2=0x7f001c1a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 -- 172.21.15.5:0/2869802792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009520 con 0x7f001c1010d0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.931+0000 7f001bfff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f000c00eb30 tx=0x7f000c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.932+0000 7f0019ffb700 1 -- 172.21.15.5:0/2869802792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c009880 con 0x7f001c1010d0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.932+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0014009d00 con 0x7f001c1010d0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.932+0000 7f0019ffb700 1 -- 172.21.15.5:0/2869802792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c0044e0 con 0x7f001c1010d0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.932+0000 7f0019ffb700 1 -- 172.21.15.5:0/2869802792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c003e90 con 0x7f001c1010d0 2023-09-23T04:52:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.932+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a62f0 con 0x7f001c1010d0 2023-09-23T04:52:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.935+0000 7f0019ffb700 1 -- 172.21.15.5:0/2869802792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f000c004000 con 0x7f001c1010d0 2023-09-23T04:52:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 Processor -- start 2023-09-23T04:52:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 -- start start 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb30e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0019ffb700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 0x7f00080a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb31bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb277460 con 0x556ccb30e400 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb268c20 con 0x556ccb30ed20 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.937+0000 7f0035caab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb266540 con 0x556ccb319bd0 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f002099c700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 0x7f00080a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0019ffb700 1 -- 172.21.15.5:0/2869802792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f000c014070 con 0x7f001c1010d0 2023-09-23T04:52:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f001bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb30e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58940/0 (socket says 172.21.15.5:58940) 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 -- 172.21.15.5:0/727332480 learned_addr learned my addr 172.21.15.5:0/727332480 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f002099c700 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb31bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 -- 172.21.15.5:0/727332480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 msgr2=0x556ccb31bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb31bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 -- 172.21.15.5:0/727332480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 msgr2=0x556ccb30e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb30e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.938+0000 7f0013fff700 1 -- 172.21.15.5:0/727332480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ccb2c48d0 con 0x556ccb30ed20 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.939+0000 7f002099c700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 0x7f00080a3bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f001400b070 tx=0x7f0014007c80 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.939+0000 7f0013fff700 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0004009c30 tx=0x7f0004009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca41002c39e3f0ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.939+0000 7f00127fc700 1 -- 172.21.15.5:0/727332480 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004011d10 con 0x556ccb30ed20 2023-09-23T04:52:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.939+0000 7f00127fc700 1 -- 172.21.15.5:0/727332480 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0004011e70 con 0x556ccb30ed20 2023-09-23T04:52:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.939+0000 7f00127fc700 1 -- 172.21.15.5:0/727332480 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004005220 con 0x556ccb30ed20 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- 172.21.15.5:0/727332480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 msgr2=0x556ccb319690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0004009c30 tx=0x7f0004009f40 comp rx=0 tx=0).stop 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- 172.21.15.5:0/727332480 shutdown_connections 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb31bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb30e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 --2- 172.21.15.5:0/727332480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb319690 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- 172.21.15.5:0/727332480 >> 172.21.15.5:0/727332480 conn(0x556ccb283260 msgr2=0x556ccb30a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- 172.21.15.5:0/727332480 shutdown_connections 2023-09-23T04:52:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- 172.21.15.5:0/727332480 wait complete. 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.940+0000 7f0035caab80 1 -- start start 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 Processor -- start 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb316840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb261e30 con 0x556ccb319bd0 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb1fe710 con 0x556ccb30ed20 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb1fe3b0 con 0x556ccb30e400 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f0035caab80 10 client.0 _ll_register_callbacks cb 0x556ccb05b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f001bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f002099c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f001bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47718/0 (socket says 172.21.15.5:47718) 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.941+0000 7f002099c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54490/0 (socket says 172.21.15.5:54490) 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f001bfff700 1 -- 172.21.15.5:0/676109574 learned_addr learned my addr 172.21.15.5:0/676109574 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f0013fff700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb316840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 -- 172.21.15.5:0/676109574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 msgr2=0x556ccb3195c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 -- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 msgr2=0x556ccb316840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb316840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 -- 172.21.15.5:0/676109574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00040098e0 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f001bfff700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f002099c700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0014009fd0 tx=0x7f001400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7712402f42209e9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7efffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001400e070 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f001401f850 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.942+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ccb302250 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0014009b80 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 10 client.17187 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556ccb08cde0 con 0x556ccb319bd0 2023-09-23T04:52:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ccb302250 con 0x556ccb319bd0 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0014027030 con 0x556ccb319bd0 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 10 client.17187 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 20 client.17187 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb329640 0x556ccb32ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 4 client.17187 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.943+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ccb25ae10 con 0x556ccb329640 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.944+0000 7f001bfff700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb329640 0x556ccb32ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.944+0000 7f001bfff700 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb329640 0x556ccb32ba60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d60ac096dfac5cb1 server_cookie=d961c15d5c382cff in_seq=0 out_seq=0 2023-09-23T04:52:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.944+0000 7efffb7fe700 10 client.17187 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:52:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7efffb7fe700 1 -- 172.21.15.5:0/676109574 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ccb25ae10 con 0x556ccb329640 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7efffb7fe700 10 client.17187 handle_command_reply: tid=0 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 client.17187 shutdown 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 msgr2=0x556ccb316ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0014009fd0 tx=0x7f001400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 shutdown_connections 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb329640 0x556ccb32ba60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb319bd0 0x556ccb316ec0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30e400 0x556ccb3195c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7efffb7fe700 0 client.17187 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:52:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 --2- 172.21.15.5:0/676109574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30ed20 0x556ccb316840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 >> 172.21.15.5:0/676109574 conn(0x556ccb283260 msgr2=0x556ccb30a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 shutdown_connections 2023-09-23T04:52:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 1 -- 172.21.15.5:0/676109574 wait complete. 2023-09-23T04:52:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.945+0000 7f0035caab80 20 client.17187 trim_cache size 0 max 16384 2023-09-23T04:52:18.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363483 5.61 MB/sec execute 174 sec latency 1527.309 ms 2023-09-23T04:52:18.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 Processor -- start 2023-09-23T04:52:18.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 -- start start 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 0x556ccb319160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb319bd0 0x556ccb313b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb0e2920 con 0x556ccb30e400 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb306dd0 con 0x556ccb319bd0 2023-09-23T04:52:18.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f0035caab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb309580 con 0x556ccb30ed20 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.959+0000 7f001bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54502/0 (socket says 172.21.15.5:54502) 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 -- 172.21.15.5:0/2030152538 learned_addr learned my addr 172.21.15.5:0/2030152538 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f002099c700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb319bd0 0x556ccb313b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f0013fff700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 0x556ccb319160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 -- 172.21.15.5:0/2030152538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 msgr2=0x556ccb319160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 0x556ccb319160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 -- 172.21.15.5:0/2030152538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb319bd0 msgr2=0x556ccb313b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb319bd0 0x556ccb313b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 -- 172.21.15.5:0/2030152538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ccb306480 con 0x556ccb30e400 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f0013fff700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 0x556ccb319160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f001bfff700 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f000c096c00 tx=0x7f000c01efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f12256fe49267a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f0012ffd700 1 -- 172.21.15.5:0/2030152538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c09ae00 con 0x556ccb30e400 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f0012ffd700 1 -- 172.21.15.5:0/2030152538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c09a700 con 0x556ccb30e400 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.960+0000 7f0012ffd700 1 -- 172.21.15.5:0/2030152538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c098aa0 con 0x556ccb30e400 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- 172.21.15.5:0/2030152538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 msgr2=0x556ccb320720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f000c096c00 tx=0x7f000c01efb0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- 172.21.15.5:0/2030152538 shutdown_connections 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb319bd0 0x556ccb313b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30e400 0x556ccb320720 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 --2- 172.21.15.5:0/2030152538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb30ed20 0x556ccb319160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- 172.21.15.5:0/2030152538 >> 172.21.15.5:0/2030152538 conn(0x556ccb283260 msgr2=0x556ccb325910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- 172.21.15.5:0/2030152538 shutdown_connections 2023-09-23T04:52:18.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- 172.21.15.5:0/2030152538 wait complete. 2023-09-23T04:52:18.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.961+0000 7f0035caab80 1 -- start start 2023-09-23T04:52:18.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 Processor -- start 2023-09-23T04:52:18.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 0x556ccb3715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb376450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb376990 con 0x556ccb30ed20 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb376ad0 con 0x556ccb30e400 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ccb376c90 con 0x556ccb319bd0 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0035caab80 10 client.0 _ll_register_callbacks cb 0x7f0014004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:18.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.962+0000 7f0013fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f001bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 0x556ccb3715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54514/0 (socket says 172.21.15.5:54514) 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 -- 172.21.15.5:0/3729798080 learned_addr learned my addr 172.21.15.5:0/3729798080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f002099c700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb376450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 -- 172.21.15.5:0/3729798080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 msgr2=0x556ccb376450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb376450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 -- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 msgr2=0x556ccb3715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 0x556ccb3715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 -- 172.21.15.5:0/3729798080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a62f0 con 0x556ccb30ed20 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f001bfff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 0x556ccb3715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0013fff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0004009bd0 tx=0x7f00040050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316d717ccab67b22 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7efffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004005440 con 0x556ccb30ed20 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.963+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0004011e70 con 0x556ccb30ed20 2023-09-23T04:52:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f00040055a0 con 0x556ccb30ed20 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0004003680 con 0x556ccb30ed20 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 10 client.17193 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556ccb075c60 con 0x556ccb30ed20 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0004011e70 con 0x556ccb30ed20 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0004023820 con 0x556ccb30ed20 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 10 client.17193 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 10 client.17193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 20 client.17193 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb37e480 0x556ccb3808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 4 client.17193 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.964+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ccb25ab30 con 0x556ccb37e480 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f001bfff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb37e480 0x556ccb3808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f001bfff700 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb37e480 0x556ccb3808a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6179dc7c890f1db8 server_cookie=e68c7b4d87b80b86 in_seq=0 out_seq=0 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7efffb7fe700 10 client.17193 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:52:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7efffb7fe700 1 -- 172.21.15.5:0/3729798080 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ccb25ab30 con 0x556ccb37e480 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7efffb7fe700 10 client.17193 handle_command_reply: tid=0 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 client.17193 shutdown 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 msgr2=0x556ccb371b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0004009bd0 tx=0x7f00040050c0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 shutdown_connections 2023-09-23T04:52:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x556ccb37e480 0x556ccb3808a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x556ccb319bd0 0x556ccb376450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556ccb30e400 0x556ccb3715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7f0035caab80 1 --2- 172.21.15.5:0/3729798080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ccb30ed20 0x556ccb371b30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.965+0000 7efffb7fe700 0 client.17193 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 >> 172.21.15.5:0/3729798080 conn(0x556ccb283260 msgr2=0x556ccb310290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 shutdown_connections 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f0035caab80 1 -- 172.21.15.5:0/3729798080 wait complete. 2023-09-23T04:52:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f0035caab80 20 client.17193 trim_cache size 0 max 16384 2023-09-23T04:52:18.976 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:52:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 msgr2=0x7f00080a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 0x7f00080a3bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f001400b070 tx=0x7f0014007c80 comp rx=0 tx=0).stop 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 msgr2=0x7f001c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f000c00eb30 tx=0x7f000c00ee40 comp rx=0 tx=0).stop 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 shutdown_connections 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f00080a1710 0x7f00080a3bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.966+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f001c073ba0 0x7f001c1a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.967+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f001c1010d0 0x7f001c073120 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.967+0000 7f00223ff700 1 --2- 172.21.15.5:0/2869802792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f001c1061f0 0x7f001c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.967+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 >> 172.21.15.5:0/2869802792 conn(0x7f001c0fc990 msgr2=0x7f001c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.967+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 shutdown_connections 2023-09-23T04:52:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:18.967+0000 7f00223ff700 1 -- 172.21.15.5:0/2869802792 wait complete. 2023-09-23T04:52:18.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:18 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:52:18.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2535100304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:52:18.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:18 smithi005 ceph-mon[97638]: pgmap v184: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 241 op/s 2023-09-23T04:52:18.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:52:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:18 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:52:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2535100304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:52:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:18 smithi118 ceph-mon[134297]: pgmap v184: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 241 op/s 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.189+0000 7fbaeedff700 1 -- 172.21.15.5:0/1304184143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81010d0 msgr2=0x7fbae81014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.189+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1304184143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81010d0 0x7fbae81014b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0098f0 tx=0x7fbadc009c00 comp rx=0 tx=0).stop 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 -- 172.21.15.5:0/1304184143 shutdown_connections 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1304184143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae81061f0 0x7fbae810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1304184143 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81010d0 0x7fbae81014b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1304184143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae8101a80 0x7fbae8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 -- 172.21.15.5:0/1304184143 >> 172.21.15.5:0/1304184143 conn(0x7fbae80fc990 msgr2=0x7fbae80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 -- 172.21.15.5:0/1304184143 shutdown_connections 2023-09-23T04:52:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.190+0000 7fbaeedff700 1 -- 172.21.15.5:0/1304184143 wait complete. 2023-09-23T04:52:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.191+0000 7fbaeedff700 1 Processor -- start 2023-09-23T04:52:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.191+0000 7fbaeedff700 1 -- start start 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81061f0 0x7fbae8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae819db90 0x7fbae819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819e610 con 0x7fbae81010d0 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819e780 con 0x7fbae819db90 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaeedff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819e940 con 0x7fbae81061f0 2023-09-23T04:52:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaecb9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaecb9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54542/0 (socket says 172.21.15.5:54542) 2023-09-23T04:52:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaecb9b700 1 -- 172.21.15.5:0/1321121376 learned_addr learned my addr 172.21.15.5:0/1321121376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbaed39c700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae819db90 0x7fbae819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.192+0000 7fbae7fff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81061f0 0x7fbae8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 -- 172.21.15.5:0/1321121376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81061f0 msgr2=0x7fbae8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81061f0 0x7fbae8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 -- 172.21.15.5:0/1321121376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae819db90 msgr2=0x7fbae819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae819db90 0x7fbae819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 -- 172.21.15.5:0/1321121376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40095f0 con 0x7fbae81010d0 2023-09-23T04:52:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009810 tx=0x7fbadc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbae5ffb700 1 -- 172.21.15.5:0/1321121376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01c070 con 0x7fbae81010d0 2023-09-23T04:52:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc009520 con 0x7fbae81010d0 2023-09-23T04:52:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbae5ffb700 1 -- 172.21.15.5:0/1321121376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbadc00bac0 con 0x7fbae81010d0 2023-09-23T04:52:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.193+0000 7fbae5ffb700 1 -- 172.21.15.5:0/1321121376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004930 con 0x7fbae81010d0 2023-09-23T04:52:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.194+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae81a6130 con 0x7fbae81010d0 2023-09-23T04:52:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.197+0000 7fbae5ffb700 1 -- 172.21.15.5:0/1321121376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fbadc020070 con 0x7fbae81010d0 2023-09-23T04:52:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 Processor -- start 2023-09-23T04:52:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 -- start start 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d0f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d11fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbae5ffb700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 0x7fbad80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1c6d460 con 0x558ce1d04d20 2023-09-23T04:52:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1c5ec20 con 0x558ce1d0fbd0 2023-09-23T04:52:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.199+0000 7fbb026deb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1c5c540 con 0x558ce1d04400 2023-09-23T04:52:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbae5ffb700 1 -- 172.21.15.5:0/1321121376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbadc0ce4e0 con 0x7fbae81010d0 2023-09-23T04:52:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaecb9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d0f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47770/0 (socket says 172.21.15.5:47770) 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 -- 172.21.15.5:0/1327572060 learned_addr learned my addr 172.21.15.5:0/1327572060 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbae7fff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 0x7fbad80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d11fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 -- 172.21.15.5:0/1327572060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 msgr2=0x558ce1d11fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d11fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 -- 172.21.15.5:0/1327572060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 msgr2=0x558ce1d0f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d0f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 -- 172.21.15.5:0/1327572060 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce1cba8d0 con 0x558ce1d04400 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.200+0000 7fbaed39c700 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009fa0 tx=0x7fbae000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a3a0a454808e525 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbae7fff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 0x7fbad80a3bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbae819a800 tx=0x7fbad4009310 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbad2ffd700 1 -- 172.21.15.5:0/1327572060 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0004bb0 con 0x558ce1d04400 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbad2ffd700 1 -- 172.21.15.5:0/1327572060 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae000f070 con 0x558ce1d04400 2023-09-23T04:52:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbad2ffd700 1 -- 172.21.15.5:0/1327572060 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0019400 con 0x558ce1d04400 2023-09-23T04:52:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 -- 172.21.15.5:0/1327572060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 msgr2=0x558ce1d047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009fa0 tx=0x7fbae000b040 comp rx=0 tx=0).stop 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 -- 172.21.15.5:0/1327572060 shutdown_connections 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d11fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d047e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 --2- 172.21.15.5:0/1327572060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d0f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.201+0000 7fbb026deb80 1 -- 172.21.15.5:0/1327572060 >> 172.21.15.5:0/1327572060 conn(0x558ce1c79260 msgr2=0x558ce1d00790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.202+0000 7fbb026deb80 1 -- 172.21.15.5:0/1327572060 shutdown_connections 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.202+0000 7fbb026deb80 1 -- 172.21.15.5:0/1327572060 wait complete. 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.202+0000 7fbb026deb80 1 -- start start 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.202+0000 7fbb026deb80 1 Processor -- start 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d0d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1c57e30 con 0x558ce1d0fbd0 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1bf4710 con 0x558ce1d04400 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1bf43b0 con 0x558ce1d04d20 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbb026deb80 10 client.0 _ll_register_callbacks cb 0x558ce1a51800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbaed39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d0d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47784/0 (socket says 172.21.15.5:47784) 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbae7fff700 1 -- 172.21.15.5:0/4008510201 learned_addr learned my addr 172.21.15.5:0/4008510201 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.203+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 -- 172.21.15.5:0/4008510201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 msgr2=0x558ce1d0f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 -- 172.21.15.5:0/4008510201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 msgr2=0x558ce1d0d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d0d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 -- 172.21.15.5:0/4008510201 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae0011040 con 0x558ce1d04d20 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbaed39c700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbae7fff700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e950 tx=0x7fbad40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d980b0e7ba1f41 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad401c8e0 con 0x558ce1d04d20 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ce1cf8250 con 0x558ce1d04d20 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbad4023070 con 0x558ce1d04d20 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad401ca40 con 0x558ce1d04d20 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbb026deb80 10 client.27256 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.204+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558ce1a82de0 con 0x558ce1d04d20 2023-09-23T04:52:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ce1cf8250 con 0x558ce1d04d20 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbad401d410 con 0x558ce1d04d20 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 10 client.27256 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 10 client.27256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 20 client.27256 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d1f810 0x558ce1d21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 4 client.27256 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ce1c50e10 con 0x558ce1d1f810 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.205+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d1f810 0x558ce1d21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.206+0000 7fbac7fff700 10 client.27256 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.206+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d1f810 0x558ce1d21c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e4bf2749f0862bd server_cookie=9458454b0cfbcc63 in_seq=0 out_seq=0 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbac7fff700 1 -- 172.21.15.5:0/4008510201 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ce1c50e10 con 0x558ce1d1f810 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbac7fff700 10 client.27256 handle_command_reply: tid=0 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 client.27256 shutdown 2023-09-23T04:52:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 msgr2=0x558ce1d0c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e950 tx=0x7fbad40047c0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 shutdown_connections 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d1f810 0x558ce1d21c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 --2- 172.21.15.5:0/4008510201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c940 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbac7fff700 0 client.27256 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:52:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 >> 172.21.15.5:0/4008510201 conn(0x558ce1c79260 msgr2=0x558ce1d00320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 shutdown_connections 2023-09-23T04:52:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 1 -- 172.21.15.5:0/4008510201 wait complete. 2023-09-23T04:52:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.207+0000 7fbb026deb80 20 client.27256 trim_cache size 0 max 16384 2023-09-23T04:52:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 Processor -- start 2023-09-23T04:52:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 -- start start 2023-09-23T04:52:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1ad8920 con 0x558ce1d0fbd0 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1cfcd60 con 0x558ce1d04400 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbb026deb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1d05bf0 con 0x558ce1d04d20 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.221+0000 7fbaed39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54568/0 (socket says 172.21.15.5:54568) 2023-09-23T04:52:19.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 -- 172.21.15.5:0/3430812989 learned_addr learned my addr 172.21.15.5:0/3430812989 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbae7fff700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 -- 172.21.15.5:0/3430812989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 msgr2=0x558ce1d0c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 -- 172.21.15.5:0/3430812989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 msgr2=0x558ce1d0c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 -- 172.21.15.5:0/3430812989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ce1a6bc60 con 0x558ce1d0fbd0 2023-09-23T04:52:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbaed39c700 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbae00093b0 tx=0x7fbae00091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b565d14989a0b5c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.222+0000 7fbad37fe700 1 -- 172.21.15.5:0/3430812989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0003a00 con 0x558ce1d0fbd0 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbad37fe700 1 -- 172.21.15.5:0/3430812989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae0005560 con 0x558ce1d0fbd0 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbad37fe700 1 -- 172.21.15.5:0/3430812989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0024800 con 0x558ce1d0fbd0 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 -- 172.21.15.5:0/3430812989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 msgr2=0x558ce1d09bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbae00093b0 tx=0x7fbae00091b0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 -- 172.21.15.5:0/3430812989 shutdown_connections 2023-09-23T04:52:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d0fbd0 0x558ce1d09bd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d04400 0x558ce1d0c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3430812989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04d20 0x558ce1d0c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 -- 172.21.15.5:0/3430812989 >> 172.21.15.5:0/3430812989 conn(0x558ce1c79260 msgr2=0x558ce1d07c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 -- 172.21.15.5:0/3430812989 shutdown_connections 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.223+0000 7fbb026deb80 1 -- 172.21.15.5:0/3430812989 wait complete. 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 -- start start 2023-09-23T04:52:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 Processor -- start 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1d14e10 con 0x558ce1d04d20 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1d14f80 con 0x558ce1d0fbd0 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ce1d15140 con 0x558ce1d04400 2023-09-23T04:52:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.224+0000 7fbb026deb80 10 client.0 _ll_register_callbacks cb 0x7fbad400e5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbaecb9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54572/0 (socket says 172.21.15.5:54572) 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 -- 172.21.15.5:0/3879737634 learned_addr learned my addr 172.21.15.5:0/3879737634 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbaed39c700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 -- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 msgr2=0x558ce1d161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 -- 172.21.15.5:0/3879737634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 msgr2=0x558ce1d14700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d14700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 -- 172.21.15.5:0/3879737634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae81a6130 con 0x558ce1d04d20 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbae7fff700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad401d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471219454e2aec71 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4005a00 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbae000f070 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.225+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fbad4023040 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad401c8f0 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 10 client.17211 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558ce1cba8d0 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae000f070 con 0x558ce1d04d20 2023-09-23T04:52:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbad4028a80 con 0x558ce1d04d20 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 10 client.17211 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 10 client.17211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 20 client.17211 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d747d0 0x558ce1d76bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 4 client.17211 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.226+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ce1c50b30 con 0x558ce1d747d0 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d747d0 0x558ce1d76bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbaecb9b700 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d747d0 0x558ce1d76bf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e4dda93395b1a49 server_cookie=78981007ac2a7e42 in_seq=0 out_seq=0 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbac7fff700 10 client.17211 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbac7fff700 1 -- 172.21.15.5:0/3879737634 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558ce1c50b30 con 0x558ce1d747d0 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbac7fff700 10 client.17211 handle_command_reply: tid=0 2023-09-23T04:52:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 client.17211 shutdown 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 msgr2=0x558ce1d16700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad401d640 comp rx=0 tx=0).stop 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 shutdown_connections 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x558ce1d747d0 0x558ce1d76bf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558ce1d0fbd0 0x558ce1d14700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558ce1d04400 0x558ce1d161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbac7fff700 0 client.17211 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 --2- 172.21.15.5:0/3879737634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ce1d04d20 0x558ce1d16700 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.227+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 >> 172.21.15.5:0/3879737634 conn(0x558ce1c79260 msgr2=0x558ce1d0e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 shutdown_connections 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbb026deb80 1 -- 172.21.15.5:0/3879737634 wait complete. 2023-09-23T04:52:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbb026deb80 20 client.17211 trim_cache size 0 max 16384 2023-09-23T04:52:19.239 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 msgr2=0x7fbad80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 0x7fbad80a3bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbae819a800 tx=0x7fbad4009310 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 msgr2=0x7fbae81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009810 tx=0x7fbadc00fa00 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 shutdown_connections 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbad80a1710 0x7fbad80a3bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbae819db90 0x7fbae819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae81010d0 0x7fbae81991f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 --2- 172.21.15.5:0/1321121376 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbae81061f0 0x7fbae8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.228+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 >> 172.21.15.5:0/1321121376 conn(0x7fbae80fc990 msgr2=0x7fbae8102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.229+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 shutdown_connections 2023-09-23T04:52:19.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.229+0000 7fbaeedff700 1 -- 172.21.15.5:0/1321121376 wait complete. 2023-09-23T04:52:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:18 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:52:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2535100304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:52:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:18 smithi069 ceph-mon[133931]: pgmap v184: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 241 op/s 2023-09-23T04:52:19.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:52:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/388318987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68101a80 msgr2=0x7f3a68105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/388318987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68101a80 0x7f3a68105ac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009990 tx=0x7f3a5c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/388318987 shutdown_connections 2023-09-23T04:52:19.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/388318987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681061f0 0x7f3a6810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/388318987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681010d0 0x7f3a681014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/388318987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a68101a80 0x7f3a68105ac0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/388318987 >> 172.21.15.5:0/388318987 conn(0x7f3a680fc990 msgr2=0x7f3a680fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/388318987 shutdown_connections 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.452+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/388318987 wait complete. 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.453+0000 7f3a6e1ff700 1 Processor -- start 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 -- start start 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681061f0 0x7f3a681974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819c3a0 con 0x7f3a6819b920 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819c510 con 0x7f3a681010d0 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6819c6d0 con 0x7f3a681061f0 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54590/0 (socket says 172.21.15.5:54590) 2023-09-23T04:52:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a67fff700 1 -- 172.21.15.5:0/1140267859 learned_addr learned my addr 172.21.15.5:0/1140267859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59012/0 (socket says 172.21.15.5:59012) 2023-09-23T04:52:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.454+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681061f0 0x7f3a681974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 -- 172.21.15.5:0/1140267859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681061f0 msgr2=0x7f3a681974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681061f0 0x7f3a681974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 -- 172.21.15.5:0/1140267859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 msgr2=0x7f3a68196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 -- 172.21.15.5:0/1140267859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0095f0 con 0x7f3a6819b920 2023-09-23T04:52:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a677fe700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000ef30 tx=0x7f3a6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a64ff9700 1 -- 172.21.15.5:0/1140267859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60018070 con 0x7f3a6819b920 2023-09-23T04:52:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a68073050 con 0x7f3a6819b920 2023-09-23T04:52:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a64ff9700 1 -- 172.21.15.5:0/1140267859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6000f040 con 0x7f3a6819b920 2023-09-23T04:52:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a64ff9700 1 -- 172.21.15.5:0/1140267859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000cd00 con 0x7f3a6819b920 2023-09-23T04:52:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.455+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680734e0 con 0x7f3a6819b920 2023-09-23T04:52:19.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.458+0000 7f3a64ff9700 1 -- 172.21.15.5:0/1140267859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3a60003a00 con 0x7f3a6819b920 2023-09-23T04:52:19.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 Processor -- start 2023-09-23T04:52:19.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 -- start start 2023-09-23T04:52:19.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a64ff9700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 0x7f3a500a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a64ff9700 1 -- 172.21.15.5:0/1140267859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3a600cd210 con 0x7f3a6819b920 2023-09-23T04:52:19.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 0x55c5a1421fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a137d460 con 0x55c5a1414400 2023-09-23T04:52:19.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a136ec20 con 0x55c5a141fbd0 2023-09-23T04:52:19.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a136c540 con 0x55c5a1414d20 2023-09-23T04:52:19.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 0x7f3a500a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.460+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54602/0 (socket says 172.21.15.5:54602) 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 -- 172.21.15.5:0/1155581409 learned_addr learned my addr 172.21.15.5:0/1155581409 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 0x55c5a1421fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 -- 172.21.15.5:0/1155581409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 msgr2=0x55c5a141f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 -- 172.21.15.5:0/1155581409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 msgr2=0x55c5a1421fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 0x55c5a1421fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 -- 172.21.15.5:0/1155581409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a13ca8d0 con 0x55c5a1414400 2023-09-23T04:52:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 0x55c5a1421fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:19.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 0x7f3a500a3bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00b2e0 tx=0x7f3a5c0053d0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a677fe700 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009910 tx=0x7f3a58009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cf86769dcc80f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.461+0000 7f3a4dffb700 1 -- 172.21.15.5:0/1155581409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58015050 con 0x55c5a1414400 2023-09-23T04:52:19.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a4dffb700 1 -- 172.21.15.5:0/1155581409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a58016040 con 0x55c5a1414400 2023-09-23T04:52:19.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a4dffb700 1 -- 172.21.15.5:0/1155581409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5801aa10 con 0x55c5a1414400 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1155581409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 msgr2=0x55c5a14147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009910 tx=0x7f3a58009260 comp rx=0 tx=0).stop 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1155581409 shutdown_connections 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a141fbd0 0x55c5a1421fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414400 0x55c5a14147e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1155581409 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1155581409 >> 172.21.15.5:0/1155581409 conn(0x55c5a1389260 msgr2=0x55c5a1410790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1155581409 shutdown_connections 2023-09-23T04:52:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.462+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1155581409 wait complete. 2023-09-23T04:52:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.463+0000 7f3a81b59b80 1 -- start start 2023-09-23T04:52:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.463+0000 7f3a81b59b80 1 Processor -- start 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414d20 0x55c5a141ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a141fbd0 0x55c5a141d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a1367e30 con 0x55c5a1414d20 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a1304710 con 0x55c5a1414400 2023-09-23T04:52:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a13043b0 con 0x55c5a141fbd0 2023-09-23T04:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a81b59b80 10 client.0 _ll_register_callbacks cb 0x55c5a1161800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59026/0 (socket says 172.21.15.5:59026) 2023-09-23T04:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a677fe700 1 -- 172.21.15.5:0/2184747679 learned_addr learned my addr 172.21.15.5:0/2184747679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.464+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414d20 0x55c5a141ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a67fff700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a141fbd0 0x55c5a141d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 -- 172.21.15.5:0/2184747679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a141fbd0 msgr2=0x55c5a141d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a141fbd0 0x55c5a141d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 -- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414d20 msgr2=0x55c5a141ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414d20 0x55c5a141ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 -- 172.21.15.5:0/2184747679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a580095c0 con 0x55c5a1414400 2023-09-23T04:52:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a677fe700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3a580059d0 tx=0x7f3a580057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede3adf7c5d9b631 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.465+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58004460 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5a1408250 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3a58005ce0 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a58004750 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 10 client.27224 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c5a1192de0 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5a1408250 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3a5802d070 con 0x55c5a1414400 2023-09-23T04:52:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 10 client.27224 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.466+0000 7f3a81b59b80 20 client.27224 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a142f640 0x55c5a1431a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a81b59b80 4 client.27224 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5a1360e10 con 0x55c5a142f640 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a142f640 0x55c5a1431a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a142f640 0x55c5a1431a60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9cd2bc1b2dfc7 server_cookie=42cd705defe39ea5 in_seq=0 out_seq=0 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.467+0000 7f3a46ffd700 10 client.27224 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a46ffd700 1 -- 172.21.15.5:0/2184747679 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5a1360e10 con 0x55c5a142f640 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a46ffd700 10 client.27224 handle_command_reply: tid=0 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 client.27224 shutdown 2023-09-23T04:52:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 msgr2=0x55c5a141c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3a580059d0 tx=0x7f3a580057d0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 shutdown_connections 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a142f640 0x55c5a1431a60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a141fbd0 0x55c5a141d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a46ffd700 0 client.27224 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a141c8f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/2184747679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a1414d20 0x55c5a141ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 >> 172.21.15.5:0/2184747679 conn(0x55c5a1389260 msgr2=0x55c5a1410380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 shutdown_connections 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.468+0000 7f3a81b59b80 1 -- 172.21.15.5:0/2184747679 wait complete. 2023-09-23T04:52:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.469+0000 7f3a81b59b80 20 client.27224 trim_cache size 0 max 16384 2023-09-23T04:52:19.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.482+0000 7f3a81b59b80 1 Processor -- start 2023-09-23T04:52:19.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.482+0000 7f3a81b59b80 1 -- start start 2023-09-23T04:52:19.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a1426720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a11e8920 con 0x55c5a141fbd0 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a1426c60 con 0x55c5a1414400 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a1418760 con 0x55c5a1414d20 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54616/0 (socket says 172.21.15.5:54616) 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 -- 172.21.15.5:0/1227429072 learned_addr learned my addr 172.21.15.5:0/1227429072 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a677fe700 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a1426720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 -- 172.21.15.5:0/1227429072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 msgr2=0x55c5a141ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 -- 172.21.15.5:0/1227429072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 msgr2=0x55c5a1426720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a1426720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 -- 172.21.15.5:0/1227429072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5a140c480 con 0x55c5a141fbd0 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.483+0000 7f3a67fff700 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c6f0 tx=0x7f3a60099cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb18bfaddf40d31d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a4e7fc700 1 -- 172.21.15.5:0/1227429072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a0070 con 0x55c5a141fbd0 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a4e7fc700 1 -- 172.21.15.5:0/1227429072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a60004ad0 con 0x55c5a141fbd0 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a4e7fc700 1 -- 172.21.15.5:0/1227429072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6009df20 con 0x55c5a141fbd0 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1227429072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 msgr2=0x55c5a141d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c6f0 tx=0x7f3a60099cd0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1227429072 shutdown_connections 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a141d3a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414400 0x55c5a1426720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/1227429072 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414d20 0x55c5a141ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1227429072 >> 172.21.15.5:0/1227429072 conn(0x55c5a1389260 msgr2=0x55c5a1417bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1227429072 shutdown_connections 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.484+0000 7f3a81b59b80 1 -- 172.21.15.5:0/1227429072 wait complete. 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.485+0000 7f3a81b59b80 1 -- start start 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.485+0000 7f3a81b59b80 1 Processor -- start 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414400 0x55c5a1477610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 0x55c5a1477b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a147ca20 con 0x55c5a141fbd0 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a147cb90 con 0x55c5a1414d20 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5a147cd50 con 0x55c5a1414400 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a81b59b80 10 client.0 _ll_register_callbacks cb 0x7f3a580097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54618/0 (socket says 172.21.15.5:54618) 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a677fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414400 0x55c5a1477610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 -- 172.21.15.5:0/751793714 learned_addr learned my addr 172.21.15.5:0/751793714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 0x55c5a1477b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 -- 172.21.15.5:0/751793714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414400 msgr2=0x55c5a1477610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414400 0x55c5a1477610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 -- 172.21.15.5:0/751793714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 msgr2=0x55c5a1477b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 0x55c5a1477b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a67fff700 1 -- 172.21.15.5:0/751793714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680734e0 con 0x55c5a141fbd0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.486+0000 7f3a66ffd700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 0x55c5a1477b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a67fff700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cd010 tx=0x7f3a60096790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a898689ece521 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600b2450 con 0x55c5a141fbd0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a58016040 con 0x55c5a141fbd0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f3a6009a4d0 con 0x55c5a141fbd0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6000ce60 con 0x55c5a141fbd0 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a81b59b80 10 client.17229 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:19.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55c5a117bc60 con 0x55c5a141fbd0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.487+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a58016040 con 0x55c5a141fbd0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3a600992c0 con 0x55c5a141fbd0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 20 client.17229 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a14841f0 0x55c5a1486610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 4 client.17229 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5a1360b30 con 0x55c5a14841f0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a677fe700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a14841f0 0x55c5a1486610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a677fe700 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a14841f0 0x55c5a1486610 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1647eb2fbe23e462 server_cookie=be62db219eb88e4f in_seq=0 out_seq=0 2023-09-23T04:52:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.488+0000 7f3a46ffd700 10 client.17229 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a46ffd700 1 -- 172.21.15.5:0/751793714 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5a1360b30 con 0x55c5a14841f0 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a46ffd700 10 client.17229 handle_command_reply: tid=0 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 client.17229 shutdown 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 msgr2=0x55c5a147c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a600cd010 tx=0x7f3a60096790 comp rx=0 tx=0).stop 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 shutdown_connections 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c5a14841f0 0x55c5a1486610 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5a141fbd0 0x55c5a147c4e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c5a1414400 0x55c5a1477610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 --2- 172.21.15.5:0/751793714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5a1414d20 0x55c5a1477b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 >> 172.21.15.5:0/751793714 conn(0x55c5a1389260 msgr2=0x55c5a1417bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a46ffd700 0 client.17229 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:19.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 shutdown_connections 2023-09-23T04:52:19.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 1 -- 172.21.15.5:0/751793714 wait complete. 2023-09-23T04:52:19.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.489+0000 7f3a81b59b80 20 client.17229 trim_cache size 0 max 16384 2023-09-23T04:52:19.501 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 msgr2=0x7f3a500a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 0x7f3a500a3bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00b2e0 tx=0x7f3a5c0053d0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 msgr2=0x7f3a6819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000ef30 tx=0x7f3a6000c3a0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 shutdown_connections 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3a500a1710 0x7f3a500a3bd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6819b920 0x7f3a6819bda0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a681010d0 0x7f3a68196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 --2- 172.21.15.5:0/1140267859 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3a681061f0 0x7f3a681974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:19.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 >> 172.21.15.5:0/1140267859 conn(0x7f3a680fc990 msgr2=0x7f3a680fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:19.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 shutdown_connections 2023-09-23T04:52:19.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:19.490+0000 7f3a6e1ff700 1 -- 172.21.15.5:0/1140267859 wait complete. 2023-09-23T04:52:19.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363483 5.57 MB/sec execute 175 sec latency 2527.427 ms 2023-09-23T04:52:20.508 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:52:20.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:52:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 -- 172.21.15.5:0/453696081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 msgr2=0x7efcfc073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 --2- 172.21.15.5:0/453696081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc073630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efcec0098f0 tx=0x7efcec009c00 comp rx=0 tx=0).stop 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 -- 172.21.15.5:0/453696081 shutdown_connections 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 --2- 172.21.15.5:0/453696081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc10d6a0 0x7efcfc10fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 --2- 172.21.15.5:0/453696081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc073630 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 --2- 172.21.15.5:0/453696081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc073b70 0x7efcfc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 -- 172.21.15.5:0/453696081 >> 172.21.15.5:0/453696081 conn(0x7efcfc0fc990 msgr2=0x7efcfc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 -- 172.21.15.5:0/453696081 shutdown_connections 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.745+0000 7efd013ff700 1 -- 172.21.15.5:0/453696081 wait complete. 2023-09-23T04:52:20.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.747+0000 7efd013ff700 1 Processor -- start 2023-09-23T04:52:20.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.747+0000 7efd013ff700 1 -- start start 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc073b70 0x7efcfc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc10d6a0 0x7efcfc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19e1a0 con 0x7efcfc10d6a0 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19e310 con 0x7efcfc073250 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efd013ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19e4d0 con 0x7efcfc073b70 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59056/0 (socket says 172.21.15.5:59056) 2023-09-23T04:52:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 -- 172.21.15.5:0/1470172210 learned_addr learned my addr 172.21.15.5:0/1470172210 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc073b70 0x7efcfc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc10d6a0 0x7efcfc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 -- 172.21.15.5:0/1470172210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc073b70 msgr2=0x7efcfc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc073b70 0x7efcfc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 -- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc10d6a0 msgr2=0x7efcfc19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.748+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc10d6a0 0x7efcfc19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efcfaffd700 1 -- 172.21.15.5:0/1470172210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec009520 con 0x7efcfc073250 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcec009810 tx=0x7efcec00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efce3fff700 1 -- 172.21.15.5:0/1470172210 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec01c070 con 0x7efcfc073250 2023-09-23T04:52:20.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efce3fff700 1 -- 172.21.15.5:0/1470172210 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcec00bbd0 con 0x7efcfc073250 2023-09-23T04:52:20.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efce3fff700 1 -- 172.21.15.5:0/1470172210 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec004930 con 0x7efcfc073250 2023-09-23T04:52:20.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc19e750 con 0x7efcfc073250 2023-09-23T04:52:20.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.749+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc19e9d0 con 0x7efcfc073250 2023-09-23T04:52:20.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.752+0000 7efce3fff700 1 -- 172.21.15.5:0/1470172210 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7efcec020070 con 0x7efcfc073250 2023-09-23T04:52:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.754+0000 7efd14c61b80 1 Processor -- start 2023-09-23T04:52:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.754+0000 7efce3fff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 0x7efce40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.754+0000 7efd14c61b80 1 -- start start 2023-09-23T04:52:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efce3fff700 1 -- 172.21.15.5:0/1470172210 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efcec0ce140 con 0x7efcfc073250 2023-09-23T04:52:20.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 0x7efce40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba5416710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5419040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba53744e0 con 0x555ba540bda0 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba5365ca0 con 0x555ba5416c50 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efd14c61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba53635c0 con 0x555ba540b480 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47886/0 (socket says 172.21.15.5:47886) 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efcfaffd700 1 -- 172.21.15.5:0/2561648685 learned_addr learned my addr 172.21.15.5:0/2561648685 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:20.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.755+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba5416710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5419040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 -- 172.21.15.5:0/2561648685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 msgr2=0x555ba5419040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5419040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 -- 172.21.15.5:0/2561648685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 msgr2=0x555ba5416710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba5416710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 -- 172.21.15.5:0/2561648685 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba53c1950 con 0x555ba540b480 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba5416710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 0x7efce40a3c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efcf400f590 tx=0x7efcf400b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efcfaffd700 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efcec097200 tx=0x7efcec009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c5e0b72b33a644 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efce17fa700 1 -- 172.21.15.5:0/2561648685 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec09ad00 con 0x555ba540b480 2023-09-23T04:52:20.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efce17fa700 1 -- 172.21.15.5:0/2561648685 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcec09ee50 con 0x555ba540b480 2023-09-23T04:52:20.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.756+0000 7efce17fa700 1 -- 172.21.15.5:0/2561648685 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec09af60 con 0x555ba540b480 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 -- 172.21.15.5:0/2561648685 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 msgr2=0x555ba540b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efcec097200 tx=0x7efcec009260 comp rx=0 tx=0).stop 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 -- 172.21.15.5:0/2561648685 shutdown_connections 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5419040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba540b860 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 --2- 172.21.15.5:0/2561648685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba5416710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 -- 172.21.15.5:0/2561648685 >> 172.21.15.5:0/2561648685 conn(0x555ba53802e0 msgr2=0x555ba5407810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 -- 172.21.15.5:0/2561648685 shutdown_connections 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.757+0000 7efd14c61b80 1 -- 172.21.15.5:0/2561648685 wait complete. 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 -- start start 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 Processor -- start 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba5417030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba5416c50 0x555ba54220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba535eeb0 con 0x555ba540b480 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba52fb790 con 0x555ba540bda0 2023-09-23T04:52:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.758+0000 7efd14c61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba52fb430 con 0x555ba5416c50 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efd14c61b80 10 client.0 _ll_register_callbacks cb 0x555ba5158880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59072/0 (socket says 172.21.15.5:59072) 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3534389476 learned_addr learned my addr 172.21.15.5:0/3534389476 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba5416c50 0x555ba54220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfaffd700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba5417030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3534389476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba5416c50 msgr2=0x555ba54220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba5416c50 0x555ba54220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 msgr2=0x555ba5417030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba5417030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3534389476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc19e750 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcf0012740 tx=0x7efcf0012a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2491e490ba22ac14 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.759+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0017070 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ba53ff2d0 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efcf0004050 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf0004210 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 10 client.27239 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555ba5189e60 con 0x555ba540bda0 2023-09-23T04:52:20.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ba53ff2d0 con 0x555ba540bda0 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efcf000f570 con 0x555ba540bda0 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 20 client.27239 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba5426360 0x555ba5428780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 4 client.27239 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.760+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ba5357e90 con 0x555ba5426360 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.761+0000 7efcfaffd700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba5426360 0x555ba5428780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.761+0000 7efcfaffd700 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba5426360 0x555ba5428780 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2aeee57cabe0c1 server_cookie=50ea1fff3211ad7 in_seq=0 out_seq=0 2023-09-23T04:52:20.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.761+0000 7efcdeff5700 10 client.27239 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efcdeff5700 1 -- 172.21.15.5:0/3534389476 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555ba5357e90 con 0x555ba5426360 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efcdeff5700 10 client.27239 handle_command_reply: tid=0 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 client.27239 shutdown 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 msgr2=0x555ba5417570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcf0012740 tx=0x7efcf0012a50 comp rx=0 tx=0).stop 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 shutdown_connections 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba5426360 0x555ba5428780 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba5416c50 0x555ba54220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba5417030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3534389476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba540bda0 0x555ba5417570 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efcdeff5700 0 client.27239 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:20.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 >> 172.21.15.5:0/3534389476 conn(0x555ba53802e0 msgr2=0x555ba54073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 shutdown_connections 2023-09-23T04:52:20.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.762+0000 7efd14c61b80 1 -- 172.21.15.5:0/3534389476 wait complete. 2023-09-23T04:52:20.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.763+0000 7efd14c61b80 20 client.27239 trim_cache size 0 max 16384 2023-09-23T04:52:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 Processor -- start 2023-09-23T04:52:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 -- start start 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba541c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5421cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba51df9a0 con 0x555ba540b480 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba5417bf0 con 0x555ba5416c50 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.781+0000 7efd14c61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba54178e0 con 0x555ba540bda0 2023-09-23T04:52:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5421cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba541c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47902/0 (socket says 172.21.15.5:47902) 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfb7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5421cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59076/0 (socket says 172.21.15.5:59076) 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3095887218 learned_addr learned my addr 172.21.15.5:0/3095887218 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3095887218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 msgr2=0x555ba5421cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5421cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3095887218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 msgr2=0x555ba541c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba541c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 -- 172.21.15.5:0/3095887218 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ba5172ce0 con 0x555ba540bda0 2023-09-23T04:52:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efcf00096d0 tx=0x7efcf000fef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c09580162bd99f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.782+0000 7efce1ffb700 1 -- 172.21.15.5:0/3095887218 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0017040 con 0x555ba540bda0 2023-09-23T04:52:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efce1ffb700 1 -- 172.21.15.5:0/3095887218 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcf0009440 con 0x555ba540bda0 2023-09-23T04:52:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efce1ffb700 1 -- 172.21.15.5:0/3095887218 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0003970 con 0x555ba540bda0 2023-09-23T04:52:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 -- 172.21.15.5:0/3095887218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 msgr2=0x555ba5421780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efcf00096d0 tx=0x7efcf000fef0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 -- 172.21.15.5:0/3095887218 shutdown_connections 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5421cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540b480 0x555ba541c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 --2- 172.21.15.5:0/3095887218 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540bda0 0x555ba5421780 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 -- 172.21.15.5:0/3095887218 >> 172.21.15.5:0/3095887218 conn(0x555ba53802e0 msgr2=0x555ba5414e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.783+0000 7efd14c61b80 1 -- 172.21.15.5:0/3095887218 shutdown_connections 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.784+0000 7efd14c61b80 1 -- 172.21.15.5:0/3095887218 wait complete. 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.784+0000 7efd14c61b80 1 -- start start 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.784+0000 7efd14c61b80 1 Processor -- start 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba541b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5474170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba54746b0 con 0x555ba540bda0 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba5474820 con 0x555ba5416c50 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ba54749e0 con 0x555ba540b480 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efd14c61b80 10 client.0 _ll_register_callbacks cb 0x7efcf0009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:20.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47914/0 (socket says 172.21.15.5:47914) 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 -- 172.21.15.5:0/1174969523 learned_addr learned my addr 172.21.15.5:0/1174969523 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba541b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfb7fe700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5474170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 -- 172.21.15.5:0/1174969523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 msgr2=0x555ba5474170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5474170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 -- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 msgr2=0x555ba541b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba541b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.785+0000 7efcfaffd700 1 -- 172.21.15.5:0/1174969523 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ba5189e60 con 0x555ba540b480 2023-09-23T04:52:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcfaffd700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efcec009c40 tx=0x7efcec0cb9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab61d24518e2d3ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0b3210 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcec09ee50 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7efcec0b3370 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcec097410 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 10 client.27298 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555ba51b02d0 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcec09ee50 con 0x555ba540b480 2023-09-23T04:52:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efcec0a2b70 con 0x555ba540b480 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 10 client.27298 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 10 client.27298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 20 client.27298 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba54849a0 0x555ba5486d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 4 client.27298 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.786+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555ba5357bb0 con 0x555ba54849a0 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.787+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba54849a0 0x555ba5486d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.787+0000 7efcfa7fc700 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba54849a0 0x555ba5486d90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27bb6b7c6456b66 server_cookie=206d63abd9a036a2 in_seq=0 out_seq=0 2023-09-23T04:52:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.787+0000 7efcdeff5700 10 client.27298 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:20.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.788+0000 7efcdeff5700 1 -- 172.21.15.5:0/1174969523 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555ba5357bb0 con 0x555ba54849a0 2023-09-23T04:52:20.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.788+0000 7efcdeff5700 10 client.27298 handle_command_reply: tid=0 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.788+0000 7efd14c61b80 1 client.27298 shutdown 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 msgr2=0x555ba541ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efcec009c40 tx=0x7efcec0cb9b0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 shutdown_connections 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555ba54849a0 0x555ba5486d90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba5416c50 0x555ba5474170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555ba540b480 0x555ba541ac30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efcdeff5700 0 client.27298 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 --2- 172.21.15.5:0/1174969523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ba540bda0 0x555ba541b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 >> 172.21.15.5:0/1174969523 conn(0x555ba53802e0 msgr2=0x555ba5406a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 shutdown_connections 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 1 -- 172.21.15.5:0/1174969523 wait complete. 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.789+0000 7efd14c61b80 20 client.27298 trim_cache size 0 max 16384 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:52:20.795 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:52:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 msgr2=0x7efce40a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 0x7efce40a3c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efcf400f590 tx=0x7efcf400b040 comp rx=0 tx=0).stop 2023-09-23T04:52:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 msgr2=0x7efcfc1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcec009810 tx=0x7efcec00fa00 comp rx=0 tx=0).stop 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 shutdown_connections 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efce40a1790 0x7efce40a3c50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc10d6a0 0x7efcfc19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcfc073250 0x7efcfc1991c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 --2- 172.21.15.5:0/1470172210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcfc073b70 0x7efcfc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 >> 172.21.15.5:0/1470172210 conn(0x7efcfc0fc990 msgr2=0x7efcfc10f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.790+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 shutdown_connections 2023-09-23T04:52:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:20.791+0000 7efd013ff700 1 -- 172.21.15.5:0/1470172210 wait complete. 2023-09-23T04:52:20.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:52:20.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 364343 5.56 MB/sec execute 176 sec latency 3054.089 ms 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 -- 172.21.15.5:0/4028166235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c101a80 msgr2=0x7f549c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 --2- 172.21.15.5:0/4028166235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c101a80 0x7f549c105ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5490009990 tx=0x7f5490009ca0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 -- 172.21.15.5:0/4028166235 shutdown_connections 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 --2- 172.21.15.5:0/4028166235 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1061f0 0x7f549c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 --2- 172.21.15.5:0/4028166235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1010d0 0x7f549c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 --2- 172.21.15.5:0/4028166235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c101a80 0x7f549c105ac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 -- 172.21.15.5:0/4028166235 >> 172.21.15.5:0/4028166235 conn(0x7f549c0fc990 msgr2=0x7f549c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 -- 172.21.15.5:0/4028166235 shutdown_connections 2023-09-23T04:52:21.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.061+0000 7f54a27ff700 1 -- 172.21.15.5:0/4028166235 wait complete. 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.062+0000 7f54a27ff700 1 Processor -- start 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.062+0000 7f54a27ff700 1 -- start start 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1061f0 0x7f549c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c19b8c0 0x7f549c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c340 con 0x7f549c1061f0 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c4b0 con 0x7f549c19b8c0 2023-09-23T04:52:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a27ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c670 con 0x7f549c1010d0 2023-09-23T04:52:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f549bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f549bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58316/0 (socket says 172.21.15.5:58316) 2023-09-23T04:52:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f549bfff700 1 -- 172.21.15.5:0/1209230166 learned_addr learned my addr 172.21.15.5:0/1209230166 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f549b7fe700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1061f0 0x7f549c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.063+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c19b8c0 0x7f549c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 -- 172.21.15.5:0/1209230166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c19b8c0 msgr2=0x7f549c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c19b8c0 0x7f549c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 -- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1061f0 msgr2=0x7f549c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1061f0 0x7f549c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 -- 172.21.15.5:0/1209230166 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900095f0 con 0x7f549c1010d0 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f549bfff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9a0 tx=0x7f548c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f54997fa700 1 -- 172.21.15.5:0/1209230166 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c009790 con 0x7f549c1010d0 2023-09-23T04:52:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c1a3bd0 con 0x7f549c1010d0 2023-09-23T04:52:21.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f54997fa700 1 -- 172.21.15.5:0/1209230166 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548c010430 con 0x7f549c1010d0 2023-09-23T04:52:21.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f54997fa700 1 -- 172.21.15.5:0/1209230166 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c018680 con 0x7f549c1010d0 2023-09-23T04:52:21.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.064+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1a40b0 con 0x7f549c1010d0 2023-09-23T04:52:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.068+0000 7f54997fa700 1 -- 172.21.15.5:0/1209230166 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f548c0105a0 con 0x7f549c1010d0 2023-09-23T04:52:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.070+0000 7f54b60bbb80 1 Processor -- start 2023-09-23T04:52:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.070+0000 7f54b60bbb80 1 -- start start 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c246690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c248fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c1a4460 con 0x56288c23bd20 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c195c20 con 0x56288c23b400 2023-09-23T04:52:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c193540 con 0x56288c246bd0 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54997fa700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 0x7f54840a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54a0d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54a0d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58000/0 (socket says 172.21.15.5:58000) 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54a0d9c700 1 -- 172.21.15.5:0/2202910680 learned_addr learned my addr 172.21.15.5:0/2202910680 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f549bfff700 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c246690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f549b7fe700 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c248fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54997fa700 1 -- 172.21.15.5:0/1209230166 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f548c014070 con 0x7f549c1010d0 2023-09-23T04:52:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.071+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 0x7f54840a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 -- 172.21.15.5:0/2202910680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 msgr2=0x56288c248fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c248fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 -- 172.21.15.5:0/2202910680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 msgr2=0x56288c246690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c246690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 -- 172.21.15.5:0/2202910680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56288c1f18d0 con 0x56288c23b400 2023-09-23T04:52:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5494009fc0 tx=0x7f5494004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac4d06c138e3fa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 0x7f54840a3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f549401b000 tx=0x7f549401d000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54827fc700 1 -- 172.21.15.5:0/2202910680 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54940095e0 con 0x56288c23b400 2023-09-23T04:52:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54827fc700 1 -- 172.21.15.5:0/2202910680 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f549400f040 con 0x56288c23b400 2023-09-23T04:52:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.072+0000 7f54827fc700 1 -- 172.21.15.5:0/2202910680 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549401ab90 con 0x56288c23b400 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 -- 172.21.15.5:0/2202910680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 msgr2=0x56288c23b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5494009fc0 tx=0x7f5494004320 comp rx=0 tx=0).stop 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 -- 172.21.15.5:0/2202910680 shutdown_connections 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c248fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c23b7e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/2202910680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c246690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 -- 172.21.15.5:0/2202910680 >> 172.21.15.5:0/2202910680 conn(0x56288c1b0260 msgr2=0x56288c237790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 -- 172.21.15.5:0/2202910680 shutdown_connections 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.073+0000 7f54b60bbb80 1 -- 172.21.15.5:0/2202910680 wait complete. 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.074+0000 7f54b60bbb80 1 -- start start 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.074+0000 7f54b60bbb80 1 Processor -- start 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 0x56288c2465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c243f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c18ee30 con 0x56288c23b400 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c12b710 con 0x56288c23bd20 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c12b3b0 con 0x56288c246bd0 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54b60bbb80 10 client.0 _ll_register_callbacks cb 0x56288bf88800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f549b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f549b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58002/0 (socket says 172.21.15.5:58002) 2023-09-23T04:52:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f549b7fe700 1 -- 172.21.15.5:0/1705531321 learned_addr learned my addr 172.21.15.5:0/1705531321 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c243f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.075+0000 7f549bfff700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 0x56288c2465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 -- 172.21.15.5:0/1705531321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 msgr2=0x56288c243f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c243f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 -- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 msgr2=0x56288c2465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 0x56288c2465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 -- 172.21.15.5:0/1705531321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494009ba0 con 0x56288c23bd20 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549bfff700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 0x56288c2465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f549b7fe700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5490009990 tx=0x7f549000b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d47074a2f83232 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f547b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490024070 con 0x56288c23bd20 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56288c22f250 con 0x56288c23bd20 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.076+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f549001f6d0 con 0x56288c23bd20 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.077+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54900056f0 con 0x56288c23bd20 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.077+0000 7f54b60bbb80 10 client.27257 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.077+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56288bfb9de0 con 0x56288c23bd20 2023-09-23T04:52:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.077+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56288c22f250 con 0x56288c23bd20 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5490004a00 con 0x56288c23bd20 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 10 client.27257 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 20 client.27257 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c256660 0x56288c258a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 4 client.27257 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56288c187e10 con 0x56288c256660 2023-09-23T04:52:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.078+0000 7f549bfff700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c256660 0x56288c258a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.079+0000 7f549bfff700 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c256660 0x56288c258a80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f620e60bfb5337a server_cookie=c48176220ce2e5e2 in_seq=0 out_seq=0 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.079+0000 7f547b7fe700 10 client.27257 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f547b7fe700 1 -- 172.21.15.5:0/1705531321 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56288c187e10 con 0x56288c256660 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f547b7fe700 10 client.27257 handle_command_reply: tid=0 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 client.27257 shutdown 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 msgr2=0x56288c2438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5490009990 tx=0x7f549000b610 comp rx=0 tx=0).stop 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 shutdown_connections 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c256660 0x56288c258a80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c246bd0 0x56288c243f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23b400 0x56288c2465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.080+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1705531321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23bd20 0x56288c2438a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.081+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 >> 172.21.15.5:0/1705531321 conn(0x56288c1b0260 msgr2=0x56288c237320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.082+0000 7f547b7fe700 0 client.27257 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.082+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 shutdown_connections 2023-09-23T04:52:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.082+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1705531321 wait complete. 2023-09-23T04:52:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.083+0000 7f54b60bbb80 20 client.27257 trim_cache size 0 max 16384 2023-09-23T04:52:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.105+0000 7f54b60bbb80 1 Processor -- start 2023-09-23T04:52:21.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 -- start start 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c251e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c246bd0 0x56288c23ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c00f920 con 0x56288c23bd20 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c23e7a0 con 0x56288c246bd0 2023-09-23T04:52:21.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c23c300 con 0x56288c23b400 2023-09-23T04:52:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f549bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f549bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:58344/0 (socket says 172.21.15.5:58344) 2023-09-23T04:52:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f549bfff700 1 -- 172.21.15.5:0/1892846243 learned_addr learned my addr 172.21.15.5:0/1892846243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f549b7fe700 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c251e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.106+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c246bd0 0x56288c23ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 -- 172.21.15.5:0/1892846243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c246bd0 msgr2=0x56288c23ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c246bd0 0x56288c23ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 -- 172.21.15.5:0/1892846243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 msgr2=0x56288c251e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c251e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 -- 172.21.15.5:0/1892846243 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56288c233480 con 0x56288c23b400 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f549bfff700 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f548c010c30 tx=0x7f548c065ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4101a1e6a052c7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f5482ffd700 1 -- 172.21.15.5:0/1892846243 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c065ea0 con 0x56288c23b400 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f5482ffd700 1 -- 172.21.15.5:0/1892846243 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f548c09ab10 con 0x56288c23b400 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.107+0000 7f5482ffd700 1 -- 172.21.15.5:0/1892846243 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0cae70 con 0x56288c23b400 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1892846243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 msgr2=0x56288c24d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f548c010c30 tx=0x7f548c065ba0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1892846243 shutdown_connections 2023-09-23T04:52:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c246bd0 0x56288c23ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23b400 0x56288c24d6f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/1892846243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c23bd20 0x56288c251e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1892846243 >> 172.21.15.5:0/1892846243 conn(0x56288c1b0260 msgr2=0x56288c258670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1892846243 shutdown_connections 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.108+0000 7f54b60bbb80 1 -- 172.21.15.5:0/1892846243 wait complete. 2023-09-23T04:52:21.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.109+0000 7f54b60bbb80 1 -- start start 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.109+0000 7f54b60bbb80 1 Processor -- start 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23bd20 0x56288c24b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c246bd0 0x56288c24bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c24dad0 con 0x56288c246bd0 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c24c150 con 0x56288c23b400 2023-09-23T04:52:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288c24c2c0 con 0x56288c23bd20 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54b60bbb80 10 client.0 _ll_register_callbacks cb 0x56288bf88800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:58024/0 (socket says 172.21.15.5:58024) 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 -- 172.21.15.5:0/3333040297 learned_addr learned my addr 172.21.15.5:0/3333040297 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549b7fe700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23bd20 0x56288c24b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f54a0d9c700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c246bd0 0x56288c24bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 -- 172.21.15.5:0/3333040297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23bd20 msgr2=0x56288c24b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23bd20 0x56288c24b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 -- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c246bd0 msgr2=0x56288c24bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c246bd0 0x56288c24bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.110+0000 7f549bfff700 1 -- 172.21.15.5:0/3333040297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1a40b0 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f549bfff700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f548c010cc0 tx=0x7f548c0cabe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ef936bc3b2a47d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c065ea0 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7f548c09e560 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f549400f040 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f548c09a410 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f54b60bbb80 10 client.27263 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56288bfa2c60 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f549400f040 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f548c09a790 con 0x56288c23b400 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f54b60bbb80 10 client.27263 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.111+0000 7f54b60bbb80 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f54b60bbb80 20 client.27263 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c2ab550 0x56288c2ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f54b60bbb80 4 client.27263 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56288c187b30 con 0x56288c2ab550 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f549b7fe700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c2ab550 0x56288c2ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f549b7fe700 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c2ab550 0x56288c2ad970 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1afc90990c48f5 server_cookie=b77b6e19b307eaea in_seq=0 out_seq=0 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.112+0000 7f547b7fe700 10 client.27263 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f547b7fe700 1 -- 172.21.15.5:0/3333040297 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56288c187b30 con 0x56288c2ab550 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f547b7fe700 10 client.27263 handle_command_reply: tid=0 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 client.27263 shutdown 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 msgr2=0x56288c24d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f548c010cc0 tx=0x7f548c0cabe0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 shutdown_connections 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56288c2ab550 0x56288c2ad970 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56288c246bd0 0x56288c24bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56288c23b400 0x56288c24d1c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 --2- 172.21.15.5:0/3333040297 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56288c23bd20 0x56288c24b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f547b7fe700 0 client.27263 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 >> 172.21.15.5:0/3333040297 conn(0x56288c1b0260 msgr2=0x56288c251e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 shutdown_connections 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.113+0000 7f54b60bbb80 1 -- 172.21.15.5:0/3333040297 wait complete. 2023-09-23T04:52:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.114+0000 7f54b60bbb80 20 client.27263 trim_cache size 0 max 16384 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:52:21.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:52:21.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:52:21.119 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:52:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.114+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 msgr2=0x7f54840a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.114+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 0x7f54840a3bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f549401b000 tx=0x7f549401d000 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.114+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 msgr2=0x7f549c196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.114+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9a0 tx=0x7f548c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 shutdown_connections 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f54840a1710 0x7f54840a3bd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f549c19b8c0 0x7f549c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f549c1010d0 0x7f549c196f70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 --2- 172.21.15.5:0/1209230166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c1061f0 0x7f549c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 >> 172.21.15.5:0/1209230166 conn(0x7f549c0fc990 msgr2=0x7f549c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 shutdown_connections 2023-09-23T04:52:21.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:21.115+0000 7f54a27ff700 1 -- 172.21.15.5:0/1209230166 wait complete. 2023-09-23T04:52:21.136 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: pgmap v185: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 KiB/s wr, 118 op/s 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:52:21.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: pgmap v185: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 KiB/s wr, 118 op/s 2023-09-23T04:52:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:52:21.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 365786 5.59 MB/sec execute 177 sec latency 16.726 ms 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: pgmap v185: 129 pgs: 129 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 KiB/s wr, 118 op/s 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:52:21.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:52:22.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 367032 5.59 MB/sec execute 178 sec latency 146.346 ms 2023-09-23T04:52:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:22 smithi005 ceph-mon[97638]: pgmap v186: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-23T04:52:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:22 smithi118 ceph-mon[134297]: pgmap v186: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-23T04:52:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:22 smithi069 ceph-mon[133931]: pgmap v186: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 158 op/s 2023-09-23T04:52:23.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 367032 5.56 MB/sec execute 179 sec latency 1146.394 ms 2023-09-23T04:52:24.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 367032 5.52 MB/sec execute 180 sec latency 2146.456 ms 2023-09-23T04:52:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:24 smithi005 ceph-mon[97638]: pgmap v187: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 865 KiB/s wr, 124 op/s 2023-09-23T04:52:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:24 smithi118 ceph-mon[134297]: pgmap v187: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 865 KiB/s wr, 124 op/s 2023-09-23T04:52:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:24 smithi069 ceph-mon[133931]: pgmap v187: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 865 KiB/s wr, 124 op/s 2023-09-23T04:52:25.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 367032 5.49 MB/sec execute 181 sec latency 3146.520 ms 2023-09-23T04:52:26.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 367166 5.46 MB/sec execute 182 sec latency 4089.344 ms 2023-09-23T04:52:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:26 smithi005 ceph-mon[97638]: pgmap v188: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 116 op/s 2023-09-23T04:52:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:26 smithi118 ceph-mon[134297]: pgmap v188: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 116 op/s 2023-09-23T04:52:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:26 smithi069 ceph-mon[133931]: pgmap v188: 129 pgs: 129 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 116 op/s 2023-09-23T04:52:27.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 369131 5.51 MB/sec execute 183 sec latency 15.011 ms 2023-09-23T04:52:28.948 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370336 5.51 MB/sec execute 184 sec latency 14.196 ms 2023-09-23T04:52:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:28 smithi005 ceph-mon[97638]: pgmap v189: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 139 op/s 2023-09-23T04:52:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:28 smithi118 ceph-mon[134297]: pgmap v189: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 139 op/s 2023-09-23T04:52:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:28 smithi069 ceph-mon[133931]: pgmap v189: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 139 op/s 2023-09-23T04:52:29.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370575 5.48 MB/sec execute 185 sec latency 832.486 ms 2023-09-23T04:52:30.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370575 5.45 MB/sec execute 186 sec latency 1832.527 ms 2023-09-23T04:52:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:31 smithi005 ceph-mon[97638]: pgmap v190: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 420 KiB/s wr, 66 op/s 2023-09-23T04:52:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:52:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:31 smithi118 ceph-mon[134297]: pgmap v190: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 420 KiB/s wr, 66 op/s 2023-09-23T04:52:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:52:31.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370695 5.42 MB/sec execute 187 sec latency 2774.732 ms 2023-09-23T04:52:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:31 smithi069 ceph-mon[133931]: pgmap v190: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 420 KiB/s wr, 66 op/s 2023-09-23T04:52:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:52:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:52:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:52:32.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:52:32.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:52:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:52:32.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:52:32.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 372664 5.46 MB/sec execute 188 sec latency 21.437 ms 2023-09-23T04:52:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:52:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:52:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:52:33.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:33 smithi005 ceph-mon[97638]: pgmap v191: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 695 KiB/s wr, 110 op/s 2023-09-23T04:52:33.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:33 smithi118 ceph-mon[134297]: pgmap v191: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 695 KiB/s wr, 110 op/s 2023-09-23T04:52:33.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 373539 5.45 MB/sec execute 189 sec latency 83.815 ms 2023-09-23T04:52:33.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:33 smithi069 ceph-mon[133931]: pgmap v191: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 695 KiB/s wr, 110 op/s 2023-09-23T04:52:34.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 374123 5.44 MB/sec execute 190 sec latency 660.094 ms 2023-09-23T04:52:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:34 smithi005 ceph-mon[97638]: pgmap v192: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 441 KiB/s wr, 72 op/s 2023-09-23T04:52:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:34 smithi118 ceph-mon[134297]: pgmap v192: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 441 KiB/s wr, 72 op/s 2023-09-23T04:52:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:34 smithi069 ceph-mon[133931]: pgmap v192: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 441 KiB/s wr, 72 op/s 2023-09-23T04:52:35.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 375768 5.45 MB/sec execute 191 sec latency 814.899 ms 2023-09-23T04:52:36.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 376439 5.45 MB/sec execute 192 sec latency 310.509 ms 2023-09-23T04:52:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:36 smithi005 ceph-mon[97638]: pgmap v193: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 415 KiB/s wr, 69 op/s 2023-09-23T04:52:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:36 smithi118 ceph-mon[134297]: pgmap v193: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 415 KiB/s wr, 69 op/s 2023-09-23T04:52:37.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:36 smithi069 ceph-mon[133931]: pgmap v193: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 415 KiB/s wr, 69 op/s 2023-09-23T04:52:37.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 377666 5.45 MB/sec execute 193 sec latency 18.530 ms 2023-09-23T04:52:38.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 377666 5.42 MB/sec execute 194 sec latency 1010.418 ms 2023-09-23T04:52:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:38 smithi005 ceph-mon[97638]: pgmap v194: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2023-09-23T04:52:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:38 smithi118 ceph-mon[134297]: pgmap v194: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2023-09-23T04:52:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:38 smithi069 ceph-mon[133931]: pgmap v194: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2023-09-23T04:52:39.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 377666 5.39 MB/sec execute 195 sec latency 2010.467 ms 2023-09-23T04:52:40.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 378795 5.40 MB/sec execute 196 sec latency 2538.911 ms 2023-09-23T04:52:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:41 smithi005 ceph-mon[97638]: pgmap v195: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-09-23T04:52:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:41 smithi118 ceph-mon[134297]: pgmap v195: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-09-23T04:52:41.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 380162 5.41 MB/sec execute 197 sec latency 15.916 ms 2023-09-23T04:52:41.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:41 smithi069 ceph-mon[133931]: pgmap v195: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2023-09-23T04:52:42.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381255 5.41 MB/sec execute 198 sec latency 179.171 ms 2023-09-23T04:52:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:42 smithi005 ceph-mon[97638]: pgmap v196: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-09-23T04:52:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:42 smithi118 ceph-mon[134297]: pgmap v196: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-09-23T04:52:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:42 smithi069 ceph-mon[133931]: pgmap v196: 129 pgs: 129 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-09-23T04:52:43.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381255 5.38 MB/sec execute 199 sec latency 1179.225 ms 2023-09-23T04:52:44.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381255 5.35 MB/sec execute 200 sec latency 2179.315 ms 2023-09-23T04:52:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:44 smithi005 ceph-mon[97638]: pgmap v197: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-23T04:52:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:44 smithi118 ceph-mon[134297]: pgmap v197: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-23T04:52:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:44 smithi069 ceph-mon[133931]: pgmap v197: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2023-09-23T04:52:45.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381255 5.33 MB/sec execute 201 sec latency 3179.367 ms 2023-09-23T04:52:46.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381390 5.30 MB/sec execute 202 sec latency 4123.374 ms 2023-09-23T04:52:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:46 smithi005 ceph-mon[97638]: pgmap v198: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2023-09-23T04:52:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:46 smithi118 ceph-mon[134297]: pgmap v198: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2023-09-23T04:52:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:46 smithi069 ceph-mon[133931]: pgmap v198: 129 pgs: 129 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2023-09-23T04:52:47.949 INFO:tasks.workunit.client.0.smithi005.stdout: 1 383360 5.34 MB/sec execute 203 sec latency 13.882 ms 2023-09-23T04:52:48.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 384557 5.34 MB/sec execute 204 sec latency 17.198 ms 2023-09-23T04:52:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:48 smithi005 ceph-mon[97638]: pgmap v199: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-09-23T04:52:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:48 smithi118 ceph-mon[134297]: pgmap v199: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-09-23T04:52:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:48 smithi069 ceph-mon[133931]: pgmap v199: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-09-23T04:52:49.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 384796 5.32 MB/sec execute 205 sec latency 714.200 ms 2023-09-23T04:52:50.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 384796 5.29 MB/sec execute 206 sec latency 1714.243 ms 2023-09-23T04:52:51.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:52:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 -- 172.21.15.5:0/306329969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0105f30 msgr2=0x7fa2b010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 --2- 172.21.15.5:0/306329969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0105f30 0x7fa2b010dab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800b0a0 tx=0x7fa2a800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 -- 172.21.15.5:0/306329969 shutdown_connections 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 --2- 172.21.15.5:0/306329969 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0105f30 0x7fa2b010dab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 --2- 172.21.15.5:0/306329969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0100e10 0x7fa2b01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 --2- 172.21.15.5:0/306329969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b01017c0 0x7fa2b0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 -- 172.21.15.5:0/306329969 >> 172.21.15.5:0/306329969 conn(0x7fa2b00fc670 msgr2=0x7fa2b00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.376+0000 7fa2af59e700 1 -- 172.21.15.5:0/306329969 shutdown_connections 2023-09-23T04:52:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.377+0000 7fa2af59e700 1 -- 172.21.15.5:0/306329969 wait complete. 2023-09-23T04:52:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 Processor -- start 2023-09-23T04:52:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 -- start start 2023-09-23T04:52:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0105f30 0x7fa2b0199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 0x7fa2b019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b019e470 con 0x7fa2b019d9f0 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b019e5e0 con 0x7fa2b0105f30 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.378+0000 7fa2af59e700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b019e7a0 con 0x7fa2b0100e10 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33742/0 (socket says 172.21.15.5:33742) 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2ae59c700 1 -- 172.21.15.5:0/265840525 learned_addr learned my addr 172.21.15.5:0/265840525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 0x7fa2b019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0105f30 0x7fa2b0199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 -- 172.21.15.5:0/265840525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 msgr2=0x7fa2b0199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 -- 172.21.15.5:0/265840525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0105f30 msgr2=0x7fa2b0199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0105f30 0x7fa2b0199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 -- 172.21.15.5:0/265840525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8009d00 con 0x7fa2b019d9f0 2023-09-23T04:52:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.379+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 0x7fa2b019de70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009fd0 tx=0x7fa2a8007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.380+0000 7fa2a77fe700 1 -- 172.21.15.5:0/265840525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800e040 con 0x7fa2b019d9f0 2023-09-23T04:52:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.380+0000 7fa2a77fe700 1 -- 172.21.15.5:0/265840525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a801d480 con 0x7fa2b019d9f0 2023-09-23T04:52:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.380+0000 7fa2a77fe700 1 -- 172.21.15.5:0/265840525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80052a0 con 0x7fa2b019d9f0 2023-09-23T04:52:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.380+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b01a5ca0 con 0x7fa2b019d9f0 2023-09-23T04:52:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.380+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b01a6070 con 0x7fa2b019d9f0 2023-09-23T04:52:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.383+0000 7fa2a77fe700 1 -- 172.21.15.5:0/265840525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa2a801d5f0 con 0x7fa2b019d9f0 2023-09-23T04:52:51.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.385+0000 7fa2c85e2b80 1 Processor -- start 2023-09-23T04:52:51.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.385+0000 7fa2a77fe700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 0x7fa29c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.385+0000 7fa2a77fe700 1 -- 172.21.15.5:0/265840525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa2a80ce230 con 0x7fa2b019d9f0 2023-09-23T04:52:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.385+0000 7fa2c85e2b80 1 -- start start 2023-09-23T04:52:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.385+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 0x7fa29c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dbd3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5db2e460 con 0x55de5dbd0d40 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5db1fc20 con 0x55de5dbc5e90 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5db1d540 con 0x55de5dbc5570 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33746/0 (socket says 172.21.15.5:33746) 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3904722978 learned_addr learned my addr 172.21.15.5:0/3904722978 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dbd3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 0x7fa29c0a3c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00115a0 tx=0x7fa2a0009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3904722978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 msgr2=0x55de5dbd0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.386+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3904722978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 msgr2=0x55de5dbd3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dbd3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3904722978 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5db7b8d0 con 0x55de5dbc5570 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dbd3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001c6d0 tx=0x7fa2a001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39fc0566b9d4611 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2a4ff9700 1 -- 172.21.15.5:0/3904722978 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0024430 con 0x55de5dbc5570 2023-09-23T04:52:51.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2a4ff9700 1 -- 172.21.15.5:0/3904722978 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a0024a50 con 0x55de5dbc5570 2023-09-23T04:52:51.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.387+0000 7fa2a4ff9700 1 -- 172.21.15.5:0/3904722978 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000fc20 con 0x55de5dbc5570 2023-09-23T04:52:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3904722978 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 msgr2=0x55de5dbc5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa2a001c6d0 tx=0x7fa2a001c000 comp rx=0 tx=0).stop 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3904722978 shutdown_connections 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dbd3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbc5950 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3904722978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3904722978 >> 172.21.15.5:0/3904722978 conn(0x55de5db3a260 msgr2=0x55de5dbc19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3904722978 shutdown_connections 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3904722978 wait complete. 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.388+0000 7fa2c85e2b80 1 -- start start 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 Processor -- start 2023-09-23T04:52:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 0x55de5dbcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5db18e30 con 0x55de5dbc5e90 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dab5710 con 0x55de5dbc5570 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dab53b0 con 0x55de5dbd0d40 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.389+0000 7fa2c85e2b80 10 client.0 _ll_register_callbacks cb 0x55de5d912800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 0x55de5dbcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:35674/0 (socket says 172.21.15.5:35674) 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2ae59c700 1 -- 172.21.15.5:0/1159554869 learned_addr learned my addr 172.21.15.5:0/1159554869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 -- 172.21.15.5:0/1159554869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 msgr2=0x55de5dbd0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 -- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 msgr2=0x55de5dbcda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 0x55de5dbcda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 -- 172.21.15.5:0/1159554869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0017050 con 0x55de5dbd0d40 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 0x55de5dbcda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbce150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80cb540 tx=0x7fa2a809a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad8783dc81b3b95 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.390+0000 7fa295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80a1450 con 0x55de5dbd0d40 2023-09-23T04:52:51.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de5dbb94a0 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa2a809b4f0 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a801db00 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 10 client.27328 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55de5d943de0 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de5dbb94a0 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa2a80b3250 con 0x55de5dbd0d40 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 10 client.27328 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 10 client.27328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 20 client.27328 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dbe0680 0x55de5dbe2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 4 client.27328 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:52:51.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.391+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de5db11e10 con 0x55de5dbe0680 2023-09-23T04:52:51.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.392+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dbe0680 0x55de5dbe2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.392+0000 7fa295ffb700 10 client.27328 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.392+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dbe0680 0x55de5dbe2aa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c793d56259c2d7 server_cookie=95e434051a97fdbd in_seq=0 out_seq=0 2023-09-23T04:52:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa295ffb700 1 -- 172.21.15.5:0/1159554869 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de5db11e10 con 0x55de5dbe0680 2023-09-23T04:52:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa295ffb700 10 client.27328 handle_command_reply: tid=0 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 client.27328 shutdown 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 msgr2=0x55de5dbce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbce150 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80cb540 tx=0x7fa2a809a7d0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 shutdown_connections 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dbe0680 0x55de5dbe2aa0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbce150 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5570 0x55de5dbd0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/1159554869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5e90 0x55de5dbcda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 >> 172.21.15.5:0/1159554869 conn(0x55de5db3a260 msgr2=0x55de5dbc15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.393+0000 7fa295ffb700 0 client.27328 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.394+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 shutdown_connections 2023-09-23T04:52:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.394+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/1159554869 wait complete. 2023-09-23T04:52:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.394+0000 7fa2c85e2b80 20 client.27328 trim_cache size 0 max 16384 2023-09-23T04:52:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.416+0000 7fa2c85e2b80 1 Processor -- start 2023-09-23T04:52:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 -- start start 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbd0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5d999920 con 0x55de5dbc5570 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dbc32d0 con 0x55de5dbc5e90 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dbbbca0 con 0x55de5dbd0d40 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52766/0 (socket says 172.21.15.5:52766) 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3897876177 learned_addr learned my addr 172.21.15.5:0/3897876177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbd0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.417+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3897876177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 msgr2=0x55de5dbd0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbd0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3897876177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 msgr2=0x55de5dbc8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbc8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3897876177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de5dbc03f0 con 0x55de5dbc5570 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2aed9d700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbd0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:52:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000f720 tx=0x7fa2a000f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8810f96204bf393 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2a57fa700 1 -- 172.21.15.5:0/3897876177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035520 con 0x55de5dbc5570 2023-09-23T04:52:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2a57fa700 1 -- 172.21.15.5:0/3897876177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a000f530 con 0x55de5dbc5570 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.418+0000 7fa2a57fa700 1 -- 172.21.15.5:0/3897876177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a000fa50 con 0x55de5dbc5570 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3897876177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 msgr2=0x55de5dbcd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000f720 tx=0x7fa2a000f400 comp rx=0 tx=0).stop 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3897876177 shutdown_connections 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbd0d40 0x55de5dbd0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbc5570 0x55de5dbcd9c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3897876177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbc8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3897876177 >> 172.21.15.5:0/3897876177 conn(0x55de5db3a260 msgr2=0x55de5dbcd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3897876177 shutdown_connections 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.419+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3897876177 wait complete. 2023-09-23T04:52:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.420+0000 7fa2c85e2b80 1 -- start start 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.420+0000 7fa2c85e2b80 1 Processor -- start 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dc33470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dbd6790 con 0x55de5dbd0d40 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dc339b0 con 0x55de5dbc5e90 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5dc33b20 con 0x55de5dbc5570 2023-09-23T04:52:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2c85e2b80 10 client.0 _ll_register_callbacks cb 0x55de5d912800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2aed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dc33470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:33764/0 (socket says 172.21.15.5:33764) 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3336302225 learned_addr learned my addr 172.21.15.5:0/3336302225 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3336302225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 msgr2=0x55de5dbd6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 msgr2=0x55de5dc33470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dc33470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.421+0000 7fa2ae59c700 1 -- 172.21.15.5:0/3336302225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5d943de0 con 0x55de5dbc5570 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa2ae59c700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0011d70 tx=0x7fa2a00054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2b6ffa14deff37 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0024630 con 0x55de5dbc5570 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2a0024a50 con 0x55de5dbc5570 2023-09-23T04:52:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9505+0+0 (secure 0 0 0) 0x7fa2a0024790 con 0x55de5dbc5570 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a000f7c0 con 0x55de5dbc5570 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa2c85e2b80 10 client.27334 fetch_fsmap learned FSMap version 37 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.422+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55de5db7b8d0 con 0x55de5dbc5570 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a0024a50 con 0x55de5dbc5570 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa2a003d070 con 0x55de5dbc5570 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 20 client.27334 populate_metadata read hostname 'smithi005' 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dc34f40 0x55de5dc37360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:52:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 4 client.27334 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de5db11b30 con 0x55de5dc34f40 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.423+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dc34f40 0x55de5dc37360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dc34f40 0x55de5dc37360 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9815dabea5435fc2 server_cookie=f4bb8edf38577847 in_seq=0 out_seq=0 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa295ffb700 10 client.27334 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa295ffb700 1 -- 172.21.15.5:0/3336302225 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55de5db11b30 con 0x55de5dc34f40 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa295ffb700 10 client.27334 handle_command_reply: tid=0 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 client.27334 shutdown 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 msgr2=0x55de5dbd5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0011d70 tx=0x7fa2a00054e0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 shutdown_connections 2023-09-23T04:52:51.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55de5dc34f40 0x55de5dc37360 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de5dbd0d40 0x55de5dc33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55de5dbc5570 0x55de5dbd5ac0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa295ffb700 0 client.27334 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 --2- 172.21.15.5:0/3336302225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55de5dbc5e90 0x55de5dbd6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.424+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 >> 172.21.15.5:0/3336302225 conn(0x55de5db3a260 msgr2=0x55de5dbcbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 shutdown_connections 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2c85e2b80 1 -- 172.21.15.5:0/3336302225 wait complete. 2023-09-23T04:52:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2c85e2b80 20 client.27334 trim_cache size 0 max 16384 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:52:51.430 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 msgr2=0x7fa29c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 0x7fa29c0a3c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00115a0 tx=0x7fa2a0009260 comp rx=0 tx=0).stop 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 msgr2=0x7fa2b019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.425+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 0x7fa2b019de70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009fd0 tx=0x7fa2a8007910 comp rx=0 tx=0).stop 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 shutdown_connections 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa29c0a1790 0x7fa29c0a3c50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b019d9f0 0x7fa2b019de70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa2b0100e10 0x7fa2b0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 --2- 172.21.15.5:0/265840525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0105f30 0x7fa2b0199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:52:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 >> 172.21.15.5:0/265840525 conn(0x7fa2b00fc670 msgr2=0x7fa2b00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:52:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 shutdown_connections 2023-09-23T04:52:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:52:51.426+0000 7fa2af59e700 1 -- 172.21.15.5:0/265840525 wait complete. 2023-09-23T04:52:51.447 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:52:51.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:51 smithi005 ceph-mon[97638]: pgmap v200: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 465 KiB/s wr, 77 op/s 2023-09-23T04:52:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:51 smithi118 ceph-mon[134297]: pgmap v200: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 465 KiB/s wr, 77 op/s 2023-09-23T04:52:51.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 384880 5.27 MB/sec execute 207 sec latency 2658.405 ms 2023-09-23T04:52:51.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:51 smithi069 ceph-mon[133931]: pgmap v200: 129 pgs: 129 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 465 KiB/s wr, 77 op/s 2023-09-23T04:52:52.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 386616 5.30 MB/sec execute 208 sec latency 14.153 ms 2023-09-23T04:52:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:52 smithi005 ceph-mon[97638]: pgmap v201: 129 pgs: 129 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 588 KiB/s wr, 101 op/s 2023-09-23T04:52:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:52 smithi118 ceph-mon[134297]: pgmap v201: 129 pgs: 129 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 588 KiB/s wr, 101 op/s 2023-09-23T04:52:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:52 smithi069 ceph-mon[133931]: pgmap v201: 129 pgs: 129 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 588 KiB/s wr, 101 op/s 2023-09-23T04:52:53.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 387624 5.29 MB/sec execute 209 sec latency 14.475 ms 2023-09-23T04:52:54.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 388368 5.28 MB/sec execute 210 sec latency 405.917 ms 2023-09-23T04:52:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:54 smithi005 ceph-mon[97638]: pgmap v202: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 531 KiB/s wr, 82 op/s 2023-09-23T04:52:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:54 smithi118 ceph-mon[134297]: pgmap v202: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 531 KiB/s wr, 82 op/s 2023-09-23T04:52:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:54 smithi069 ceph-mon[133931]: pgmap v202: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 531 KiB/s wr, 82 op/s 2023-09-23T04:52:55.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 390327 5.32 MB/sec execute 211 sec latency 534.781 ms 2023-09-23T04:52:56.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391499 5.31 MB/sec execute 212 sec latency 13.473 ms 2023-09-23T04:52:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:56 smithi005 ceph-mon[97638]: pgmap v203: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 412 KiB/s wr, 66 op/s 2023-09-23T04:52:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:56 smithi118 ceph-mon[134297]: pgmap v203: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 412 KiB/s wr, 66 op/s 2023-09-23T04:52:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:56 smithi069 ceph-mon[133931]: pgmap v203: 129 pgs: 129 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 412 KiB/s wr, 66 op/s 2023-09-23T04:52:57.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391911 5.30 MB/sec execute 213 sec latency 620.037 ms 2023-09-23T04:52:58.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391911 5.27 MB/sec execute 214 sec latency 1620.100 ms 2023-09-23T04:52:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:52:58 smithi005 ceph-mon[97638]: pgmap v204: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 KiB/s wr, 142 op/s 2023-09-23T04:52:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:52:58 smithi118 ceph-mon[134297]: pgmap v204: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 KiB/s wr, 142 op/s 2023-09-23T04:52:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:52:58 smithi069 ceph-mon[133931]: pgmap v204: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 947 KiB/s wr, 142 op/s 2023-09-23T04:52:59.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391911 5.25 MB/sec execute 215 sec latency 2620.163 ms 2023-09-23T04:53:00.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 392996 5.25 MB/sec execute 216 sec latency 3145.810 ms 2023-09-23T04:53:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:01 smithi005 ceph-mon[97638]: pgmap v205: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 803 KiB/s wr, 116 op/s 2023-09-23T04:53:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:01 smithi118 ceph-mon[134297]: pgmap v205: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 803 KiB/s wr, 116 op/s 2023-09-23T04:53:01.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 393155 5.23 MB/sec execute 217 sec latency 924.062 ms 2023-09-23T04:53:01.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:01 smithi069 ceph-mon[133931]: pgmap v205: 129 pgs: 129 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 803 KiB/s wr, 116 op/s 2023-09-23T04:53:02.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 393853 5.23 MB/sec execute 218 sec latency 61.499 ms 2023-09-23T04:53:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:02 smithi005 ceph-mon[97638]: pgmap v206: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 KiB/s wr, 154 op/s 2023-09-23T04:53:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:02 smithi118 ceph-mon[134297]: pgmap v206: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 KiB/s wr, 154 op/s 2023-09-23T04:53:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:02 smithi069 ceph-mon[133931]: pgmap v206: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1015 KiB/s wr, 154 op/s 2023-09-23T04:53:03.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 394988 5.23 MB/sec execute 219 sec latency 14.237 ms 2023-09-23T04:53:04.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 395459 5.22 MB/sec execute 220 sec latency 590.696 ms 2023-09-23T04:53:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:04 smithi005 ceph-mon[97638]: pgmap v207: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 916 KiB/s wr, 145 op/s 2023-09-23T04:53:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:04 smithi118 ceph-mon[134297]: pgmap v207: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 916 KiB/s wr, 145 op/s 2023-09-23T04:53:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:04 smithi069 ceph-mon[133931]: pgmap v207: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 916 KiB/s wr, 145 op/s 2023-09-23T04:53:05.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 395459 5.19 MB/sec execute 221 sec latency 1590.754 ms 2023-09-23T04:53:06.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 395543 5.17 MB/sec execute 222 sec latency 2539.938 ms 2023-09-23T04:53:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:06 smithi005 ceph-mon[97638]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T04:53:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:06 smithi005 ceph-mon[97638]: pgmap v209: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 925 KiB/s wr, 154 op/s 2023-09-23T04:53:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:06 smithi118 ceph-mon[134297]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T04:53:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:06 smithi118 ceph-mon[134297]: pgmap v209: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 925 KiB/s wr, 154 op/s 2023-09-23T04:53:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:06 smithi069 ceph-mon[133931]: osdmap e93: 12 total, 12 up, 12 in 2023-09-23T04:53:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:06 smithi069 ceph-mon[133931]: pgmap v209: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 925 KiB/s wr, 154 op/s 2023-09-23T04:53:07.950 INFO:tasks.workunit.client.0.smithi005.stdout: 1 397534 5.21 MB/sec execute 223 sec latency 57.604 ms 2023-09-23T04:53:08.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 398378 5.19 MB/sec execute 224 sec latency 167.808 ms 2023-09-23T04:53:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:08 smithi005 ceph-mon[97638]: pgmap v210: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2023-09-23T04:53:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:08 smithi118 ceph-mon[134297]: pgmap v210: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2023-09-23T04:53:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:08 smithi069 ceph-mon[133931]: pgmap v210: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2023-09-23T04:53:09.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 399001 5.18 MB/sec execute 225 sec latency 397.510 ms 2023-09-23T04:53:10.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 399001 5.16 MB/sec execute 226 sec latency 1397.591 ms 2023-09-23T04:53:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:11 smithi005 ceph-mon[97638]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T04:53:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:11 smithi005 ceph-mon[97638]: pgmap v212: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 980 KiB/s wr, 104 op/s 2023-09-23T04:53:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:11 smithi118 ceph-mon[134297]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T04:53:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:11 smithi118 ceph-mon[134297]: pgmap v212: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 980 KiB/s wr, 104 op/s 2023-09-23T04:53:11.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 399085 5.14 MB/sec execute 227 sec latency 2355.007 ms 2023-09-23T04:53:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:11 smithi069 ceph-mon[133931]: osdmap e94: 12 total, 12 up, 12 in 2023-09-23T04:53:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:11 smithi069 ceph-mon[133931]: pgmap v212: 129 pgs: 129 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 980 KiB/s wr, 104 op/s 2023-09-23T04:53:12.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 400807 5.17 MB/sec execute 228 sec latency 38.147 ms 2023-09-23T04:53:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:12 smithi005 ceph-mon[97638]: pgmap v213: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2023-09-23T04:53:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:12 smithi118 ceph-mon[134297]: pgmap v213: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2023-09-23T04:53:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:12 smithi069 ceph-mon[133931]: pgmap v213: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2023-09-23T04:53:13.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 401802 5.16 MB/sec execute 229 sec latency 15.883 ms 2023-09-23T04:53:14.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 402551 5.16 MB/sec execute 230 sec latency 407.137 ms 2023-09-23T04:53:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:14 smithi005 ceph-mon[97638]: pgmap v214: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2023-09-23T04:53:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:14 smithi118 ceph-mon[134297]: pgmap v214: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2023-09-23T04:53:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:14 smithi069 ceph-mon[133931]: pgmap v214: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2023-09-23T04:53:15.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 404480 5.19 MB/sec execute 231 sec latency 530.028 ms 2023-09-23T04:53:16.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 405460 5.18 MB/sec execute 232 sec latency 14.447 ms 2023-09-23T04:53:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:16 smithi005 ceph-mon[97638]: pgmap v215: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2023-09-23T04:53:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:16 smithi118 ceph-mon[134297]: pgmap v215: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2023-09-23T04:53:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:16 smithi069 ceph-mon[133931]: pgmap v215: 129 pgs: 129 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 186 op/s 2023-09-23T04:53:17.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 406092 5.17 MB/sec execute 233 sec latency 528.805 ms 2023-09-23T04:53:18.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 406092 5.15 MB/sec execute 234 sec latency 1528.869 ms 2023-09-23T04:53:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:18 smithi005 ceph-mon[97638]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2023-09-23T04:53:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:18 smithi118 ceph-mon[134297]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2023-09-23T04:53:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:18 smithi069 ceph-mon[133931]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2023-09-23T04:53:19.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 406092 5.13 MB/sec execute 235 sec latency 2528.933 ms 2023-09-23T04:53:20.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 407113 5.13 MB/sec execute 236 sec latency 3063.002 ms 2023-09-23T04:53:21.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:53:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 -- 172.21.15.5:0/3015115504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 msgr2=0x7fae04069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/3015115504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae04069480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fae000098f0 tx=0x7fae00009c00 comp rx=0 tx=0).stop 2023-09-23T04:53:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 -- 172.21.15.5:0/3015115504 shutdown_connections 2023-09-23T04:53:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/3015115504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae0410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/3015115504 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae04069480 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/3015115504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04069a50 0x7fae04105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 -- 172.21.15.5:0/3015115504 >> 172.21.15.5:0/3015115504 conn(0x7fae04077ca0 msgr2=0x7fae040780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 -- 172.21.15.5:0/3015115504 shutdown_connections 2023-09-23T04:53:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.720+0000 7fae0b9ff700 1 -- 172.21.15.5:0/3015115504 wait complete. 2023-09-23T04:53:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.721+0000 7fae0b9ff700 1 Processor -- start 2023-09-23T04:53:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.721+0000 7fae0b9ff700 1 -- start start 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae04100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04100a70 0x7fae041a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae04101f40 con 0x7fae04106210 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae041a9ee0 con 0x7fae04100a70 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0b9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae041aa020 con 0x7fae040690a0 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46990/0 (socket says 172.21.15.5:46990) 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae0979b700 1 -- 172.21.15.5:0/2741661214 learned_addr learned my addr 172.21.15.5:0/2741661214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae08f9a700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae04100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.722+0000 7fae09f9c700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04100a70 0x7fae041a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 -- 172.21.15.5:0/2741661214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04100a70 msgr2=0x7fae041a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04100a70 0x7fae041a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 -- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 msgr2=0x7fae04100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae04100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 -- 172.21.15.5:0/2741661214 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00009520 con 0x7fae040690a0 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae08f9a700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae04100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:53:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0979b700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae00009810 tx=0x7fae0000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fadfa7fc700 1 -- 172.21.15.5:0/2741661214 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0001c070 con 0x7fae040690a0 2023-09-23T04:53:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fadfa7fc700 1 -- 172.21.15.5:0/2741661214 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0000bb40 con 0x7fae040690a0 2023-09-23T04:53:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae041aa2a0 con 0x7fae040690a0 2023-09-23T04:53:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.723+0000 7fadfa7fc700 1 -- 172.21.15.5:0/2741661214 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00004930 con 0x7fae040690a0 2023-09-23T04:53:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.724+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041aa720 con 0x7fae040690a0 2023-09-23T04:53:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.728+0000 7fadfa7fc700 1 -- 172.21.15.5:0/2741661214 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fae00020070 con 0x7fae040690a0 2023-09-23T04:53:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.729+0000 7fae1f410b80 1 Processor -- start 2023-09-23T04:53:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.729+0000 7fae1f410b80 1 -- start start 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc51690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fbaf460 con 0x55ba4fc46400 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fba0c20 con 0x55ba4fc51bd0 2023-09-23T04:53:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae1f410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fb9e540 con 0x55ba4fc46d20 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae0979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc51690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37446/0 (socket says 172.21.15.5:37446) 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae0979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc51690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46998/0 (socket says 172.21.15.5:46998) 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae09f9c700 1 -- 172.21.15.5:0/1015129052 learned_addr learned my addr 172.21.15.5:0/1015129052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.730+0000 7fae08f9a700 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 -- 172.21.15.5:0/1015129052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 msgr2=0x55ba4fc51690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc51690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 -- 172.21.15.5:0/1015129052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 msgr2=0x55ba4fc53fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc53fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 -- 172.21.15.5:0/1015129052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4fbfc8d0 con 0x55ba4fc46400 2023-09-23T04:53:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae08f9a700 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fadfa7fc700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 0x7fadf00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.731+0000 7fae09f9c700 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fadfc011670 tx=0x7fadfc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b0a06fc92b18fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fadfa7fc700 1 -- 172.21.15.5:0/2741661214 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fae000ce410 con 0x7fae040690a0 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fadef7fe700 1 -- 172.21.15.5:0/1015129052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00e040 con 0x55ba4fc46400 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fae0979b700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 0x7fadf00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fadef7fe700 1 -- 172.21.15.5:0/1015129052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadfc00f040 con 0x55ba4fc46400 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fadef7fe700 1 -- 172.21.15.5:0/1015129052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc007500 con 0x55ba4fc46400 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fae1f410b80 1 -- 172.21.15.5:0/1015129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 msgr2=0x55ba4fc467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fae1f410b80 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fadfc011670 tx=0x7fadfc009920 comp rx=0 tx=0).stop 2023-09-23T04:53:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.732+0000 7fae0979b700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 0x7fadf00a3c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fae000979f0 tx=0x7fae000a1040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 -- 172.21.15.5:0/1015129052 shutdown_connections 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc467e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 --2- 172.21.15.5:0/1015129052 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc51690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 -- 172.21.15.5:0/1015129052 >> 172.21.15.5:0/1015129052 conn(0x55ba4fbbb260 msgr2=0x55ba4fc42790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 -- 172.21.15.5:0/1015129052 shutdown_connections 2023-09-23T04:53:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 -- 172.21.15.5:0/1015129052 wait complete. 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.733+0000 7fae1f410b80 1 -- start start 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 Processor -- start 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 0x55ba4fc4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 0x55ba4fc4edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fb99e30 con 0x55ba4fc46d20 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fb36710 con 0x55ba4fc51bd0 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fb363b0 con 0x55ba4fc46400 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae1f410b80 10 client.0 _ll_register_callbacks cb 0x55ba4f993800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.734+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 0x55ba4fc4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 0x55ba4fc4e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:47002/0 (socket says 172.21.15.5:47002) 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 -- 172.21.15.5:0/3924578191 learned_addr learned my addr 172.21.15.5:0/3924578191 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 0x55ba4fc4edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 -- 172.21.15.5:0/3924578191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 msgr2=0x55ba4fc4e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 0x55ba4fc4e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 -- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 msgr2=0x55ba4fc4edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 0x55ba4fc4edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 -- 172.21.15.5:0/3924578191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf4009cd0 con 0x55ba4fc51bd0 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 0x55ba4fc4edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4f330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fadfc005b90 tx=0x7fadfc004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ff8c6974ba5654 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.735+0000 7fadecff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:53:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00e040 con 0x55ba4fc51bd0 2023-09-23T04:53:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fadfc003c10 con 0x55ba4fc51bd0 2023-09-23T04:53:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba4fc3a250 con 0x55ba4fc51bd0 2023-09-23T04:53:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadfc003dd0 con 0x55ba4fc51bd0 2023-09-23T04:53:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fae1f410b80 10 client.27293 fetch_fsmap learned FSMap version 37 2023-09-23T04:53:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fadfc011350 con 0x55ba4fc51bd0 2023-09-23T04:53:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.736+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba4fc3a250 con 0x55ba4fc51bd0 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fadfc015430 con 0x55ba4fc51bd0 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 10 client.27293 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 10 client.27293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 20 client.27293 populate_metadata read hostname 'smithi005' 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fc61670 0x55ba4fc63a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 4 client.27293 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba4fb92e10 con 0x55ba4fc61670 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.737+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fc61670 0x55ba4fc63a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.738+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fc61670 0x55ba4fc63a90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa20b3310929b1de server_cookie=359cd5e9c0cd4def in_seq=0 out_seq=0 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.738+0000 7fadecff9700 10 client.27293 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.738+0000 7fadecff9700 1 -- 172.21.15.5:0/3924578191 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ba4fb92e10 con 0x55ba4fc61670 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.738+0000 7fadecff9700 10 client.27293 handle_command_reply: tid=0 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 client.27293 shutdown 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 msgr2=0x55ba4fc4f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4f330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fadfc005b90 tx=0x7fadfc004450 comp rx=0 tx=0).stop 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 shutdown_connections 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fc61670 0x55ba4fc63a90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4f330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46400 0x55ba4fc4e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fadecff9700 0 client.27293 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3924578191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46d20 0x55ba4fc4edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 >> 172.21.15.5:0/3924578191 conn(0x55ba4fbbb260 msgr2=0x55ba4fc42320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 shutdown_connections 2023-09-23T04:53:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.739+0000 7fae1f410b80 1 -- 172.21.15.5:0/3924578191 wait complete. 2023-09-23T04:53:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.740+0000 7fae1f410b80 20 client.27293 trim_cache size 0 max 16384 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 Processor -- start 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 -- start start 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 0x55ba4fc58710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fa1a920 con 0x55ba4fc46400 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fc49760 con 0x55ba4fc46d20 2023-09-23T04:53:21.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae1f410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fc3c8b0 con 0x55ba4fc51bd0 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae08f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 0x55ba4fc58710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc4db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37472/0 (socket says 172.21.15.5:37472) 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae08f9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 0x55ba4fc58710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:42204/0 (socket says 172.21.15.5:42204) 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.766+0000 7fae09f9c700 1 -- 172.21.15.5:0/2752744299 learned_addr learned my addr 172.21.15.5:0/2752744299 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 -- 172.21.15.5:0/2752744299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 msgr2=0x55ba4fc58710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 0x55ba4fc58710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 -- 172.21.15.5:0/2752744299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 msgr2=0x55ba4fc4db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc4db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 -- 172.21.15.5:0/2752744299 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4fc3e480 con 0x55ba4fc51bd0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fae09f9c700 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4bb60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009a00 tx=0x7fadfc005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b1420f040b80bc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fadeffff700 1 -- 172.21.15.5:0/2752744299 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00e050 con 0x55ba4fc51bd0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fadeffff700 1 -- 172.21.15.5:0/2752744299 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadfc015430 con 0x55ba4fc51bd0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.767+0000 7fadeffff700 1 -- 172.21.15.5:0/2752744299 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc003f70 con 0x55ba4fc51bd0 2023-09-23T04:53:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 -- 172.21.15.5:0/2752744299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 msgr2=0x55ba4fc4bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4bb60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009a00 tx=0x7fadfc005ae0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 -- 172.21.15.5:0/2752744299 shutdown_connections 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc4bb60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc46400 0x55ba4fc4db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 --2- 172.21.15.5:0/2752744299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46d20 0x55ba4fc58710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.768+0000 7fae1f410b80 1 -- 172.21.15.5:0/2752744299 >> 172.21.15.5:0/2752744299 conn(0x55ba4fbbb260 msgr2=0x55ba4fc49bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.769+0000 7fae1f410b80 1 -- 172.21.15.5:0/2752744299 shutdown_connections 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.769+0000 7fae1f410b80 1 -- 172.21.15.5:0/2752744299 wait complete. 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.769+0000 7fae1f410b80 1 -- start start 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.769+0000 7fae1f410b80 1 Processor -- start 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46400 0x55ba4fc56700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc57180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fcb4460 con 0x55ba4fc51bd0 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fcb45d0 con 0x55ba4fc46400 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4fcb4790 con 0x55ba4fc46d20 2023-09-23T04:53:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae1f410b80 10 client.0 _ll_register_callbacks cb 0x7fadfc011570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46400 0x55ba4fc56700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc57180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae09f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc57180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37478/0 (socket says 172.21.15.5:37478) 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae09f9c700 1 -- 172.21.15.5:0/3028773714 learned_addr learned my addr 172.21.15.5:0/3028773714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.770+0000 7fae08f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 -- 172.21.15.5:0/3028773714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46400 msgr2=0x55ba4fc56700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46400 0x55ba4fc56700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 -- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 msgr2=0x55ba4fc57180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc57180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 -- 172.21.15.5:0/3028773714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc011350 con 0x55ba4fc46d20 2023-09-23T04:53:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fae08f9a700 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc56c40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55ba4fc4ff60 tx=0x7fadf4021f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ee70c31ca20f17 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.771+0000 7fadecff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:53:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4017040 con 0x55ba4fc46d20 2023-09-23T04:53:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fadf4004030 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadfc00f040 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadf40041f0 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fae1f410b80 10 client.27352 fetch_fsmap learned FSMap version 37 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ba4f9c4de0 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadfc00f040 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.772+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fadf402a440 con 0x55ba4fc46d20 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 10 client.27352 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 20 client.27352 populate_metadata read hostname 'smithi005' 2023-09-23T04:53:21.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fcb6030 0x55ba4fcb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 4 client.27352 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba4fb92b30 con 0x55ba4fcb6030 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.773+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fcb6030 0x55ba4fcb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fadecff9700 10 client.27352 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae09f9c700 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fcb6030 0x55ba4fcb8450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad23c6eb5b563f9 server_cookie=52ac742e254e7ac in_seq=0 out_seq=0 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fadecff9700 1 -- 172.21.15.5:0/3028773714 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ba4fb92b30 con 0x55ba4fcb6030 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fadecff9700 10 client.27352 handle_command_reply: tid=0 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 client.27352 shutdown 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 msgr2=0x55ba4fc56c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc56c40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55ba4fc4ff60 tx=0x7fadf4021f30 comp rx=0 tx=0).stop 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 shutdown_connections 2023-09-23T04:53:21.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ba4fcb6030 0x55ba4fcb8450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba4fc51bd0 0x55ba4fc57180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba4fc46400 0x55ba4fc56700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 --2- 172.21.15.5:0/3028773714 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ba4fc46d20 0x55ba4fc56c40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 >> 172.21.15.5:0/3028773714 conn(0x55ba4fbbb260 msgr2=0x55ba4fc4e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.774+0000 7fadecff9700 0 client.27352 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.775+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 shutdown_connections 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.775+0000 7fae1f410b80 1 -- 172.21.15.5:0/3028773714 wait complete. 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.775+0000 7fae1f410b80 20 client.27352 trim_cache size 0 max 16384 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:53:21.779 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:53:21.780 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 msgr2=0x7fadf00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 0x7fadf00a3c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fae000979f0 tx=0x7fae000a1040 comp rx=0 tx=0).stop 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 msgr2=0x7fae040ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae00009810 tx=0x7fae0000fa00 comp rx=0 tx=0).stop 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 shutdown_connections 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fadf00a1790 0x7fadf00a3c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae04100a70 0x7fae041a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fae040690a0 0x7fae040ffff0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 --2- 172.21.15.5:0/2741661214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae04106210 0x7fae04100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 >> 172.21.15.5:0/2741661214 conn(0x7fae04077ca0 msgr2=0x7fae040fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 shutdown_connections 2023-09-23T04:53:21.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:21.776+0000 7fae0b9ff700 1 -- 172.21.15.5:0/2741661214 wait complete. 2023-09-23T04:53:21.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:21 smithi005 ceph-mon[97638]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-09-23T04:53:21.797 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:53:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:21 smithi118 ceph-mon[134297]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-09-23T04:53:21.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 408431 5.14 MB/sec execute 237 sec latency 13.586 ms 2023-09-23T04:53:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:21 smithi069 ceph-mon[133931]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 212 op/s 2023-09-23T04:53:22.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409641 5.14 MB/sec execute 238 sec latency 140.210 ms 2023-09-23T04:53:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:22 smithi005 ceph-mon[97638]: pgmap v218: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2023-09-23T04:53:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:22 smithi118 ceph-mon[134297]: pgmap v218: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2023-09-23T04:53:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:22 smithi069 ceph-mon[133931]: pgmap v218: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 221 op/s 2023-09-23T04:53:23.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409641 5.12 MB/sec execute 239 sec latency 1140.262 ms 2023-09-23T04:53:24.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409641 5.10 MB/sec execute 240 sec latency 2140.311 ms 2023-09-23T04:53:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:24 smithi005 ceph-mon[97638]: pgmap v219: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2023-09-23T04:53:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:24 smithi118 ceph-mon[134297]: pgmap v219: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2023-09-23T04:53:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:24 smithi069 ceph-mon[133931]: pgmap v219: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2023-09-23T04:53:25.951 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409641 5.08 MB/sec execute 241 sec latency 3140.355 ms 2023-09-23T04:53:26.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409775 5.06 MB/sec execute 242 sec latency 4082.143 ms 2023-09-23T04:53:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:26 smithi005 ceph-mon[97638]: pgmap v220: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 KiB/s wr, 124 op/s 2023-09-23T04:53:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:26 smithi118 ceph-mon[134297]: pgmap v220: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 KiB/s wr, 124 op/s 2023-09-23T04:53:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:26 smithi069 ceph-mon[133931]: pgmap v220: 129 pgs: 129 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 KiB/s wr, 124 op/s 2023-09-23T04:53:27.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 411726 5.09 MB/sec execute 243 sec latency 13.189 ms 2023-09-23T04:53:28.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 412901 5.09 MB/sec execute 244 sec latency 12.329 ms 2023-09-23T04:53:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:28 smithi005 ceph-mon[97638]: pgmap v221: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2023-09-23T04:53:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:28 smithi118 ceph-mon[134297]: pgmap v221: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2023-09-23T04:53:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:28 smithi069 ceph-mon[133931]: pgmap v221: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2023-09-23T04:53:29.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 413183 5.07 MB/sec execute 245 sec latency 784.484 ms 2023-09-23T04:53:30.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 413183 5.05 MB/sec execute 246 sec latency 1784.524 ms 2023-09-23T04:53:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:31 smithi005 ceph-mon[97638]: pgmap v222: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 527 KiB/s wr, 69 op/s 2023-09-23T04:53:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:31 smithi118 ceph-mon[134297]: pgmap v222: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 527 KiB/s wr, 69 op/s 2023-09-23T04:53:31.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 413302 5.03 MB/sec execute 247 sec latency 2727.765 ms 2023-09-23T04:53:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:31 smithi069 ceph-mon[133931]: pgmap v222: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 527 KiB/s wr, 69 op/s 2023-09-23T04:53:32.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:53:32.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:53:32.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 415257 5.07 MB/sec execute 248 sec latency 15.789 ms 2023-09-23T04:53:32.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:53:33.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:33 smithi005 ceph-mon[97638]: pgmap v223: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 KiB/s wr, 112 op/s 2023-09-23T04:53:33.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:33 smithi118 ceph-mon[134297]: pgmap v223: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 KiB/s wr, 112 op/s 2023-09-23T04:53:33.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 416148 5.06 MB/sec execute 249 sec latency 35.143 ms 2023-09-23T04:53:34.005 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:33 smithi069 ceph-mon[133931]: pgmap v223: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 KiB/s wr, 112 op/s 2023-09-23T04:53:34.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:53:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:53:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:53:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:53:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:53:34.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:53:34.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 416732 5.05 MB/sec execute 250 sec latency 626.228 ms 2023-09-23T04:53:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:53:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:53:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:53:35.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:35 smithi005 ceph-mon[97638]: pgmap v224: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-23T04:53:35.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:35 smithi118 ceph-mon[134297]: pgmap v224: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-23T04:53:35.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 418663 5.08 MB/sec execute 251 sec latency 748.602 ms 2023-09-23T04:53:35.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:35 smithi069 ceph-mon[133931]: pgmap v224: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 472 KiB/s wr, 72 op/s 2023-09-23T04:53:36.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 419719 5.07 MB/sec execute 252 sec latency 16.466 ms 2023-09-23T04:53:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:36 smithi005 ceph-mon[97638]: pgmap v225: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 447 KiB/s wr, 69 op/s 2023-09-23T04:53:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:36 smithi118 ceph-mon[134297]: pgmap v225: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 447 KiB/s wr, 69 op/s 2023-09-23T04:53:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:36 smithi069 ceph-mon[133931]: pgmap v225: 129 pgs: 129 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 447 KiB/s wr, 69 op/s 2023-09-23T04:53:37.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 420275 5.06 MB/sec execute 253 sec latency 508.131 ms 2023-09-23T04:53:38.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 420275 5.04 MB/sec execute 254 sec latency 1508.182 ms 2023-09-23T04:53:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:38 smithi005 ceph-mon[97638]: pgmap v226: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 KiB/s wr, 139 op/s 2023-09-23T04:53:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:38 smithi118 ceph-mon[134297]: pgmap v226: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 KiB/s wr, 139 op/s 2023-09-23T04:53:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:38 smithi069 ceph-mon[133931]: pgmap v226: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 KiB/s wr, 139 op/s 2023-09-23T04:53:39.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 420275 5.02 MB/sec execute 255 sec latency 2508.243 ms 2023-09-23T04:53:40.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 421345 5.03 MB/sec execute 256 sec latency 3036.605 ms 2023-09-23T04:53:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:41 smithi005 ceph-mon[97638]: pgmap v227: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 857 KiB/s wr, 115 op/s 2023-09-23T04:53:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:41 smithi118 ceph-mon[134297]: pgmap v227: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 857 KiB/s wr, 115 op/s 2023-09-23T04:53:41.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 422682 5.04 MB/sec execute 257 sec latency 15.657 ms 2023-09-23T04:53:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:41 smithi069 ceph-mon[133931]: pgmap v227: 129 pgs: 129 active+clean; 273 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 857 KiB/s wr, 115 op/s 2023-09-23T04:53:42.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423824 5.04 MB/sec execute 258 sec latency 45.301 ms 2023-09-23T04:53:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:42 smithi005 ceph-mon[97638]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-23T04:53:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:42 smithi118 ceph-mon[134297]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-23T04:53:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:42 smithi069 ceph-mon[133931]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2023-09-23T04:53:43.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423824 5.02 MB/sec execute 259 sec latency 1045.355 ms 2023-09-23T04:53:44.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423824 5.00 MB/sec execute 260 sec latency 2045.405 ms 2023-09-23T04:53:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:44 smithi005 ceph-mon[97638]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:53:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:44 smithi118 ceph-mon[134297]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:53:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:44 smithi069 ceph-mon[133931]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:53:45.952 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423824 4.98 MB/sec execute 261 sec latency 3045.458 ms 2023-09-23T04:53:46.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423959 4.96 MB/sec execute 262 sec latency 3985.543 ms 2023-09-23T04:53:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:46 smithi005 ceph-mon[97638]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 116 op/s 2023-09-23T04:53:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:46 smithi118 ceph-mon[134297]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 116 op/s 2023-09-23T04:53:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:46 smithi069 ceph-mon[133931]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1002 KiB/s wr, 116 op/s 2023-09-23T04:53:47.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 425500 4.98 MB/sec execute 263 sec latency 279.163 ms 2023-09-23T04:53:48.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 426143 4.98 MB/sec execute 264 sec latency 571.052 ms 2023-09-23T04:53:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:48 smithi005 ceph-mon[97638]: pgmap v231: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-09-23T04:53:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:48 smithi118 ceph-mon[134297]: pgmap v231: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-09-23T04:53:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:48 smithi069 ceph-mon[133931]: pgmap v231: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2023-09-23T04:53:49.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 427232 4.98 MB/sec execute 265 sec latency 29.923 ms 2023-09-23T04:53:50.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 427365 4.96 MB/sec execute 266 sec latency 882.917 ms 2023-09-23T04:53:51.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:53:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:51 smithi005 ceph-mon[97638]: pgmap v232: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 760 KiB/s wr, 77 op/s 2023-09-23T04:53:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:51 smithi118 ceph-mon[134297]: pgmap v232: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 760 KiB/s wr, 77 op/s 2023-09-23T04:53:51.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 427379 4.94 MB/sec execute 267 sec latency 1832.477 ms 2023-09-23T04:53:51.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:51 smithi069 ceph-mon[133931]: pgmap v232: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 760 KiB/s wr, 77 op/s 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.043+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/1952090823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb681083f0 msgr2=0x7fbb6810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.043+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/1952090823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb681083f0 0x7fbb6810ff70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009990 tx=0x7fbb60009ca0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/1952090823 shutdown_connections 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/1952090823 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb681083f0 0x7fbb6810ff70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/1952090823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb681036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/1952090823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb68103c80 0x7fbb68107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/1952090823 >> 172.21.15.5:0/1952090823 conn(0x7fbb680feb50 msgr2=0x7fbb68100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/1952090823 shutdown_connections 2023-09-23T04:53:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.044+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/1952090823 wait complete. 2023-09-23T04:53:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.045+0000 7fbb6c9ff700 1 Processor -- start 2023-09-23T04:53:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.045+0000 7fbb6c9ff700 1 -- start start 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb68199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 0x7fbb68199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 0x7fbb6819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819e560 con 0x7fbb681032d0 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819e6d0 con 0x7fbb681083f0 2023-09-23T04:53:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6c9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6819e890 con 0x7fbb6819dae0 2023-09-23T04:53:52.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb68199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb66d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 0x7fbb6819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb68199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58904/0 (socket says 172.21.15.5:58904) 2023-09-23T04:53:52.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.046+0000 7fbb6659c700 1 -- 172.21.15.5:0/3673978081 learned_addr learned my addr 172.21.15.5:0/3673978081 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 0x7fbb68199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 -- 172.21.15.5:0/3673978081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 msgr2=0x7fbb68199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 0x7fbb68199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 -- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 msgr2=0x7fbb68199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb68199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 -- 172.21.15.5:0/3673978081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600095f0 con 0x7fbb6819dae0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 0x7fbb68199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb66d9d700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 0x7fbb6819df60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb600098e0 tx=0x7fbb60009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.047+0000 7fbb577fe700 1 -- 172.21.15.5:0/3673978081 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6001d070 con 0x7fbb6819dae0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.048+0000 7fbb577fe700 1 -- 172.21.15.5:0/3673978081 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb600047e0 con 0x7fbb6819dae0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.048+0000 7fbb577fe700 1 -- 172.21.15.5:0/3673978081 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60005290 con 0x7fbb6819dae0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.048+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb681a5d90 con 0x7fbb6819dae0 2023-09-23T04:53:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.048+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb681a6100 con 0x7fbb6819dae0 2023-09-23T04:53:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.052+0000 7fbb577fe700 1 -- 172.21.15.5:0/3673978081 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fbb600053f0 con 0x7fbb6819dae0 2023-09-23T04:53:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.053+0000 7fbb80255b80 1 Processor -- start 2023-09-23T04:53:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.053+0000 7fbb80255b80 1 -- start start 2023-09-23T04:53:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e02400 0x562698e027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e0dbd0 0x562698e0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698d6b460 con 0x562698e0dbd0 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698d5cc20 con 0x562698e02400 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb80255b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698d5a540 con 0x562698e02d20 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:41370/0 (socket says 172.21.15.5:41370) 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb6659c700 1 -- 172.21.15.5:0/2555345905 learned_addr learned my addr 172.21.15.5:0/2555345905 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e02400 0x562698e027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e0dbd0 0x562698e0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.054+0000 7fbb577fe700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 0x7fbb500a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 -- 172.21.15.5:0/2555345905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e02400 msgr2=0x562698e027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e02400 0x562698e027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 -- 172.21.15.5:0/2555345905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e0dbd0 msgr2=0x562698e0ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e0dbd0 0x562698e0ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 -- 172.21.15.5:0/2555345905 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562698db88d0 con 0x562698e02d20 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb577fe700 1 -- 172.21.15.5:0/3673978081 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fbb600ce890 con 0x7fbb6819dae0 2023-09-23T04:53:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb66d9d700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 0x7fbb500a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e660 tx=0x7fbb58009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=449e3bee4e6969c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb54ff9700 1 -- 172.21.15.5:0/2555345905 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800e040 con 0x562698e02d20 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb54ff9700 1 -- 172.21.15.5:0/2555345905 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb5800f040 con 0x562698e02d20 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.055+0000 7fbb54ff9700 1 -- 172.21.15.5:0/2555345905 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007500 con 0x562698e02d20 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 -- 172.21.15.5:0/2555345905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 msgr2=0x562698e0d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb66d9d700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 0x7fbb500a3bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb60097e70 tx=0x7fbb6001a040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e660 tx=0x7fbb58009920 comp rx=0 tx=0).stop 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 -- 172.21.15.5:0/2555345905 shutdown_connections 2023-09-23T04:53:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e0dbd0 0x562698e0ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e02400 0x562698e027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2555345905 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02d20 0x562698e0d690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 -- 172.21.15.5:0/2555345905 >> 172.21.15.5:0/2555345905 conn(0x562698d77260 msgr2=0x562698dfe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.056+0000 7fbb80255b80 1 -- 172.21.15.5:0/2555345905 shutdown_connections 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.057+0000 7fbb80255b80 1 -- 172.21.15.5:0/2555345905 wait complete. 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.057+0000 7fbb80255b80 1 -- start start 2023-09-23T04:53:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.057+0000 7fbb80255b80 1 Processor -- start 2023-09-23T04:53:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e0a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698d55e30 con 0x562698e02d20 2023-09-23T04:53:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698cf2710 con 0x562698e0dbd0 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698cf23b0 con 0x562698e02400 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb80255b80 10 client.0 _ll_register_callbacks cb 0x562698b4f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb65d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.058+0000 7fbb65d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:41386/0 (socket says 172.21.15.5:41386) 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 -- 172.21.15.5:0/2950245134 learned_addr learned my addr 172.21.15.5:0/2950245134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e0a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 -- 172.21.15.5:0/2950245134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 msgr2=0x562698e0aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 -- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 msgr2=0x562698e0a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e0a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 -- 172.21.15.5:0/2950245134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5800e340 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c003f50 tx=0x7fbb5c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38243878c38a0fd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.059+0000 7fbb4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c012e10 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fbb5c00f450 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562698df6250 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb5c0266e0 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 10 client.27364 fetch_fsmap learned FSMap version 37 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562698b80de0 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562698df6250 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbb5c00f610 con 0x562698e02400 2023-09-23T04:53:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 10 client.27364 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.060+0000 7fbb80255b80 20 client.27364 populate_metadata read hostname 'smithi005' 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.061+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e1d630 0x562698e1fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.061+0000 7fbb80255b80 4 client.27364 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.061+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562698d4ee10 con 0x562698e1d630 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.061+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e1d630 0x562698e1fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.062+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e1d630 0x562698e1fa50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cf32447e9ee488 server_cookie=73f06b5d0a715f0d in_seq=0 out_seq=0 2023-09-23T04:53:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.062+0000 7fbb4dffb700 10 client.27364 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.062+0000 7fbb4dffb700 1 -- 172.21.15.5:0/2950245134 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562698d4ee10 con 0x562698e1d630 2023-09-23T04:53:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.062+0000 7fbb4dffb700 10 client.27364 handle_command_reply: tid=0 2023-09-23T04:53:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.062+0000 7fbb80255b80 1 client.27364 shutdown 2023-09-23T04:53:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 msgr2=0x562698e0d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c003f50 tx=0x7fbb5c0047c0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 shutdown_connections 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e1d630 0x562698e1fa50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e0d5a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2950245134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e0a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 >> 172.21.15.5:0/2950245134 conn(0x562698d77260 msgr2=0x562698dfe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb4dffb700 0 client.27364 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 shutdown_connections 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 1 -- 172.21.15.5:0/2950245134 wait complete. 2023-09-23T04:53:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.063+0000 7fbb80255b80 20 client.27364 trim_cache size 0 max 16384 2023-09-23T04:53:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.084+0000 7fbb80255b80 1 Processor -- start 2023-09-23T04:53:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.084+0000 7fbb80255b80 1 -- start start 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698bd6920 con 0x562698e02d20 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698e0be30 con 0x562698e0dbd0 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb80255b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698df88e0 con 0x562698e02400 2023-09-23T04:53:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58934/0 (socket says 172.21.15.5:58934) 2023-09-23T04:53:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb6659c700 1 -- 172.21.15.5:0/2589004564 learned_addr learned my addr 172.21.15.5:0/2589004564 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.085+0000 7fbb65d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 -- 172.21.15.5:0/2589004564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 msgr2=0x562698e04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 -- 172.21.15.5:0/2589004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 msgr2=0x562698e0a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 -- 172.21.15.5:0/2589004564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562698b69c60 con 0x562698e02d20 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:53:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.086+0000 7fbb6659c700 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e660 tx=0x7fbb58009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d76dcdcd77e8ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.087+0000 7fbb557fa700 1 -- 172.21.15.5:0/2589004564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58024040 con 0x562698e02d20 2023-09-23T04:53:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.087+0000 7fbb557fa700 1 -- 172.21.15.5:0/2589004564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb58003ec0 con 0x562698e02d20 2023-09-23T04:53:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.087+0000 7fbb557fa700 1 -- 172.21.15.5:0/2589004564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800e040 con 0x562698e02d20 2023-09-23T04:53:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- 172.21.15.5:0/2589004564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 msgr2=0x562698e050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e660 tx=0x7fbb58009b90 comp rx=0 tx=0).stop 2023-09-23T04:53:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- 172.21.15.5:0/2589004564 shutdown_connections 2023-09-23T04:53:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e0a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 --2- 172.21.15.5:0/2589004564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e050b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- 172.21.15.5:0/2589004564 >> 172.21.15.5:0/2589004564 conn(0x562698d77260 msgr2=0x562698e04460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- 172.21.15.5:0/2589004564 shutdown_connections 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- 172.21.15.5:0/2589004564 wait complete. 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.088+0000 7fbb80255b80 1 -- start start 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 Processor -- start 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e06c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e07180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698e076c0 con 0x562698e02d20 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698e07830 con 0x562698e0dbd0 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698e079f0 con 0x562698e02400 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb80255b80 10 client.0 _ll_register_callbacks cb 0x7fbb5c003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb65d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb65d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:41414/0 (socket says 172.21.15.5:41414) 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb65d9b700 1 -- 172.21.15.5:0/1553082162 learned_addr learned my addr 172.21.15.5:0/1553082162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.089+0000 7fbb6659c700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e06c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e07180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 -- 172.21.15.5:0/1553082162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 msgr2=0x562698e07180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e07180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 -- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 msgr2=0x562698e06c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e06c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 -- 172.21.15.5:0/1553082162 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562698db88d0 con 0x562698e02400 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb65d9b700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c003b30 tx=0x7fbb5c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d95af23261d18c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c009eb0 con 0x562698e02400 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fbb5c000e40 con 0x562698e02400 2023-09-23T04:53:52.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.090+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb5800f040 con 0x562698e02400 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb5c012e10 con 0x562698e02400 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 10 client.27370 fetch_fsmap learned FSMap version 37 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562698ba7250 con 0x562698e02400 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb5800f040 con 0x562698e02400 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbb5c032d20 con 0x562698e02400 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 10 client.27370 fetch_fsmap finished waiting for FSMap version 37 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 10 client.27370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 20 client.27370 populate_metadata read hostname 'smithi005' 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e728e0 0x562698e74d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:53:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 4 client.27370 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.091+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562698d4eb30 con 0x562698e728e0 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.092+0000 7fbb6659c700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e728e0 0x562698e74d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.092+0000 7fbb6659c700 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e728e0 0x562698e74d00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd77c911b08776ce server_cookie=f7ce5cbd3b1f979a in_seq=0 out_seq=0 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.092+0000 7fbb4dffb700 10 client.27370 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.093+0000 7fbb4dffb700 1 -- 172.21.15.5:0/1553082162 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562698d4eb30 con 0x562698e728e0 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.093+0000 7fbb4dffb700 10 client.27370 handle_command_reply: tid=0 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.093+0000 7fbb80255b80 1 client.27370 shutdown 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 msgr2=0x562698e146c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c003b30 tx=0x7fbb5c014040 comp rx=0 tx=0).stop 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 shutdown_connections 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562698e728e0 0x562698e74d00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb4dffb700 0 client.27370 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562698e0dbd0 0x562698e07180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562698e02400 0x562698e146c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 --2- 172.21.15.5:0/1553082162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562698e02d20 0x562698e06c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 >> 172.21.15.5:0/1553082162 conn(0x562698d77260 msgr2=0x562698e04cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 shutdown_connections 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 1 -- 172.21.15.5:0/1553082162 wait complete. 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.094+0000 7fbb80255b80 20 client.27370 trim_cache size 0 max 16384 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:53:52.097 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:53:52.098 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 msgr2=0x7fbb500a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 0x7fbb500a3bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb60097e70 tx=0x7fbb6001a040 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 msgr2=0x7fbb6819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 0x7fbb6819df60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb600098e0 tx=0x7fbb60009580 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 shutdown_connections 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbb500a1710 0x7fbb500a3bd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbb6819dae0 0x7fbb6819df60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb681032d0 0x7fbb68199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 --2- 172.21.15.5:0/3673978081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbb681083f0 0x7fbb68199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:53:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 >> 172.21.15.5:0/3673978081 conn(0x7fbb680feb50 msgr2=0x7fbb680ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:53:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 shutdown_connections 2023-09-23T04:53:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:53:52.095+0000 7fbb6c9ff700 1 -- 172.21.15.5:0/3673978081 wait complete. 2023-09-23T04:53:52.114 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:53:52.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 429187 4.97 MB/sec execute 268 sec latency 15.907 ms 2023-09-23T04:53:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:52 smithi005 ceph-mon[97638]: pgmap v233: 129 pgs: 129 active+clean; 280 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-09-23T04:53:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:52 smithi118 ceph-mon[134297]: pgmap v233: 129 pgs: 129 active+clean; 280 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-09-23T04:53:53.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:52 smithi069 ceph-mon[133931]: pgmap v233: 129 pgs: 129 active+clean; 280 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2023-09-23T04:53:53.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 430261 4.96 MB/sec execute 269 sec latency 11.969 ms 2023-09-23T04:53:54.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 430914 4.96 MB/sec execute 270 sec latency 421.486 ms 2023-09-23T04:53:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:54 smithi005 ceph-mon[97638]: pgmap v234: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2023-09-23T04:53:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:54 smithi118 ceph-mon[134297]: pgmap v234: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2023-09-23T04:53:55.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:54 smithi069 ceph-mon[133931]: pgmap v234: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 142 op/s 2023-09-23T04:53:55.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 432842 4.99 MB/sec execute 271 sec latency 541.895 ms 2023-09-23T04:53:56.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 433901 4.98 MB/sec execute 272 sec latency 17.373 ms 2023-09-23T04:53:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:56 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:53:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:56 smithi005 ceph-mon[97638]: pgmap v235: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2023-09-23T04:53:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:56 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:53:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:56 smithi118 ceph-mon[134297]: pgmap v235: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2023-09-23T04:53:57.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:56 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:53:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:56 smithi069 ceph-mon[133931]: pgmap v235: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2023-09-23T04:53:57.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 434458 4.97 MB/sec execute 273 sec latency 660.136 ms 2023-09-23T04:53:58.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 434458 4.96 MB/sec execute 274 sec latency 1660.185 ms 2023-09-23T04:53:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:53:58 smithi005 ceph-mon[97638]: pgmap v236: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2023-09-23T04:53:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:53:58 smithi118 ceph-mon[134297]: pgmap v236: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2023-09-23T04:53:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:53:58 smithi069 ceph-mon[133931]: pgmap v236: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2023-09-23T04:53:59.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 434458 4.94 MB/sec execute 275 sec latency 2660.243 ms 2023-09-23T04:54:00.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 435524 4.94 MB/sec execute 276 sec latency 3181.619 ms 2023-09-23T04:54:01.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:01 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:01.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:01 smithi005 ceph-mon[97638]: pgmap v237: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-09-23T04:54:01.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:01 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:01.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:01 smithi118 ceph-mon[134297]: pgmap v237: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-09-23T04:54:01.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:01 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:01.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:01 smithi069 ceph-mon[133931]: pgmap v237: 129 pgs: 129 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2023-09-23T04:54:01.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 436121 4.94 MB/sec execute 277 sec latency 59.408 ms 2023-09-23T04:54:02.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 437226 4.94 MB/sec execute 278 sec latency 13.126 ms 2023-09-23T04:54:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:02 smithi005 ceph-mon[97638]: pgmap v238: 129 pgs: 129 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2023-09-23T04:54:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:02 smithi118 ceph-mon[134297]: pgmap v238: 129 pgs: 129 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2023-09-23T04:54:03.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:02 smithi069 ceph-mon[133931]: pgmap v238: 129 pgs: 129 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 226 op/s 2023-09-23T04:54:03.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 438005 4.94 MB/sec execute 279 sec latency 490.766 ms 2023-09-23T04:54:04.953 INFO:tasks.workunit.client.0.smithi005.stdout: 1 438005 4.92 MB/sec execute 280 sec latency 1490.825 ms 2023-09-23T04:54:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:04 smithi005 ceph-mon[97638]: pgmap v239: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-23T04:54:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:04 smithi118 ceph-mon[134297]: pgmap v239: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-23T04:54:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:04 smithi069 ceph-mon[133931]: pgmap v239: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 179 op/s 2023-09-23T04:54:05.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 438005 4.90 MB/sec execute 281 sec latency 2490.872 ms 2023-09-23T04:54:06.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 438089 4.88 MB/sec execute 282 sec latency 3434.524 ms 2023-09-23T04:54:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:06 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:06 smithi005 ceph-mon[97638]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T04:54:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:06 smithi005 ceph-mon[97638]: pgmap v241: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2023-09-23T04:54:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:06 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:06 smithi118 ceph-mon[134297]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T04:54:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:06 smithi118 ceph-mon[134297]: pgmap v241: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2023-09-23T04:54:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:06 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:06 smithi069 ceph-mon[133931]: osdmap e95: 12 total, 12 up, 12 in 2023-09-23T04:54:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:06 smithi069 ceph-mon[133931]: pgmap v241: 129 pgs: 129 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2023-09-23T04:54:07.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 440072 4.91 MB/sec execute 283 sec latency 18.327 ms 2023-09-23T04:54:08.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 441145 4.90 MB/sec execute 284 sec latency 12.929 ms 2023-09-23T04:54:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:08 smithi005 ceph-mon[97638]: pgmap v242: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 770 KiB/s wr, 124 op/s 2023-09-23T04:54:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:08 smithi118 ceph-mon[134297]: pgmap v242: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 770 KiB/s wr, 124 op/s 2023-09-23T04:54:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:08 smithi069 ceph-mon[133931]: pgmap v242: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 770 KiB/s wr, 124 op/s 2023-09-23T04:54:09.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 441547 4.90 MB/sec execute 285 sec latency 613.438 ms 2023-09-23T04:54:10.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 441547 4.88 MB/sec execute 286 sec latency 1613.481 ms 2023-09-23T04:54:11.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:11 smithi005 ceph-mon[97638]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T04:54:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:11 smithi005 ceph-mon[97638]: pgmap v244: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 541 KiB/s wr, 88 op/s 2023-09-23T04:54:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:11 smithi118 ceph-mon[134297]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T04:54:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:11 smithi118 ceph-mon[134297]: pgmap v244: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 541 KiB/s wr, 88 op/s 2023-09-23T04:54:11.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 441681 4.86 MB/sec execute 287 sec latency 2553.301 ms 2023-09-23T04:54:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:11 smithi069 ceph-mon[133931]: osdmap e96: 12 total, 12 up, 12 in 2023-09-23T04:54:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:11 smithi069 ceph-mon[133931]: pgmap v244: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 541 KiB/s wr, 88 op/s 2023-09-23T04:54:12.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 443615 4.89 MB/sec execute 288 sec latency 14.735 ms 2023-09-23T04:54:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:12 smithi005 ceph-mon[97638]: pgmap v245: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 KiB/s wr, 95 op/s 2023-09-23T04:54:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:12 smithi118 ceph-mon[134297]: pgmap v245: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 KiB/s wr, 95 op/s 2023-09-23T04:54:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:12 smithi069 ceph-mon[133931]: pgmap v245: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 779 KiB/s wr, 95 op/s 2023-09-23T04:54:13.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 444572 4.89 MB/sec execute 289 sec latency 55.546 ms 2023-09-23T04:54:14.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 445096 4.88 MB/sec execute 290 sec latency 626.407 ms 2023-09-23T04:54:15.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:14 smithi005 ceph-mon[97638]: pgmap v246: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 806 KiB/s wr, 105 op/s 2023-09-23T04:54:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:14 smithi118 ceph-mon[134297]: pgmap v246: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 806 KiB/s wr, 105 op/s 2023-09-23T04:54:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:14 smithi069 ceph-mon[133931]: pgmap v246: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 806 KiB/s wr, 105 op/s 2023-09-23T04:54:15.958 INFO:tasks.workunit.client.0.smithi005.stdout: 1 446900 4.90 MB/sec execute 291 sec latency 745.750 ms 2023-09-23T04:54:16.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 447870 4.90 MB/sec execute 292 sec latency 15.038 ms 2023-09-23T04:54:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:16 smithi005 ceph-mon[97638]: pgmap v247: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 718 KiB/s wr, 94 op/s 2023-09-23T04:54:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:16 smithi118 ceph-mon[134297]: pgmap v247: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 718 KiB/s wr, 94 op/s 2023-09-23T04:54:17.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:16 smithi069 ceph-mon[133931]: pgmap v247: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 718 KiB/s wr, 94 op/s 2023-09-23T04:54:17.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 448638 4.90 MB/sec execute 293 sec latency 414.669 ms 2023-09-23T04:54:18.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 448638 4.88 MB/sec execute 294 sec latency 1414.724 ms 2023-09-23T04:54:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:18 smithi005 ceph-mon[97638]: pgmap v248: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-09-23T04:54:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:18 smithi118 ceph-mon[134297]: pgmap v248: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-09-23T04:54:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:18 smithi069 ceph-mon[133931]: pgmap v248: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-09-23T04:54:19.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 448638 4.86 MB/sec execute 295 sec latency 2414.790 ms 2023-09-23T04:54:20.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 449548 4.85 MB/sec execute 296 sec latency 2947.092 ms 2023-09-23T04:54:21.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:21 smithi005 ceph-mon[97638]: pgmap v249: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2023-09-23T04:54:21.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:21 smithi118 ceph-mon[134297]: pgmap v249: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2023-09-23T04:54:21.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 450963 4.88 MB/sec execute 297 sec latency 13.475 ms 2023-09-23T04:54:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:21 smithi069 ceph-mon[133931]: pgmap v249: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2023-09-23T04:54:22.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:54:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.399+0000 7fb780dff700 1 -- 172.21.15.5:0/368368805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103cc0 msgr2=0x7fb77c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 --2- 172.21.15.5:0/368368805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103cc0 0x7fb77c107d00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb770009990 tx=0x7fb770009ca0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 -- 172.21.15.5:0/368368805 shutdown_connections 2023-09-23T04:54:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 --2- 172.21.15.5:0/368368805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c108430 0x7fb77c10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 --2- 172.21.15.5:0/368368805 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c103310 0x7fb77c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 --2- 172.21.15.5:0/368368805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103cc0 0x7fb77c107d00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 -- 172.21.15.5:0/368368805 >> 172.21.15.5:0/368368805 conn(0x7fb77c0feb50 msgr2=0x7fb77c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 -- 172.21.15.5:0/368368805 shutdown_connections 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.400+0000 7fb780dff700 1 -- 172.21.15.5:0/368368805 wait complete. 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.401+0000 7fb780dff700 1 Processor -- start 2023-09-23T04:54:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 -- start start 2023-09-23T04:54:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c108430 0x7fb77c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 0x7fb77c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c19e5d0 con 0x7fb77c19db50 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c19e740 con 0x7fb77c103310 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb780dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c19e900 con 0x7fb77c108430 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.402+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c108430 0x7fb77c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38400/0 (socket says 172.21.15.5:38400) 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 -- 172.21.15.5:0/3936378439 learned_addr learned my addr 172.21.15.5:0/3936378439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 0x7fb77c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 -- 172.21.15.5:0/3936378439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c108430 msgr2=0x7fb77c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c108430 0x7fb77c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 -- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 msgr2=0x7fb77c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 0x7fb77c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 -- 172.21.15.5:0/3936378439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700095f0 con 0x7fb77c103310 2023-09-23T04:54:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 0x7fb77c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb77a59c700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00e9a0 tx=0x7fb76c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb76b7fe700 1 -- 172.21.15.5:0/3936378439 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c009790 con 0x7fb77c103310 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb76b7fe700 1 -- 172.21.15.5:0/3936378439 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76c00cbd0 con 0x7fb77c103310 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.403+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c1a5e60 con 0x7fb77c103310 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.404+0000 7fb76b7fe700 1 -- 172.21.15.5:0/3936378439 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c018680 con 0x7fb77c103310 2023-09-23T04:54:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.404+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c1a6330 con 0x7fb77c103310 2023-09-23T04:54:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.408+0000 7fb76b7fe700 1 -- 172.21.15.5:0/3936378439 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb76c00cd40 con 0x7fb77c103310 2023-09-23T04:54:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.410+0000 7fb7946fab80 1 Processor -- start 2023-09-23T04:54:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 -- start start 2023-09-23T04:54:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a46f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4703690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4705fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a4661460 con 0x5571a4703bd0 2023-09-23T04:54:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a4652c20 con 0x5571a46f8400 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb7946fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a4650540 con 0x5571a46f8d20 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb76b7fe700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 0x7fb7640a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb76b7fe700 1 -- 172.21.15.5:0/3936378439 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb76c014070 con 0x7fb77c103310 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.411+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a46f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4703690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4703690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:50286/0 (socket says 172.21.15.5:50286) 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77a59c700 1 -- 172.21.15.5:0/113200987 learned_addr learned my addr 172.21.15.5:0/113200987 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb779d9b700 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4705fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 0x7fb7640a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 -- 172.21.15.5:0/113200987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 msgr2=0x5571a4703690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4703690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 -- 172.21.15.5:0/113200987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 msgr2=0x5571a4705fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4705fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.412+0000 7fb77ad9d700 1 -- 172.21.15.5:0/113200987 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a46ae8d0 con 0x5571a46f8400 2023-09-23T04:54:22.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.413+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a46f87e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb774014bf0 tx=0x7fb774014f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d586b17ba5c4ebb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.413+0000 7fb768ff9700 1 -- 172.21.15.5:0/113200987 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400e040 con 0x5571a46f8400 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.413+0000 7fb768ff9700 1 -- 172.21.15.5:0/113200987 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7740092b0 con 0x5571a46f8400 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.413+0000 7fb768ff9700 1 -- 172.21.15.5:0/113200987 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740197a0 con 0x5571a46f8400 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 -- 172.21.15.5:0/113200987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 msgr2=0x5571a46f87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 0x7fb7640a3c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb774021000 tx=0x7fb77401c000 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a46f87e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb774014bf0 tx=0x7fb774014f00 comp rx=0 tx=0).stop 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 -- 172.21.15.5:0/113200987 shutdown_connections 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4705fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a46f87e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 --2- 172.21.15.5:0/113200987 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4703690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 -- 172.21.15.5:0/113200987 >> 172.21.15.5:0/113200987 conn(0x5571a466d260 msgr2=0x5571a46f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 -- 172.21.15.5:0/113200987 shutdown_connections 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.414+0000 7fb7946fab80 1 -- 172.21.15.5:0/113200987 wait complete. 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.415+0000 7fb7946fab80 1 -- start start 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.415+0000 7fb7946fab80 1 Processor -- start 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a47035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4700f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a464be30 con 0x5571a4703bd0 2023-09-23T04:54:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a45e8710 con 0x5571a46f8400 2023-09-23T04:54:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a45e83b0 con 0x5571a46f8d20 2023-09-23T04:54:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb7946fab80 10 client.0 _ll_register_callbacks cb 0x5571a4445800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:54:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb779d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:50302/0 (socket says 172.21.15.5:50302) 2023-09-23T04:54:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb779d9b700 1 -- 172.21.15.5:0/1139194179 learned_addr learned my addr 172.21.15.5:0/1139194179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4700f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.416+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a47035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1139194179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 msgr2=0x5571a4700870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1139194179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 msgr2=0x5571a47035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a47035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1139194179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7740147a0 con 0x5571a4703bd0 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb779d9b700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4700f10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb774015010 tx=0x7fb7740198c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a25ff7ea0ba4499 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774004c70 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.417+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a46ec250 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb774004dd0 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7740077e0 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb7946fab80 10 client.17331 fetch_fsmap learned FSMap version 40 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5571a4476de0 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571a46ec250 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb774014050 con 0x5571a4703bd0 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb7946fab80 10 client.17331 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T04:54:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb7946fab80 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.418+0000 7fb7946fab80 20 client.17331 populate_metadata read hostname 'smithi005' 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.419+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a4713470 0x5571a4715890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.419+0000 7fb7946fab80 4 client.17331 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.419+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a4644e10 con 0x5571a4713470 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.419+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a4713470 0x5571a4715890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.420+0000 7fb761ffb700 10 client.17331 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.420+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a4713470 0x5571a4715890 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5844257be2e2aafb server_cookie=9ee3028397eab4ae in_seq=0 out_seq=0 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.420+0000 7fb761ffb700 1 -- 172.21.15.5:0/1139194179 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5571a4644e10 con 0x5571a4713470 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb761ffb700 10 client.17331 handle_command_reply: tid=0 2023-09-23T04:54:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 client.17331 shutdown 2023-09-23T04:54:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 msgr2=0x5571a4700f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4700f10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb774015010 tx=0x7fb7740198c0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 shutdown_connections 2023-09-23T04:54:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a4713470 0x5571a4715890 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4700f10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8400 0x5571a47035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1139194179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a4700870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 >> 172.21.15.5:0/1139194179 conn(0x5571a466d260 msgr2=0x5571a46f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb761ffb700 0 client.17331 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.421+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 shutdown_connections 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.422+0000 7fb7946fab80 1 -- 172.21.15.5:0/1139194179 wait complete. 2023-09-23T04:54:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.422+0000 7fb7946fab80 20 client.17331 trim_cache size 0 max 16384 2023-09-23T04:54:22.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.448+0000 7fb7946fab80 1 Processor -- start 2023-09-23T04:54:22.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.448+0000 7fb7946fab80 1 -- start start 2023-09-23T04:54:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 0x5571a46fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a46fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a44cc920 con 0x5571a46f8400 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a46f55e0 con 0x5571a4703bd0 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb7946fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a46fa290 con 0x5571a46f8d20 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 0x5571a46fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38430/0 (socket says 172.21.15.5:38430) 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77ad9d700 1 -- 172.21.15.5:0/3923343245 learned_addr learned my addr 172.21.15.5:0/3923343245 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 0x5571a46fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48952/0 (socket says 172.21.15.5:48952) 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb779d9b700 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a46fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77ad9d700 1 -- 172.21.15.5:0/3923343245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 msgr2=0x5571a46fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.449+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a46fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb77ad9d700 1 -- 172.21.15.5:0/3923343245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 msgr2=0x5571a46fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 0x5571a46fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb77ad9d700 1 -- 172.21.15.5:0/3923343245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a445fc60 con 0x5571a4703bd0 2023-09-23T04:54:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb774019950 tx=0x7fb774014f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c179c04c3a9bd0ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb7697fa700 1 -- 172.21.15.5:0/3923343245 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400e040 con 0x5571a4703bd0 2023-09-23T04:54:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb7697fa700 1 -- 172.21.15.5:0/3923343245 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb774005850 con 0x5571a4703bd0 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.450+0000 7fb7697fa700 1 -- 172.21.15.5:0/3923343245 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774004cd0 con 0x5571a4703bd0 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 -- 172.21.15.5:0/3923343245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 msgr2=0x5571a4700050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb774019950 tx=0x7fb774014f00 comp rx=0 tx=0).stop 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 -- 172.21.15.5:0/3923343245 shutdown_connections 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a4703bd0 0x5571a4700050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a46f8400 0x5571a46fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 --2- 172.21.15.5:0/3923343245 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8d20 0x5571a46fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 -- 172.21.15.5:0/3923343245 >> 172.21.15.5:0/3923343245 conn(0x5571a466d260 msgr2=0x5571a4715480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 -- 172.21.15.5:0/3923343245 shutdown_connections 2023-09-23T04:54:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.451+0000 7fb7946fab80 1 -- 172.21.15.5:0/3923343245 wait complete. 2023-09-23T04:54:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.452+0000 7fb7946fab80 1 -- start start 2023-09-23T04:54:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.452+0000 7fb7946fab80 1 Processor -- start 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8400 0x5571a4708700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8d20 0x5571a4708c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a47662f0 con 0x5571a4703bd0 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a4766430 con 0x5571a46f8d20 2023-09-23T04:54:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a47665f0 con 0x5571a46f8400 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb7946fab80 10 client.0 _ll_register_callbacks cb 0x7fb7740097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb77ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48966/0 (socket says 172.21.15.5:48966) 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.453+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1709842578 learned_addr learned my addr 172.21.15.5:0/1709842578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1709842578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8400 msgr2=0x5571a4708700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8400 0x5571a4708700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb779d9b700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8d20 0x5571a4708c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8400 0x5571a4708700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1709842578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8d20 msgr2=0x5571a4708c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8d20 0x5571a4708c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 -- 172.21.15.5:0/1709842578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571a4476de0 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb77ad9d700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb774014a90 tx=0x7fb774005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2637bffb93bd0d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.454+0000 7fb761ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400e040 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7740092b0 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb774019c50 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb774019e10 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb7946fab80 10 client.17337 fetch_fsmap learned FSMap version 40 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5571a449d250 con 0x5571a4703bd0 2023-09-23T04:54:22.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7740092b0 con 0x5571a4703bd0 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb774017350 con 0x5571a4703bd0 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb7946fab80 10 client.17337 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.455+0000 7fb7946fab80 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb7946fab80 20 client.17337 populate_metadata read hostname 'smithi005' 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a47679c0 0x5571a4769de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb7946fab80 4 client.17337 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571a4644b30 con 0x5571a47679c0 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a47679c0 0x5571a4769de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb77a59c700 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a47679c0 0x5571a4769de0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97238ae0e0837b5 server_cookie=cb0aefd7c9e7fb8d in_seq=0 out_seq=0 2023-09-23T04:54:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.456+0000 7fb761ffb700 10 client.17337 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb761ffb700 1 -- 172.21.15.5:0/1709842578 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5571a4644b30 con 0x5571a47679c0 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb761ffb700 10 client.17337 handle_command_reply: tid=0 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 client.17337 shutdown 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 msgr2=0x5571a4709180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb774014a90 tx=0x7fb774005bf0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 shutdown_connections 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5571a47679c0 0x5571a4769de0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571a4703bd0 0x5571a4709180 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5571a46f8400 0x5571a4708700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 --2- 172.21.15.5:0/1709842578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571a46f8d20 0x5571a4708c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 >> 172.21.15.5:0/1709842578 conn(0x5571a466d260 msgr2=0x5571a46fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb761ffb700 0 client.17337 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 shutdown_connections 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.457+0000 7fb7946fab80 1 -- 172.21.15.5:0/1709842578 wait complete. 2023-09-23T04:54:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.458+0000 7fb7946fab80 20 client.17337 trim_cache size 0 max 16384 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:54:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:54:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:54:22.465 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:54:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.458+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 msgr2=0x7fb7640a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.458+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 0x7fb7640a3c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb774021000 tx=0x7fb77401c000 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.458+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 msgr2=0x7fb77c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.458+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00e9a0 tx=0x7fb76c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 shutdown_connections 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb7640a1790 0x7fb7640a3c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb77c19db50 0x7fb77c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77c103310 0x7fb77c1991b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 --2- 172.21.15.5:0/3936378439 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb77c108430 0x7fb77c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 >> 172.21.15.5:0/3936378439 conn(0x7fb77c0feb50 msgr2=0x7fb77c0ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 shutdown_connections 2023-09-23T04:54:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:22.459+0000 7fb780dff700 1 -- 172.21.15.5:0/3936378439 wait complete. 2023-09-23T04:54:22.482 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:54:22.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452135 4.88 MB/sec execute 298 sec latency 16.306 ms 2023-09-23T04:54:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:22 smithi005 ceph-mon[97638]: pgmap v250: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-09-23T04:54:23.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:22 smithi118 ceph-mon[134297]: pgmap v250: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-09-23T04:54:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:22 smithi069 ceph-mon[133931]: pgmap v250: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 135 op/s 2023-09-23T04:54:23.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452187 4.86 MB/sec execute 299 sec latency 948.267 ms 2023-09-23T04:54:24.954 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452187 4.84 MB/sec execute 300 sec latency 1948.319 ms 2023-09-23T04:54:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:24 smithi005 ceph-mon[97638]: pgmap v251: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-09-23T04:54:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:24 smithi118 ceph-mon[134297]: pgmap v251: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-09-23T04:54:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:24 smithi069 ceph-mon[133931]: pgmap v251: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2023-09-23T04:54:25.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452187 4.83 MB/sec execute 301 sec latency 2948.361 ms 2023-09-23T04:54:26.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452270 4.81 MB/sec execute 302 sec latency 3911.219 ms 2023-09-23T04:54:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:26 smithi005 ceph-mon[97638]: pgmap v252: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:54:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:26 smithi118 ceph-mon[134297]: pgmap v252: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:54:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:26 smithi069 ceph-mon[133931]: pgmap v252: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2023-09-23T04:54:27.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 454248 4.84 MB/sec execute 303 sec latency 21.102 ms 2023-09-23T04:54:28.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455297 4.83 MB/sec execute 304 sec latency 11.613 ms 2023-09-23T04:54:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:28 smithi005 ceph-mon[97638]: pgmap v253: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 208 op/s 2023-09-23T04:54:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:28 smithi118 ceph-mon[134297]: pgmap v253: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 208 op/s 2023-09-23T04:54:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:28 smithi069 ceph-mon[133931]: pgmap v253: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 208 op/s 2023-09-23T04:54:29.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455730 4.83 MB/sec execute 305 sec latency 763.793 ms 2023-09-23T04:54:30.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455730 4.81 MB/sec execute 306 sec latency 1763.836 ms 2023-09-23T04:54:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:31 smithi005 ceph-mon[97638]: pgmap v254: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-09-23T04:54:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:31 smithi118 ceph-mon[134297]: pgmap v254: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-09-23T04:54:31.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455864 4.80 MB/sec execute 307 sec latency 2703.312 ms 2023-09-23T04:54:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:31 smithi069 ceph-mon[133931]: pgmap v254: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-09-23T04:54:32.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 457846 4.82 MB/sec execute 308 sec latency 12.143 ms 2023-09-23T04:54:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:32 smithi005 ceph-mon[97638]: pgmap v255: 129 pgs: 129 active+clean; 303 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-09-23T04:54:33.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:32 smithi118 ceph-mon[134297]: pgmap v255: 129 pgs: 129 active+clean; 303 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-09-23T04:54:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:32 smithi069 ceph-mon[133931]: pgmap v255: 129 pgs: 129 active+clean; 303 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 177 op/s 2023-09-23T04:54:33.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 458533 4.84 MB/sec execute 309 sec latency 74.981 ms 2023-09-23T04:54:34.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:54:34.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:54:34.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:54:34.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 459491 4.87 MB/sec execute 310 sec latency 47.297 ms 2023-09-23T04:54:35.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:34 smithi069 ceph-mon[133931]: pgmap v256: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-09-23T04:54:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:54:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:54:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:54:35.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:34 smithi005 ceph-mon[97638]: pgmap v256: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-09-23T04:54:35.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:54:35.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:54:35.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:54:35.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:34 smithi118 ceph-mon[134297]: pgmap v256: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2023-09-23T04:54:35.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:54:35.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:54:35.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:54:35.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460030 4.86 MB/sec execute 311 sec latency 634.554 ms 2023-09-23T04:54:36.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460043 4.85 MB/sec execute 312 sec latency 1574.990 ms 2023-09-23T04:54:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:36 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:36 smithi069 ceph-mon[133931]: pgmap v257: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 138 op/s 2023-09-23T04:54:37.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:36 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:37.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:36 smithi005 ceph-mon[97638]: pgmap v257: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 138 op/s 2023-09-23T04:54:37.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:36 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:37.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:36 smithi118 ceph-mon[134297]: pgmap v257: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 138 op/s 2023-09-23T04:54:37.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460043 4.83 MB/sec execute 313 sec latency 1026.774 ms 2023-09-23T04:54:38.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:37 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:38.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:37 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:38.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:37 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:38.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460043 4.82 MB/sec execute 314 sec latency 2026.840 ms 2023-09-23T04:54:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:38 smithi069 ceph-mon[133931]: pgmap v258: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2023-09-23T04:54:39.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:38 smithi005 ceph-mon[97638]: pgmap v258: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2023-09-23T04:54:39.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:38 smithi118 ceph-mon[134297]: pgmap v258: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2023-09-23T04:54:39.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460043 4.80 MB/sec execute 315 sec latency 3026.906 ms 2023-09-23T04:54:40.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460184 4.79 MB/sec execute 316 sec latency 3169.193 ms 2023-09-23T04:54:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:41 smithi005 ceph-mon[97638]: pgmap v259: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 848 KiB/s wr, 103 op/s 2023-09-23T04:54:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:41 smithi118 ceph-mon[134297]: pgmap v259: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 848 KiB/s wr, 103 op/s 2023-09-23T04:54:41.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 460317 4.77 MB/sec execute 317 sec latency 1759.898 ms 2023-09-23T04:54:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:41 smithi069 ceph-mon[133931]: pgmap v259: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 848 KiB/s wr, 103 op/s 2023-09-23T04:54:42.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461709 4.79 MB/sec execute 318 sec latency 247.772 ms 2023-09-23T04:54:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:42 smithi005 ceph-mon[97638]: pgmap v260: 129 pgs: 129 active+clean; 323 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 129 op/s 2023-09-23T04:54:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:42 smithi118 ceph-mon[134297]: pgmap v260: 129 pgs: 129 active+clean; 323 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 129 op/s 2023-09-23T04:54:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:42 smithi069 ceph-mon[133931]: pgmap v260: 129 pgs: 129 active+clean; 323 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 129 op/s 2023-09-23T04:54:43.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461709 4.77 MB/sec execute 319 sec latency 1247.818 ms 2023-09-23T04:54:44.955 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461709 4.76 MB/sec execute 320 sec latency 2247.865 ms 2023-09-23T04:54:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:44 smithi005 ceph-mon[97638]: pgmap v261: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.0 MiB/s wr, 95 op/s 2023-09-23T04:54:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:44 smithi118 ceph-mon[134297]: pgmap v261: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.0 MiB/s wr, 95 op/s 2023-09-23T04:54:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:44 smithi069 ceph-mon[133931]: pgmap v261: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.0 MiB/s wr, 95 op/s 2023-09-23T04:54:45.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461722 4.74 MB/sec execute 321 sec latency 2616.486 ms 2023-09-23T04:54:46.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461722 4.73 MB/sec execute 322 sec latency 1614.092 ms 2023-09-23T04:54:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:46 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:46 smithi005 ceph-mon[97638]: pgmap v262: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 2.9 MiB/s wr, 90 op/s 2023-09-23T04:54:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:46 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:46 smithi118 ceph-mon[134297]: pgmap v262: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 2.9 MiB/s wr, 90 op/s 2023-09-23T04:54:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:46 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:46 smithi069 ceph-mon[133931]: pgmap v262: 129 pgs: 129 active+clean; 334 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 2.9 MiB/s wr, 90 op/s 2023-09-23T04:54:47.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461722 4.71 MB/sec execute 323 sec latency 2614.145 ms 2023-09-23T04:54:48.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461722 4.70 MB/sec execute 324 sec latency 3614.216 ms 2023-09-23T04:54:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:48 smithi005 ceph-mon[97638]: pgmap v263: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 108 op/s 2023-09-23T04:54:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:48 smithi118 ceph-mon[134297]: pgmap v263: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 108 op/s 2023-09-23T04:54:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:48 smithi069 ceph-mon[133931]: pgmap v263: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 108 op/s 2023-09-23T04:54:49.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461722 4.68 MB/sec execute 325 sec latency 4614.277 ms 2023-09-23T04:54:50.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 462730 4.68 MB/sec execute 326 sec latency 4734.691 ms 2023-09-23T04:54:51.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:51 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:51.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:51 smithi005 ceph-mon[97638]: pgmap v264: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.5 MiB/s wr, 55 op/s 2023-09-23T04:54:51.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:51 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:51.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:51 smithi118 ceph-mon[134297]: pgmap v264: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.5 MiB/s wr, 55 op/s 2023-09-23T04:54:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:51 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:54:51.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:51 smithi069 ceph-mon[133931]: pgmap v264: 129 pgs: 129 active+clean; 329 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.5 MiB/s wr, 55 op/s 2023-09-23T04:54:51.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 463746 4.69 MB/sec execute 327 sec latency 437.203 ms 2023-09-23T04:54:52.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 -- 172.21.15.5:0/304450087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281010d0 msgr2=0x7f6e281014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 --2- 172.21.15.5:0/304450087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281010d0 0x7f6e281014b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e24009ca0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 -- 172.21.15.5:0/304450087 shutdown_connections 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 --2- 172.21.15.5:0/304450087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281061f0 0x7f6e2810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 --2- 172.21.15.5:0/304450087 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281010d0 0x7f6e281014b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 --2- 172.21.15.5:0/304450087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e28101a80 0x7f6e28105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 -- 172.21.15.5:0/304450087 >> 172.21.15.5:0/304450087 conn(0x7f6e280fc990 msgr2=0x7f6e280fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 -- 172.21.15.5:0/304450087 shutdown_connections 2023-09-23T04:54:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.740+0000 7f6e307ff700 1 -- 172.21.15.5:0/304450087 wait complete. 2023-09-23T04:54:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.742+0000 7f6e307ff700 1 Processor -- start 2023-09-23T04:54:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.742+0000 7f6e307ff700 1 -- start start 2023-09-23T04:54:52.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.742+0000 7f6e307ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.742+0000 7f6e307ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 0x7f6e281974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e307ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2819b940 0x7f6e2819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e307ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819c3c0 con 0x7f6e2819b940 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e307ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819c530 con 0x7f6e281010d0 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e307ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819c6f0 con 0x7f6e281061f0 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56106/0 (socket says 172.21.15.5:56106) 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 -- 172.21.15.5:0/1173189576 learned_addr learned my addr 172.21.15.5:0/1173189576 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 0x7f6e281974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2819b940 0x7f6e2819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 -- 172.21.15.5:0/1173189576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 msgr2=0x7f6e281974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 0x7f6e281974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 -- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2819b940 msgr2=0x7f6e2819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2819b940 0x7f6e2819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2e59b700 1 -- 172.21.15.5:0/1173189576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240095f0 con 0x7f6e281010d0 2023-09-23T04:54:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.743+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 0x7f6e281974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:54:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005910 tx=0x7f6e24004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e1f7fe700 1 -- 172.21.15.5:0/1173189576 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401d070 con 0x7f6e281010d0 2023-09-23T04:54:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e28073050 con 0x7f6e281010d0 2023-09-23T04:54:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e1f7fe700 1 -- 172.21.15.5:0/1173189576 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24022470 con 0x7f6e281010d0 2023-09-23T04:54:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e1f7fe700 1 -- 172.21.15.5:0/1173189576 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24017650 con 0x7f6e281010d0 2023-09-23T04:54:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.744+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e280734f0 con 0x7f6e281010d0 2023-09-23T04:54:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.748+0000 7f6e1f7fe700 1 -- 172.21.15.5:0/1173189576 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f6e240225e0 con 0x7f6e281010d0 2023-09-23T04:54:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.750+0000 7f6e1f7fe700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 0x7f6e140a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.750+0000 7f6e1f7fe700 1 -- 172.21.15.5:0/1173189576 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6e240ce950 con 0x7f6e281010d0 2023-09-23T04:54:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 0x7f6e140a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 Processor -- start 2023-09-23T04:54:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 -- start start 2023-09-23T04:54:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223c977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 0x7f6e140a3bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e281985b0 tx=0x7f6e18009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223c00460 con 0x562223c97400 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223bf1c20 con 0x562223c97d20 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.751+0000 7f6e44156b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223bef540 con 0x562223ca2bd0 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39842/0 (socket says 172.21.15.5:39842) 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 -- 172.21.15.5:0/225909720 learned_addr learned my addr 172.21.15.5:0/225909720 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223c977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 -- 172.21.15.5:0/225909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 msgr2=0x562223ca2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 -- 172.21.15.5:0/225909720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 msgr2=0x562223c977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223c977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 -- 172.21.15.5:0/225909720 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562223c4d8d0 con 0x562223ca2bd0 2023-09-23T04:54:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.752+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e20011680 tx=0x7f6e20009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46554d75fb740f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e1cff9700 1 -- 172.21.15.5:0/225909720 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000e040 con 0x562223ca2bd0 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e1cff9700 1 -- 172.21.15.5:0/225909720 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2000f040 con 0x562223ca2bd0 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e1cff9700 1 -- 172.21.15.5:0/225909720 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e200048c0 con 0x562223ca2bd0 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 -- 172.21.15.5:0/225909720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 msgr2=0x562223ca4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e20011680 tx=0x7f6e20009920 comp rx=0 tx=0).stop 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 -- 172.21.15.5:0/225909720 shutdown_connections 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca4fc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223c977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 --2- 172.21.15.5:0/225909720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.753+0000 7f6e44156b80 1 -- 172.21.15.5:0/225909720 >> 172.21.15.5:0/225909720 conn(0x562223c0c260 msgr2=0x562223c93790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.754+0000 7f6e44156b80 1 -- 172.21.15.5:0/225909720 shutdown_connections 2023-09-23T04:54:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.754+0000 7f6e44156b80 1 -- 172.21.15.5:0/225909720 wait complete. 2023-09-23T04:54:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.754+0000 7f6e44156b80 1 -- start start 2023-09-23T04:54:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.754+0000 7f6e44156b80 1 Processor -- start 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97400 0x562223c9f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223ca0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223beae30 con 0x562223ca2bd0 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223b87710 con 0x562223c97400 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223b873b0 con 0x562223c97d20 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e44156b80 10 client.0 _ll_register_callbacks cb 0x5622239e4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e2dd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e2dd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39856/0 (socket says 172.21.15.5:39856) 2023-09-23T04:54:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/3174737792 learned_addr learned my addr 172.21.15.5:0/3174737792 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223ca0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.755+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97400 0x562223c9f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/3174737792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97400 msgr2=0x562223c9f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97400 0x562223c9f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 msgr2=0x562223ca0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223ca0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/3174737792 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e20011330 con 0x562223c97d20 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e180037f0 tx=0x7f6e180047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3897b35bfd05a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1801f810 con 0x562223c97d20 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562223c8b250 con 0x562223c97d20 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.756+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6e1801f970 con 0x562223c97d20 2023-09-23T04:54:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.757+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1801fb30 con 0x562223c97d20 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.757+0000 7f6e44156b80 10 client.27397 fetch_fsmap learned FSMap version 44 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.757+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562223a15de0 con 0x562223c97d20 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.757+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562223c8b250 con 0x562223c97d20 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.757+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 44) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6e1801e400 con 0x562223c97d20 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 10 client.27397 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 10 client.27397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 20 client.27397 populate_metadata read hostname 'smithi005' 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223cb2810 0x562223cb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 4 client.27397 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562223be3e10 con 0x562223cb2810 2023-09-23T04:54:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.758+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223cb2810 0x562223cb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.759+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223cb2810 0x562223cb4c30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6fbebc06e7903be server_cookie=3578879fa23e8cc0 in_seq=0 out_seq=0 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.759+0000 7f6e11ffb700 10 client.27397 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.759+0000 7f6e11ffb700 1 -- 172.21.15.5:0/3174737792 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562223be3e10 con 0x562223cb2810 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.759+0000 7f6e11ffb700 10 client.27397 handle_command_reply: tid=0 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.759+0000 7f6e44156b80 1 client.27397 shutdown 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 msgr2=0x562223c9ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e180037f0 tx=0x7f6e180047c0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 shutdown_connections 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223cb2810 0x562223cb4c30 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223ca0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97400 0x562223c9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 --2- 172.21.15.5:0/3174737792 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97d20 0x562223c9ff10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 >> 172.21.15.5:0/3174737792 conn(0x562223c0c260 msgr2=0x562223c93320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e11ffb700 0 client.27397 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 shutdown_connections 2023-09-23T04:54:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 1 -- 172.21.15.5:0/3174737792 wait complete. 2023-09-23T04:54:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.760+0000 7f6e44156b80 20 client.27397 trim_cache size 0 max 16384 2023-09-23T04:54:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 Processor -- start 2023-09-23T04:54:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 -- start start 2023-09-23T04:54:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 0x562223c9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223c9cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223a6b920 con 0x562223ca2bd0 2023-09-23T04:54:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223c9a800 con 0x562223c97d20 2023-09-23T04:54:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.785+0000 7f6e44156b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223ca9c50 con 0x562223c97400 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2ed9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223c9cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:56132/0 (socket says 172.21.15.5:56132) 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/4170611716 learned_addr learned my addr 172.21.15.5:0/4170611716 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 0x562223c9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/4170611716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 msgr2=0x562223c9a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 0x562223c9a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/4170611716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 msgr2=0x562223c9cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223c9cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/4170611716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562223c8f480 con 0x562223c97d20 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 0x562223c9a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.786+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223c9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.787+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e180037c0 tx=0x7f6e18009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fee8f2dce3c5ce4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.787+0000 7f6e1d7fa700 1 -- 172.21.15.5:0/4170611716 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1801e400 con 0x562223c97d20 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.787+0000 7f6e1d7fa700 1 -- 172.21.15.5:0/4170611716 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e1801e560 con 0x562223c97d20 2023-09-23T04:54:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.787+0000 7f6e1d7fa700 1 -- 172.21.15.5:0/4170611716 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1801fc40 con 0x562223c97d20 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 -- 172.21.15.5:0/4170611716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 msgr2=0x562223ca9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e180037c0 tx=0x7f6e18009e90 comp rx=0 tx=0).stop 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 -- 172.21.15.5:0/4170611716 shutdown_connections 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223ca2bd0 0x562223c9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223c97400 0x562223c9a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 --2- 172.21.15.5:0/4170611716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca9710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 -- 172.21.15.5:0/4170611716 >> 172.21.15.5:0/4170611716 conn(0x562223c0c260 msgr2=0x562223c9acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 -- 172.21.15.5:0/4170611716 shutdown_connections 2023-09-23T04:54:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.788+0000 7f6e44156b80 1 -- 172.21.15.5:0/4170611716 wait complete. 2023-09-23T04:54:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 -- start start 2023-09-23T04:54:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 Processor -- start 2023-09-23T04:54:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223ca8180 con 0x562223c97400 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223ca82f0 con 0x562223c97d20 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.789+0000 7f6e44156b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x562223ca84b0 con 0x562223ca2bd0 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e44156b80 10 client.0 _ll_register_callbacks cb 0x7f6e180045e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2e59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2e59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32808/0 (socket says 172.21.15.5:32808) 2023-09-23T04:54:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2e59b700 1 -- 172.21.15.5:0/181420681 learned_addr learned my addr 172.21.15.5:0/181420681 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2ed9c700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/181420681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 msgr2=0x562223ca7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 msgr2=0x562223ca9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2dd9a700 1 -- 172.21.15.5:0/181420681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562223c4d8d0 con 0x562223c97d20 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.790+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.791+0000 7f6e2dd9a700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca7700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x562223c94ca0 tx=0x7f6e1801ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3694c619e8614ccd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.791+0000 7f6e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.791+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1801f610 con 0x562223c97d20 2023-09-23T04:54:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.791+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e2000f040 con 0x562223c97d20 2023-09-23T04:54:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.791+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6e1801f770 con 0x562223c97d20 2023-09-23T04:54:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1801f990 con 0x562223c97d20 2023-09-23T04:54:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 10 client.27350 fetch_fsmap learned FSMap version 44 2023-09-23T04:54:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f6e1802d040 con 0x562223c97d20 2023-09-23T04:54:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2000f040 con 0x562223c97d20 2023-09-23T04:54:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 44) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6e1803c070 con 0x562223c97d20 2023-09-23T04:54:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 10 client.27350 fetch_fsmap finished waiting for FSMap version 44 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 10 client.27350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 20 client.27350 populate_metadata read hostname 'smithi005' 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223d07680 0x562223d09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 4 client.27350 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.795+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562223be3b30 con 0x562223d07680 2023-09-23T04:54:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.796+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223d07680 0x562223d09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.796+0000 7f6e11ffb700 10 client.27350 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.796+0000 7f6e2e59b700 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223d07680 0x562223d09a70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c924c62df84a0f server_cookie=5a6d6d48a28f5575 in_seq=0 out_seq=0 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e11ffb700 1 -- 172.21.15.5:0/181420681 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562223be3b30 con 0x562223d07680 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e11ffb700 10 client.27350 handle_command_reply: tid=0 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 client.27350 shutdown 2023-09-23T04:54:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 msgr2=0x562223ca7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca7700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x562223c94ca0 tx=0x7f6e1801ebb0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 shutdown_connections 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x562223d07680 0x562223d09a70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x562223ca2bd0 0x562223ca7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562223c97400 0x562223ca9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e11ffb700 0 client.27350 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 --2- 172.21.15.5:0/181420681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562223c97d20 0x562223ca7700 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.797+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 >> 172.21.15.5:0/181420681 conn(0x562223c0c260 msgr2=0x562223c9f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.798+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 shutdown_connections 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.798+0000 7f6e44156b80 1 -- 172.21.15.5:0/181420681 wait complete. 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.798+0000 7f6e44156b80 20 client.27350 trim_cache size 0 max 16384 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:54:52.799 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:54:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 msgr2=0x7f6e140a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 0x7f6e140a3bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e281985b0 tx=0x7f6e18009260 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 msgr2=0x7f6e28196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005910 tx=0x7f6e24004730 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 shutdown_connections 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6e140a1710 0x7f6e140a3bd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2819b940 0x7f6e2819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e281010d0 0x7f6e28196fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 --2- 172.21.15.5:0/1173189576 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6e281061f0 0x7f6e281974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 >> 172.21.15.5:0/1173189576 conn(0x7f6e280fc990 msgr2=0x7f6e28104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 shutdown_connections 2023-09-23T04:54:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:54:52.799+0000 7f6e307ff700 1 -- 172.21.15.5:0/1173189576 wait complete. 2023-09-23T04:54:52.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:52 smithi005 ceph-mon[97638]: pgmap v265: 129 pgs: 129 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 87 op/s 2023-09-23T04:54:52.820 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:54:52.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 465688 4.71 MB/sec execute 328 sec latency 12.304 ms 2023-09-23T04:54:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:52 smithi118 ceph-mon[134297]: pgmap v265: 129 pgs: 129 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 87 op/s 2023-09-23T04:54:53.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:52 smithi069 ceph-mon[133931]: pgmap v265: 129 pgs: 129 active+clean; 331 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 2.7 MiB/s wr, 87 op/s 2023-09-23T04:54:53.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 467005 4.71 MB/sec execute 329 sec latency 15.304 ms 2023-09-23T04:54:54.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 467138 4.70 MB/sec execute 330 sec latency 930.448 ms 2023-09-23T04:54:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:54 smithi005 ceph-mon[97638]: pgmap v266: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.5 MiB/s wr, 77 op/s 2023-09-23T04:54:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:54 smithi118 ceph-mon[134297]: pgmap v266: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.5 MiB/s wr, 77 op/s 2023-09-23T04:54:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:54 smithi069 ceph-mon[133931]: pgmap v266: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.5 MiB/s wr, 77 op/s 2023-09-23T04:54:55.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 467138 4.69 MB/sec execute 331 sec latency 1930.487 ms 2023-09-23T04:54:56.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 467247 4.67 MB/sec execute 332 sec latency 2875.821 ms 2023-09-23T04:54:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:56 smithi005 ceph-mon[97638]: pgmap v267: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 430 KiB/s wr, 67 op/s 2023-09-23T04:54:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:56 smithi118 ceph-mon[134297]: pgmap v267: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 430 KiB/s wr, 67 op/s 2023-09-23T04:54:57.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:56 smithi069 ceph-mon[133931]: pgmap v267: 129 pgs: 129 active+clean; 333 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 430 KiB/s wr, 67 op/s 2023-09-23T04:54:57.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 469231 4.70 MB/sec execute 333 sec latency 17.223 ms 2023-09-23T04:54:58.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 470486 4.70 MB/sec execute 334 sec latency 20.198 ms 2023-09-23T04:54:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:54:58 smithi005 ceph-mon[97638]: pgmap v268: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 798 KiB/s wr, 111 op/s 2023-09-23T04:54:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:54:58 smithi118 ceph-mon[134297]: pgmap v268: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 798 KiB/s wr, 111 op/s 2023-09-23T04:54:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:54:58 smithi069 ceph-mon[133931]: pgmap v268: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 798 KiB/s wr, 111 op/s 2023-09-23T04:54:59.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 470685 4.69 MB/sec execute 335 sec latency 882.709 ms 2023-09-23T04:55:00.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 470685 4.67 MB/sec execute 336 sec latency 1882.786 ms 2023-09-23T04:55:01.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:01 smithi005 ceph-mon[97638]: pgmap v269: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 773 KiB/s wr, 92 op/s 2023-09-23T04:55:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:01 smithi118 ceph-mon[134297]: pgmap v269: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 773 KiB/s wr, 92 op/s 2023-09-23T04:55:01.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 470769 4.66 MB/sec execute 337 sec latency 2822.300 ms 2023-09-23T04:55:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:01 smithi069 ceph-mon[133931]: pgmap v269: 129 pgs: 129 active+clean; 335 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 773 KiB/s wr, 92 op/s 2023-09-23T04:55:02.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 471404 4.65 MB/sec execute 338 sec latency 61.309 ms 2023-09-23T04:55:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:02 smithi005 ceph-mon[97638]: pgmap v270: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1000 KiB/s wr, 127 op/s 2023-09-23T04:55:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:02 smithi118 ceph-mon[134297]: pgmap v270: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1000 KiB/s wr, 127 op/s 2023-09-23T04:55:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:02 smithi069 ceph-mon[133931]: pgmap v270: 129 pgs: 129 active+clean; 337 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1000 KiB/s wr, 127 op/s 2023-09-23T04:55:03.956 INFO:tasks.workunit.client.0.smithi005.stdout: 1 472978 4.67 MB/sec execute 339 sec latency 11.655 ms 2023-09-23T04:55:04.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 474231 4.67 MB/sec execute 340 sec latency 185.478 ms 2023-09-23T04:55:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:04 smithi005 ceph-mon[97638]: pgmap v271: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 880 KiB/s wr, 123 op/s 2023-09-23T04:55:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:04 smithi118 ceph-mon[134297]: pgmap v271: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 880 KiB/s wr, 123 op/s 2023-09-23T04:55:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:04 smithi069 ceph-mon[133931]: pgmap v271: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 880 KiB/s wr, 123 op/s 2023-09-23T04:55:05.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 475135 4.67 MB/sec execute 341 sec latency 705.311 ms 2023-09-23T04:55:06.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476644 4.69 MB/sec execute 342 sec latency 13.786 ms 2023-09-23T04:55:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:06 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:06 smithi005 ceph-mon[97638]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T04:55:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:06 smithi005 ceph-mon[97638]: pgmap v273: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 853 KiB/s wr, 129 op/s 2023-09-23T04:55:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:06 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:06 smithi118 ceph-mon[134297]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T04:55:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:06 smithi118 ceph-mon[134297]: pgmap v273: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 853 KiB/s wr, 129 op/s 2023-09-23T04:55:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:06 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:06 smithi069 ceph-mon[133931]: osdmap e97: 12 total, 12 up, 12 in 2023-09-23T04:55:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:06 smithi069 ceph-mon[133931]: pgmap v273: 129 pgs: 129 active+clean; 338 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 853 KiB/s wr, 129 op/s 2023-09-23T04:55:07.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.67 MB/sec execute 343 sec latency 949.336 ms 2023-09-23T04:55:08.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.66 MB/sec execute 344 sec latency 1949.402 ms 2023-09-23T04:55:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:08 smithi005 ceph-mon[97638]: pgmap v274: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-23T04:55:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:08 smithi118 ceph-mon[134297]: pgmap v274: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-23T04:55:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:08 smithi069 ceph-mon[133931]: pgmap v274: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2023-09-23T04:55:09.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.65 MB/sec execute 345 sec latency 2949.449 ms 2023-09-23T04:55:10.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.63 MB/sec execute 346 sec latency 3949.498 ms 2023-09-23T04:55:11.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:11 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:11.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:11 smithi005 ceph-mon[97638]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T04:55:11.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:11 smithi005 ceph-mon[97638]: pgmap v276: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1011 KiB/s wr, 139 op/s 2023-09-23T04:55:11.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:11 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:11.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:11 smithi118 ceph-mon[134297]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T04:55:11.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:11 smithi118 ceph-mon[134297]: pgmap v276: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1011 KiB/s wr, 139 op/s 2023-09-23T04:55:11.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:11 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:11.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:11 smithi069 ceph-mon[133931]: osdmap e98: 12 total, 12 up, 12 in 2023-09-23T04:55:11.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:11 smithi069 ceph-mon[133931]: pgmap v276: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1011 KiB/s wr, 139 op/s 2023-09-23T04:55:11.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.62 MB/sec execute 347 sec latency 4949.549 ms 2023-09-23T04:55:12.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.61 MB/sec execute 348 sec latency 5949.606 ms 2023-09-23T04:55:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:12 smithi005 ceph-mon[97638]: pgmap v277: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 898 KiB/s wr, 103 op/s 2023-09-23T04:55:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:12 smithi118 ceph-mon[134297]: pgmap v277: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 898 KiB/s wr, 103 op/s 2023-09-23T04:55:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:12 smithi069 ceph-mon[133931]: pgmap v277: 129 pgs: 129 active+clean; 340 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 898 KiB/s wr, 103 op/s 2023-09-23T04:55:13.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.59 MB/sec execute 349 sec latency 6949.670 ms 2023-09-23T04:55:14.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476685 4.58 MB/sec execute 350 sec latency 7949.727 ms 2023-09-23T04:55:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:14 smithi005 ceph-mon[97638]: pgmap v278: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 854 KiB/s wr, 98 op/s 2023-09-23T04:55:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:14 smithi118 ceph-mon[134297]: pgmap v278: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 854 KiB/s wr, 98 op/s 2023-09-23T04:55:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:14 smithi069 ceph-mon[133931]: pgmap v278: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 854 KiB/s wr, 98 op/s 2023-09-23T04:55:15.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 477618 4.58 MB/sec execute 351 sec latency 8340.991 ms 2023-09-23T04:55:16.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 477915 4.57 MB/sec execute 352 sec latency 842.052 ms 2023-09-23T04:55:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:16 smithi005 ceph-mon[97638]: pgmap v279: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-09-23T04:55:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:16 smithi118 ceph-mon[134297]: pgmap v279: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-09-23T04:55:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:16 smithi069 ceph-mon[133931]: pgmap v279: 129 pgs: 129 active+clean; 341 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 762 KiB/s wr, 87 op/s 2023-09-23T04:55:17.958 INFO:tasks.workunit.client.0.smithi005.stdout: 1 479918 4.59 MB/sec execute 353 sec latency 13.985 ms 2023-09-23T04:55:18.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481161 4.60 MB/sec execute 354 sec latency 18.401 ms 2023-09-23T04:55:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:18 smithi005 ceph-mon[97638]: pgmap v280: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 639 KiB/s wr, 59 op/s 2023-09-23T04:55:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:18 smithi118 ceph-mon[134297]: pgmap v280: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 639 KiB/s wr, 59 op/s 2023-09-23T04:55:19.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:18 smithi069 ceph-mon[133931]: pgmap v280: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 639 KiB/s wr, 59 op/s 2023-09-23T04:55:19.958 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481360 4.58 MB/sec execute 355 sec latency 806.195 ms 2023-09-23T04:55:20.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481360 4.57 MB/sec execute 356 sec latency 1806.239 ms 2023-09-23T04:55:21.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:21 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:21.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:21 smithi005 ceph-mon[97638]: pgmap v281: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 632 KiB/s wr, 59 op/s 2023-09-23T04:55:21.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:21 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:21.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:21 smithi118 ceph-mon[134297]: pgmap v281: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 632 KiB/s wr, 59 op/s 2023-09-23T04:55:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:21 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:55:21.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:21 smithi069 ceph-mon[133931]: pgmap v281: 129 pgs: 129 active+clean; 346 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 632 KiB/s wr, 59 op/s 2023-09-23T04:55:21.957 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481494 4.56 MB/sec execute 357 sec latency 2746.940 ms 2023-09-23T04:55:22.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:55:22.958 INFO:tasks.workunit.client.0.smithi005.stdout: 1 483464 4.58 MB/sec execute 358 sec latency 13.170 ms 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.089+0000 7fee87fff700 1 -- 172.21.15.5:0/2180663210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee80105100 msgr2=0x7fee8010cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.089+0000 7fee87fff700 1 --2- 172.21.15.5:0/2180663210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee80105100 0x7fee8010cc80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee7800d210 tx=0x7fee7800d520 comp rx=0 tx=0).stop 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 -- 172.21.15.5:0/2180663210 shutdown_connections 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 --2- 172.21.15.5:0/2180663210 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee80105100 0x7fee8010cc80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 --2- 172.21.15.5:0/2180663210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee800fffe0 0x7fee801003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 --2- 172.21.15.5:0/2180663210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee80100990 0x7fee801049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 -- 172.21.15.5:0/2180663210 >> 172.21.15.5:0/2180663210 conn(0x7fee80076f30 msgr2=0x7fee80077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 -- 172.21.15.5:0/2180663210 shutdown_connections 2023-09-23T04:55:23.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.090+0000 7fee87fff700 1 -- 172.21.15.5:0/2180663210 wait complete. 2023-09-23T04:55:23.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.091+0000 7fee87fff700 1 Processor -- start 2023-09-23T04:55:23.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 -- start start 2023-09-23T04:55:23.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee80105100 0x7fee80199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 0x7fee8019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8019e700 con 0x7fee8019dc80 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8019e870 con 0x7fee80105100 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.092+0000 7fee87fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8019ea30 con 0x7fee800fffe0 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:53884/0 (socket says 172.21.15.5:53884) 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 -- 172.21.15.5:0/3892165719 learned_addr learned my addr 172.21.15.5:0/3892165719 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee8659c700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 0x7fee8019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee8559a700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee80105100 0x7fee80199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 -- 172.21.15.5:0/3892165719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee80105100 msgr2=0x7fee80199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee80105100 0x7fee80199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 -- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 msgr2=0x7fee8019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 0x7fee8019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 -- 172.21.15.5:0/3892165719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0095f0 con 0x7fee800fffe0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee8659c700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 0x7fee8019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.093+0000 7fee85d9b700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00ec10 tx=0x7fee7c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.094+0000 7fee76ffd700 1 -- 172.21.15.5:0/3892165719 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00cd40 con 0x7fee800fffe0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.094+0000 7fee76ffd700 1 -- 172.21.15.5:0/3892165719 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee7c0044e0 con 0x7fee800fffe0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.094+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7800be50 con 0x7fee800fffe0 2023-09-23T04:55:23.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.094+0000 7fee76ffd700 1 -- 172.21.15.5:0/3892165719 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c010690 con 0x7fee800fffe0 2023-09-23T04:55:23.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.094+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee801a6290 con 0x7fee800fffe0 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.099+0000 7fee9b99db80 1 Processor -- start 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.099+0000 7fee76ffd700 1 -- 172.21.15.5:0/3892165719 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fee7c003710 con 0x7fee800fffe0 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.099+0000 7fee9b99db80 1 -- start start 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c7312460 con 0x5566c73a9400 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c7303c20 con 0x5566c73a9d20 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee9b99db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c7301540 con 0x5566c73b4bd0 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee85d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee8659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee8659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36714/0 (socket says 172.21.15.5:36714) 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee8659c700 1 -- 172.21.15.5:0/3569397725 learned_addr learned my addr 172.21.15.5:0/3569397725 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.100+0000 7fee8559a700 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8659c700 1 -- 172.21.15.5:0/3569397725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 msgr2=0x5566c73b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8659c700 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8659c700 1 -- 172.21.15.5:0/3569397725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 msgr2=0x5566c73b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8659c700 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8659c700 1 -- 172.21.15.5:0/3569397725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566c735f8d0 con 0x5566c73a9400 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.101+0000 7fee8559a700 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee8659c700 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fee78009730 tx=0x7fee78009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86f237ccf4c5592 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee76ffd700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 0x7fee6c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee6bfff700 1 -- 172.21.15.5:0/3569397725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78010040 con 0x5566c73a9400 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee6bfff700 1 -- 172.21.15.5:0/3569397725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee78009980 con 0x5566c73a9400 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee6bfff700 1 -- 172.21.15.5:0/3569397725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800b5a0 con 0x5566c73a9400 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee76ffd700 1 -- 172.21.15.5:0/3892165719 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fee7c014070 con 0x7fee800fffe0 2023-09-23T04:55:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.102+0000 7fee85d9b700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 0x7fee6c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 -- 172.21.15.5:0/3569397725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 msgr2=0x5566c73a97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fee78009730 tx=0x7fee78009760 comp rx=0 tx=0).stop 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 -- 172.21.15.5:0/3569397725 shutdown_connections 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73a97e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3569397725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 -- 172.21.15.5:0/3569397725 >> 172.21.15.5:0/3569397725 conn(0x5566c731e260 msgr2=0x5566c73a5790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 -- 172.21.15.5:0/3569397725 shutdown_connections 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee85d9b700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 0x7fee6c0a3c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cb020 tx=0x7fee7c0d15a0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.103+0000 7fee9b99db80 1 -- 172.21.15.5:0/3569397725 wait complete. 2023-09-23T04:55:23.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.104+0000 7fee9b99db80 1 -- start start 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.104+0000 7fee9b99db80 1 Processor -- start 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c72fce30 con 0x5566c73a9400 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c7299710 con 0x5566c73a9d20 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c72993b0 con 0x5566c73b4bd0 2023-09-23T04:55:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee9b99db80 10 client.0 _ll_register_callbacks cb 0x5566c70f6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee8559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee8659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee8559a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:53600/0 (socket says 172.21.15.5:53600) 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee8559a700 1 -- 172.21.15.5:0/3809695364 learned_addr learned my addr 172.21.15.5:0/3809695364 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.105+0000 7fee8659c700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 -- 172.21.15.5:0/3809695364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 msgr2=0x5566c73b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 -- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 msgr2=0x5566c73b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 -- 172.21.15.5:0/3809695364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee700098e0 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8659c700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee8559a700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee70011970 tx=0x7fee70011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aef7ac6d50dafce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee697fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70016070 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566c739d250 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.106+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fee7000faa0 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee7000fc60 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 10 client.27362 fetch_fsmap learned FSMap version 47 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fee8010d4f0 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566c739d250 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fee70005470 con 0x5566c73a9d20 2023-09-23T04:55:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 10 client.27362 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 20 client.27362 populate_metadata read hostname 'smithi005' 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c73c4750 0x5566c73c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 4 client.27362 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.107+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566c72f5e10 con 0x5566c73c4750 2023-09-23T04:55:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.108+0000 7fee8659c700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c73c4750 0x5566c73c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.108+0000 7fee697fa700 10 client.27362 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.108+0000 7fee8659c700 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c73c4750 0x5566c73c6b70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126d9a5369a7ab31 server_cookie=3120dd1904a469eb in_seq=0 out_seq=0 2023-09-23T04:55:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.109+0000 7fee697fa700 1 -- 172.21.15.5:0/3809695364 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5566c72f5e10 con 0x5566c73c4750 2023-09-23T04:55:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.109+0000 7fee697fa700 10 client.27362 handle_command_reply: tid=0 2023-09-23T04:55:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.109+0000 7fee9b99db80 1 client.27362 shutdown 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 msgr2=0x5566c73b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee70011970 tx=0x7fee70011c80 comp rx=0 tx=0).stop 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 shutdown_connections 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c73c4750 0x5566c73c6b70 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73b4bd0 0x5566c73b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9400 0x5566c73b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 --2- 172.21.15.5:0/3809695364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73a9d20 0x5566c73b1860 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee697fa700 0 client.27362 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 >> 172.21.15.5:0/3809695364 conn(0x5566c731e260 msgr2=0x5566c73a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 shutdown_connections 2023-09-23T04:55:23.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 1 -- 172.21.15.5:0/3809695364 wait complete. 2023-09-23T04:55:23.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.110+0000 7fee9b99db80 20 client.27362 trim_cache size 0 max 16384 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.133+0000 7fee9b99db80 1 Processor -- start 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.133+0000 7fee9b99db80 1 -- start start 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.133+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee9b99db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c717d920 con 0x5566c73a9d20 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee9b99db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c73a1d60 con 0x5566c73b4bd0 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee9b99db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c73a4580 con 0x5566c73a9400 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:53918/0 (socket says 172.21.15.5:53918) 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 -- 172.21.15.5:0/1912246650 learned_addr learned my addr 172.21.15.5:0/1912246650 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8559a700 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 -- 172.21.15.5:0/1912246650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 msgr2=0x5566c73aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 -- 172.21.15.5:0/1912246650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 msgr2=0x5566c73b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.134+0000 7fee8659c700 1 -- 172.21.15.5:0/1912246650 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566c7127de0 con 0x5566c73a9400 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.135+0000 7fee8659c700 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee7800f010 tx=0x7fee7800bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0d100274f93ddb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.135+0000 7fee74ff9700 1 -- 172.21.15.5:0/1912246650 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78010040 con 0x5566c73a9400 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.135+0000 7fee74ff9700 1 -- 172.21.15.5:0/1912246650 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee7802d070 con 0x5566c73a9400 2023-09-23T04:55:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.135+0000 7fee74ff9700 1 -- 172.21.15.5:0/1912246650 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7802f820 con 0x5566c73a9400 2023-09-23T04:55:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 -- 172.21.15.5:0/1912246650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 msgr2=0x5566c73bb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee7800f010 tx=0x7fee7800bcd0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 -- 172.21.15.5:0/1912246650 shutdown_connections 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb6c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1912246650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 -- 172.21.15.5:0/1912246650 >> 172.21.15.5:0/1912246650 conn(0x5566c731e260 msgr2=0x5566c73c0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 -- 172.21.15.5:0/1912246650 shutdown_connections 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.136+0000 7fee9b99db80 1 -- 172.21.15.5:0/1912246650 wait complete. 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.137+0000 7fee9b99db80 1 -- start start 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.137+0000 7fee9b99db80 1 Processor -- start 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c73b9ba0 con 0x5566c73a9d20 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c73b9d10 con 0x5566c73b4bd0 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566c73b9ed0 con 0x5566c73a9400 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee9b99db80 10 client.0 _ll_register_callbacks cb 0x7fee70011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee8659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee8659c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:53924/0 (socket says 172.21.15.5:53924) 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee8659c700 1 -- 172.21.15.5:0/1193981591 learned_addr learned my addr 172.21.15.5:0/1193981591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee8559a700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.138+0000 7fee8659c700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 -- 172.21.15.5:0/1193981591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 msgr2=0x5566c73b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 -- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 msgr2=0x5566c73bb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73bb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 -- 172.21.15.5:0/1193981591 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566c735f8d0 con 0x5566c73a9400 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee8659c700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fee78003c90 tx=0x7fee78028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bf5df9cc6b72a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee697fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78010040 con 0x5566c73a9400 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee78009980 con 0x5566c73a9400 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.139+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fee7802f2e0 con 0x5566c73a9400 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee7802f4e0 con 0x5566c73a9400 2023-09-23T04:55:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 10 client.27418 fetch_fsmap learned FSMap version 47 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5566c7110c60 con 0x5566c73a9400 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee78009980 con 0x5566c73a9400 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fee7801b070 con 0x5566c73a9400 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 10 client.27418 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 10 client.27418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 20 client.27418 populate_metadata read hostname 'smithi005' 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c7419190 0x5566c741b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 4 client.27418 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.140+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566c72f5b30 con 0x5566c7419190 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.141+0000 7fee8559a700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c7419190 0x5566c741b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.142+0000 7fee8559a700 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c7419190 0x5566c741b5b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7493acab4496e0 server_cookie=ec1e082a1edf9e79 in_seq=0 out_seq=0 2023-09-23T04:55:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.142+0000 7fee697fa700 10 client.27418 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:23.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.142+0000 7fee697fa700 1 -- 172.21.15.5:0/1193981591 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5566c72f5b30 con 0x5566c7419190 2023-09-23T04:55:23.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.142+0000 7fee697fa700 10 client.27418 handle_command_reply: tid=0 2023-09-23T04:55:23.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.142+0000 7fee9b99db80 1 client.27418 shutdown 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 msgr2=0x5566c73bb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fee78003c90 tx=0x7fee78028040 comp rx=0 tx=0).stop 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 shutdown_connections 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5566c7419190 0x5566c741b5b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5566c73b4bd0 0x5566c73b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5566c73a9400 0x5566c73bb0d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 --2- 172.21.15.5:0/1193981591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566c73a9d20 0x5566c73bb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee697fa700 0 client.27418 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 >> 172.21.15.5:0/1193981591 conn(0x5566c731e260 msgr2=0x5566c73a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 shutdown_connections 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 1 -- 172.21.15.5:0/1193981591 wait complete. 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.143+0000 7fee9b99db80 20 client.27418 trim_cache size 0 max 16384 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:55:23.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 msgr2=0x7fee6c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 0x7fee6c0a3c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0cb020 tx=0x7fee7c0d15a0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 msgr2=0x7fee801992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00ec10 tx=0x7fee7c00ef20 comp rx=0 tx=0).stop 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 shutdown_connections 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fee6c0a1790 0x7fee6c0a3c50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee8019dc80 0x7fee8019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fee800fffe0 0x7fee801992e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 --2- 172.21.15.5:0/3892165719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee80105100 0x7fee80199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 >> 172.21.15.5:0/3892165719 conn(0x7fee80076f30 msgr2=0x7fee800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 shutdown_connections 2023-09-23T04:55:23.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:23.144+0000 7fee87fff700 1 -- 172.21.15.5:0/3892165719 wait complete. 2023-09-23T04:55:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:22 smithi005 ceph-mon[97638]: pgmap v282: 129 pgs: 129 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 770 KiB/s wr, 81 op/s 2023-09-23T04:55:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:22 smithi118 ceph-mon[134297]: pgmap v282: 129 pgs: 129 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 770 KiB/s wr, 81 op/s 2023-09-23T04:55:23.165 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:55:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:22 smithi069 ceph-mon[133931]: pgmap v282: 129 pgs: 129 active+clean; 348 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 770 KiB/s wr, 81 op/s 2023-09-23T04:55:23.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 484666 4.59 MB/sec execute 359 sec latency 39.637 ms 2023-09-23T04:55:24.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 484911 4.57 MB/sec execute 360 sec latency 716.041 ms 2023-09-23T04:55:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:24 smithi005 ceph-mon[97638]: pgmap v283: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-09-23T04:55:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:24 smithi118 ceph-mon[134297]: pgmap v283: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-09-23T04:55:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:24 smithi069 ceph-mon[133931]: pgmap v283: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 805 KiB/s wr, 84 op/s 2023-09-23T04:55:25.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 484911 4.56 MB/sec execute 361 sec latency 1716.108 ms 2023-09-23T04:55:26.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 485045 4.55 MB/sec execute 362 sec latency 2652.278 ms 2023-09-23T04:55:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:26 smithi005 ceph-mon[97638]: pgmap v284: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 769 KiB/s wr, 80 op/s 2023-09-23T04:55:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:26 smithi118 ceph-mon[134297]: pgmap v284: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 769 KiB/s wr, 80 op/s 2023-09-23T04:55:27.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:26 smithi069 ceph-mon[133931]: pgmap v284: 129 pgs: 129 active+clean; 349 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 769 KiB/s wr, 80 op/s 2023-09-23T04:55:27.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 487011 4.57 MB/sec execute 363 sec latency 13.863 ms 2023-09-23T04:55:28.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 488291 4.58 MB/sec execute 364 sec latency 15.888 ms 2023-09-23T04:55:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:28 smithi005 ceph-mon[97638]: pgmap v285: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2023-09-23T04:55:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:28 smithi118 ceph-mon[134297]: pgmap v285: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2023-09-23T04:55:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:28 smithi069 ceph-mon[133931]: pgmap v285: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2023-09-23T04:55:29.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 488453 4.56 MB/sec execute 365 sec latency 855.592 ms 2023-09-23T04:55:30.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 490319 4.58 MB/sec execute 366 sec latency 973.776 ms 2023-09-23T04:55:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:31 smithi005 ceph-mon[97638]: pgmap v286: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2023-09-23T04:55:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:31 smithi118 ceph-mon[134297]: pgmap v286: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2023-09-23T04:55:31.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 491452 4.58 MB/sec execute 367 sec latency 15.112 ms 2023-09-23T04:55:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:31 smithi069 ceph-mon[133931]: pgmap v286: 129 pgs: 129 active+clean; 353 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.0 MiB/s wr, 97 op/s 2023-09-23T04:55:32.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 492000 4.58 MB/sec execute 368 sec latency 642.837 ms 2023-09-23T04:55:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:32 smithi005 ceph-mon[97638]: pgmap v287: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2023-09-23T04:55:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:32 smithi118 ceph-mon[134297]: pgmap v287: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2023-09-23T04:55:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:32 smithi069 ceph-mon[133931]: pgmap v287: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2023-09-23T04:55:33.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 492000 4.57 MB/sec execute 369 sec latency 1642.926 ms 2023-09-23T04:55:34.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 492000 4.55 MB/sec execute 370 sec latency 2642.975 ms 2023-09-23T04:55:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:34 smithi005 ceph-mon[97638]: pgmap v288: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-09-23T04:55:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:34 smithi118 ceph-mon[134297]: pgmap v288: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-09-23T04:55:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:34 smithi069 ceph-mon[133931]: pgmap v288: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-09-23T04:55:35.959 INFO:tasks.workunit.client.0.smithi005.stdout: 1 492799 4.55 MB/sec execute 371 sec latency 3172.981 ms 2023-09-23T04:55:36.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:35 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:55:36.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:35 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:55:36.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:35 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:55:36.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 494309 4.57 MB/sec execute 372 sec latency 15.334 ms 2023-09-23T04:55:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:55:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:55:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:55:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:36 smithi005 ceph-mon[97638]: pgmap v289: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-09-23T04:55:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:55:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:55:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:55:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:36 smithi118 ceph-mon[134297]: pgmap v289: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-09-23T04:55:37.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:55:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:55:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:55:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:36 smithi069 ceph-mon[133931]: pgmap v289: 129 pgs: 129 active+clean; 359 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-09-23T04:55:37.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495544 4.57 MB/sec execute 373 sec latency 189.789 ms 2023-09-23T04:55:38.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495544 4.56 MB/sec execute 374 sec latency 1189.835 ms 2023-09-23T04:55:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:38 smithi005 ceph-mon[97638]: pgmap v290: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:55:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:38 smithi118 ceph-mon[134297]: pgmap v290: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:55:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:38 smithi069 ceph-mon[133931]: pgmap v290: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:55:39.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495544 4.54 MB/sec execute 375 sec latency 2189.883 ms 2023-09-23T04:55:40.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495544 4.53 MB/sec execute 376 sec latency 3189.936 ms 2023-09-23T04:55:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:41 smithi005 ceph-mon[97638]: pgmap v291: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-09-23T04:55:41.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:41 smithi118 ceph-mon[134297]: pgmap v291: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-09-23T04:55:41.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495678 4.52 MB/sec execute 377 sec latency 4131.944 ms 2023-09-23T04:55:41.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:41 smithi069 ceph-mon[133931]: pgmap v291: 129 pgs: 129 active+clean; 363 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-09-23T04:55:42.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 497667 4.54 MB/sec execute 378 sec latency 15.381 ms 2023-09-23T04:55:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:42 smithi005 ceph-mon[97638]: pgmap v292: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-09-23T04:55:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:42 smithi118 ceph-mon[134297]: pgmap v292: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-09-23T04:55:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:42 smithi069 ceph-mon[133931]: pgmap v292: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2023-09-23T04:55:43.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 499040 4.55 MB/sec execute 379 sec latency 17.134 ms 2023-09-23T04:55:44.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 499092 4.53 MB/sec execute 380 sec latency 946.917 ms 2023-09-23T04:55:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:44 smithi005 ceph-mon[97638]: pgmap v293: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-23T04:55:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:44 smithi118 ceph-mon[134297]: pgmap v293: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-23T04:55:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:44 smithi069 ceph-mon[133931]: pgmap v293: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2023-09-23T04:55:45.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 499092 4.52 MB/sec execute 381 sec latency 1946.993 ms 2023-09-23T04:55:46.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 499224 4.51 MB/sec execute 382 sec latency 2886.416 ms 2023-09-23T04:55:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:46 smithi005 ceph-mon[97638]: pgmap v294: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2023-09-23T04:55:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:46 smithi118 ceph-mon[134297]: pgmap v294: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2023-09-23T04:55:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:46 smithi069 ceph-mon[133931]: pgmap v294: 129 pgs: 129 active+clean; 364 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2023-09-23T04:55:47.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 501198 4.53 MB/sec execute 383 sec latency 13.732 ms 2023-09-23T04:55:48.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 502471 4.54 MB/sec execute 384 sec latency 15.755 ms 2023-09-23T04:55:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:48 smithi005 ceph-mon[97638]: pgmap v295: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2023-09-23T04:55:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:48 smithi118 ceph-mon[134297]: pgmap v295: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2023-09-23T04:55:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:48 smithi069 ceph-mon[133931]: pgmap v295: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2023-09-23T04:55:49.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 502633 4.53 MB/sec execute 385 sec latency 905.255 ms 2023-09-23T04:55:50.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 504281 4.54 MB/sec execute 386 sec latency 1025.473 ms 2023-09-23T04:55:51.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:51 smithi118 ceph-mon[134297]: pgmap v296: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 828 KiB/s wr, 78 op/s 2023-09-23T04:55:51.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 505299 4.54 MB/sec execute 387 sec latency 53.142 ms 2023-09-23T04:55:51.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:51 smithi069 ceph-mon[133931]: pgmap v296: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 828 KiB/s wr, 78 op/s 2023-09-23T04:55:52.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:51 smithi005 ceph-mon[97638]: pgmap v296: 129 pgs: 129 active+clean; 368 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 828 KiB/s wr, 78 op/s 2023-09-23T04:55:52.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 506183 4.54 MB/sec execute 388 sec latency 472.124 ms 2023-09-23T04:55:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:52 smithi005 ceph-mon[97638]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-09-23T04:55:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:52 smithi118 ceph-mon[134297]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-09-23T04:55:53.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:55:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:52 smithi069 ceph-mon[133931]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2023-09-23T04:55:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 -- 172.21.15.5:0/4205584131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 msgr2=0x7fc4201024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 --2- 172.21.15.5:0/4205584131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201024a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b0a0 tx=0x7fc41400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 -- 172.21.15.5:0/4205584131 shutdown_connections 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 --2- 172.21.15.5:0/4205584131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 --2- 172.21.15.5:0/4205584131 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201024a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 --2- 172.21.15.5:0/4205584131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 0x7fc42010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 -- 172.21.15.5:0/4205584131 >> 172.21.15.5:0/4205584131 conn(0x7fc420076f30 msgr2=0x7fc420077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 -- 172.21.15.5:0/4205584131 shutdown_connections 2023-09-23T04:55:53.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.380+0000 7fc4271ff700 1 -- 172.21.15.5:0/4205584131 wait complete. 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.381+0000 7fc4271ff700 1 Processor -- start 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 -- start start 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 0x7fc420199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42019e3a0 con 0x7fc4201029e0 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42019e510 con 0x7fc42010b490 2023-09-23T04:55:53.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc4271ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42019e6d0 con 0x7fc4201020c0 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46608/0 (socket says 172.21.15.5:46608) 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc424f9b700 1 -- 172.21.15.5:0/2170924049 learned_addr learned my addr 172.21.15.5:0/2170924049 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.382+0000 7fc41ffff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 0x7fc420199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc42579c700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 -- 172.21.15.5:0/2170924049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 msgr2=0x7fc42019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42019dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 -- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 msgr2=0x7fc420199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 0x7fc420199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 -- 172.21.15.5:0/2170924049 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c009520 con 0x7fc4201020c0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc42579c700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc424f9b700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b070 tx=0x7fc41400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc41dffb700 1 -- 172.21.15.5:0/2170924049 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400e040 con 0x7fc4201020c0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc41dffb700 1 -- 172.21.15.5:0/2170924049 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc414003e10 con 0x7fc4201020c0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc414009d00 con 0x7fc4201020c0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.383+0000 7fc41dffb700 1 -- 172.21.15.5:0/2170924049 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41401b6a0 con 0x7fc4201020c0 2023-09-23T04:55:53.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.384+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42019ebe0 con 0x7fc4201020c0 2023-09-23T04:55:53.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.387+0000 7fc41dffb700 1 -- 172.21.15.5:0/2170924049 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fc41401d470 con 0x7fc4201020c0 2023-09-23T04:55:53.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.389+0000 7fc41dffb700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 0x7fc4100a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.389+0000 7fc41dffb700 1 -- 172.21.15.5:0/2170924049 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc4140ceac0 con 0x7fc4201020c0 2023-09-23T04:55:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.389+0000 7fc41ffff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 0x7fc4100a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.390+0000 7fc41ffff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 0x7fc4100a3bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc42019a9d0 tx=0x7fc40c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.390+0000 7fc43ab70b80 1 Processor -- start 2023-09-23T04:55:53.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.390+0000 7fc43ab70b80 1 -- start start 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.390+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x56409006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.390+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 0x564090077690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090079fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffd5460 con 0x56409006c400 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffc6c20 con 0x564090077bd0 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffc4540 con 0x56409006cd20 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 0x564090077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 0x564090077690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46622/0 (socket says 172.21.15.5:46622) 2023-09-23T04:55:53.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc41ffff700 1 -- 172.21.15.5:0/1275612249 learned_addr learned my addr 172.21.15.5:0/1275612249 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:53.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x56409006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc42579c700 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090079fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 -- 172.21.15.5:0/1275612249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 msgr2=0x564090077690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 0x564090077690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 -- 172.21.15.5:0/1275612249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 msgr2=0x564090079fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090079fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc424f9b700 1 -- 172.21.15.5:0/1275612249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640900228d0 con 0x56409006c400 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.391+0000 7fc42579c700 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090079fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.392+0000 7fc424f9b700 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x56409006c7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4140cbd80 tx=0x7fc41400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ffdaeab2fcf92e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.392+0000 7fc40affd700 1 -- 172.21.15.5:0/1275612249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140860f0 con 0x56409006c400 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.392+0000 7fc40affd700 1 -- 172.21.15.5:0/1275612249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc41409bd30 con 0x56409006c400 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.392+0000 7fc40affd700 1 -- 172.21.15.5:0/1275612249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41409baf0 con 0x56409006c400 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1275612249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 msgr2=0x56409006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x56409006c7e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4140cbd80 tx=0x7fc41400b3b0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1275612249 shutdown_connections 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090079fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x56409006c7e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1275612249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006cd20 0x564090077690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1275612249 >> 172.21.15.5:0/1275612249 conn(0x56408ffe1260 msgr2=0x564090068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1275612249 shutdown_connections 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.393+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1275612249 wait complete. 2023-09-23T04:55:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.394+0000 7fc43ab70b80 1 -- start start 2023-09-23T04:55:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.394+0000 7fc43ab70b80 1 Processor -- start 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090074f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffbfe30 con 0x56409006cd20 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ff5c710 con 0x564090077bd0 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ff5c3b0 con 0x56409006c400 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc43ab70b80 10 client.0 _ll_register_callbacks cb 0x56408fdb9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:55:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46628/0 (socket says 172.21.15.5:46628) 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc424f9b700 1 -- 172.21.15.5:0/144946821 learned_addr learned my addr 172.21.15.5:0/144946821 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.395+0000 7fc42579c700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 -- 172.21.15.5:0/144946821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 msgr2=0x564090074f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090074f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 -- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 msgr2=0x5640900748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900748c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 -- 172.21.15.5:0/144946821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42019ebe0 con 0x56409006c400 2023-09-23T04:55:53.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc424f9b700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4140cbde0 tx=0x7fc41409b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6c571567e1edfe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc3fffff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140a3930 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc41409ad30 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564090060250 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc41409b890 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc43ab70b80 10 client.27430 fetch_fsmap learned FSMap version 47 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.396+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56408fdeade0 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564090060250 con 0x56409006c400 2023-09-23T04:55:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc41401d720 con 0x56409006c400 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 10 client.27430 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 10 client.27430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 20 client.27430 populate_metadata read hostname 'smithi005' 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564090086f70 0x564090089360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 4 client.27430 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56408ffb8e10 con 0x564090086f70 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.397+0000 7fc41ffff700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564090086f70 0x564090089360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.398+0000 7fc41ffff700 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564090086f70 0x564090089360 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c04e0caf87a236 server_cookie=2f52b22f7f168412 in_seq=0 out_seq=0 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.398+0000 7fc3fffff700 10 client.27430 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc3fffff700 1 -- 172.21.15.5:0/144946821 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56408ffb8e10 con 0x564090086f70 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc3fffff700 10 client.27430 handle_command_reply: tid=0 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 client.27430 shutdown 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 msgr2=0x5640900775a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4140cbde0 tx=0x7fc41409b630 comp rx=0 tx=0).stop 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 shutdown_connections 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564090086f70 0x564090089360 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564090077bd0 0x564090074f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56409006c400 0x5640900775a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/144946821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 >> 172.21.15.5:0/144946821 conn(0x56408ffe1260 msgr2=0x564090068320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc3fffff700 0 client.27430 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 shutdown_connections 2023-09-23T04:55:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 1 -- 172.21.15.5:0/144946821 wait complete. 2023-09-23T04:55:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.399+0000 7fc43ab70b80 20 client.27430 trim_cache size 0 max 16384 2023-09-23T04:55:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.420+0000 7fc43ab70b80 1 Processor -- start 2023-09-23T04:55:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.420+0000 7fc43ab70b80 1 -- start start 2023-09-23T04:55:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x564090071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408fe40920 con 0x56409006cd20 2023-09-23T04:55:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409006de90 con 0x56409006c400 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564090070770 con 0x564090077bd0 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc42579c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x564090071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc41ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38984/0 (socket says 172.21.15.5:38984) 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:47170/0 (socket says 172.21.15.5:47170) 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.421+0000 7fc41ffff700 1 -- 172.21.15.5:0/3923017951 learned_addr learned my addr 172.21.15.5:0/3923017951 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 -- 172.21.15.5:0/3923017951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 msgr2=0x564090071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x564090071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 -- 172.21.15.5:0/3923017951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 msgr2=0x56409007e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 -- 172.21.15.5:0/3923017951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56408fdd3c60 con 0x56409006cd20 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc424f9b700 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc41ffff700 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009cb0 tx=0x7fc40c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff1b9488db05cef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc40b7fe700 1 -- 172.21.15.5:0/3923017951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c004200 con 0x56409006cd20 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc40b7fe700 1 -- 172.21.15.5:0/3923017951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc40c023040 con 0x56409006cd20 2023-09-23T04:55:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.422+0000 7fc40b7fe700 1 -- 172.21.15.5:0/3923017951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00ec00 con 0x56409006cd20 2023-09-23T04:55:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 -- 172.21.15.5:0/3923017951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 msgr2=0x5640900770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009cb0 tx=0x7fc40c014000 comp rx=0 tx=0).stop 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 -- 172.21.15.5:0/3923017951 shutdown_connections 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x564090071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006c400 0x56409007e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/3923017951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006cd20 0x5640900770d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 -- 172.21.15.5:0/3923017951 >> 172.21.15.5:0/3923017951 conn(0x56408ffe1260 msgr2=0x56409006fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 -- 172.21.15.5:0/3923017951 shutdown_connections 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.423+0000 7fc43ab70b80 1 -- 172.21.15.5:0/3923017951 wait complete. 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.424+0000 7fc43ab70b80 1 -- start start 2023-09-23T04:55:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.424+0000 7fc43ab70b80 1 Processor -- start 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.424+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x5640900dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.424+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006cd20 0x5640900dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc43ab70b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x5640900db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640900dbb00 con 0x56409006c400 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640900dbc70 con 0x56409006cd20 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc43ab70b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640900dbe30 con 0x564090077bd0 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc43ab70b80 10 client.0 _ll_register_callbacks cb 0x7fc40c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x5640900dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc424f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x5640900dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38996/0 (socket says 172.21.15.5:38996) 2023-09-23T04:55:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x5640900db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc424f9b700 1 -- 172.21.15.5:0/1712428156 learned_addr learned my addr 172.21.15.5:0/1712428156 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc41ffff700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006cd20 0x5640900dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 -- 172.21.15.5:0/1712428156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006cd20 msgr2=0x5640900dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006cd20 0x5640900dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 -- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 msgr2=0x5640900dc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x5640900dc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.425+0000 7fc42579c700 1 -- 172.21.15.5:0/1712428156 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56408fdeade0 con 0x564090077bd0 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc42579c700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x5640900db550 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564090076020 tx=0x7fc418012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a2797ee5268756 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:55:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc3fffff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418017070 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc418004260 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc41409bd30 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc41800bee0 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc43ab70b80 10 client.27436 fetch_fsmap learned FSMap version 47 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.426+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fc40c017070 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc41409bd30 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc41801c650 con 0x564090077bd0 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 10 client.27436 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 10 client.27436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:55:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 20 client.27436 populate_metadata read hostname 'smithi005' 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5640900e48a0 0x5640900e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 4 client.27436 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56408ffb8b30 con 0x5640900e48a0 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.427+0000 7fc424f9b700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5640900e48a0 0x5640900e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc424f9b700 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5640900e48a0 0x5640900e6cc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96bf02ded1b157a2 server_cookie=213f0ee260d898ba in_seq=0 out_seq=0 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc3fffff700 10 client.27436 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc3fffff700 1 -- 172.21.15.5:0/1712428156 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56408ffb8b30 con 0x5640900e48a0 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc3fffff700 10 client.27436 handle_command_reply: tid=0 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 client.27436 shutdown 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 msgr2=0x5640900db550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x5640900db550 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x564090076020 tx=0x7fc418012730 comp rx=0 tx=0).stop 2023-09-23T04:55:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 shutdown_connections 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5640900e48a0 0x5640900e6cc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564090077bd0 0x5640900db550 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56409006c400 0x5640900dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc3fffff700 0 client.27436 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 --2- 172.21.15.5:0/1712428156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56409006cd20 0x5640900dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 >> 172.21.15.5:0/1712428156 conn(0x56408ffe1260 msgr2=0x56409006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 shutdown_connections 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.428+0000 7fc43ab70b80 1 -- 172.21.15.5:0/1712428156 wait complete. 2023-09-23T04:55:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc43ab70b80 20 client.27436 trim_cache size 0 max 16384 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:55:53.436 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 msgr2=0x7fc4100a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 0x7fc4100a3bd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc42019a9d0 tx=0x7fc40c009260 comp rx=0 tx=0).stop 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 msgr2=0x7fc4201993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b070 tx=0x7fc41400bce0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 shutdown_connections 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc4100a1710 0x7fc4100a3bd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc42010b490 0x7fc42019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc4201020c0 0x7fc4201993c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 --2- 172.21.15.5:0/2170924049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4201029e0 0x7fc420199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:55:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.429+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 >> 172.21.15.5:0/2170924049 conn(0x7fc420076f30 msgr2=0x7fc420107270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:55:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.430+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 shutdown_connections 2023-09-23T04:55:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:55:53.430+0000 7fc4271ff700 1 -- 172.21.15.5:0/2170924049 wait complete. 2023-09-23T04:55:53.454 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:55:53.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 506183 4.53 MB/sec execute 389 sec latency 1472.167 ms 2023-09-23T04:55:54.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 506183 4.52 MB/sec execute 390 sec latency 2472.216 ms 2023-09-23T04:55:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:54 smithi005 ceph-mon[97638]: pgmap v298: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-09-23T04:55:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:54 smithi118 ceph-mon[134297]: pgmap v298: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-09-23T04:55:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:54 smithi069 ceph-mon[133931]: pgmap v298: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2023-09-23T04:55:55.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 507208 4.52 MB/sec execute 391 sec latency 2994.716 ms 2023-09-23T04:55:56.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 508576 4.53 MB/sec execute 392 sec latency 15.954 ms 2023-09-23T04:55:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:56 smithi005 ceph-mon[97638]: pgmap v299: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-09-23T04:55:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:56 smithi118 ceph-mon[134297]: pgmap v299: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-09-23T04:55:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:56 smithi069 ceph-mon[133931]: pgmap v299: 129 pgs: 129 active+clean; 374 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2023-09-23T04:55:57.960 INFO:tasks.workunit.client.0.smithi005.stdout: 1 509726 4.53 MB/sec execute 393 sec latency 144.564 ms 2023-09-23T04:55:58.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 509726 4.52 MB/sec execute 394 sec latency 1144.631 ms 2023-09-23T04:55:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:55:58 smithi005 ceph-mon[97638]: pgmap v300: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-09-23T04:55:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:55:58 smithi118 ceph-mon[134297]: pgmap v300: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-09-23T04:55:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:55:58 smithi069 ceph-mon[133931]: pgmap v300: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2023-09-23T04:55:59.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 509726 4.51 MB/sec execute 395 sec latency 2144.676 ms 2023-09-23T04:56:00.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 509726 4.50 MB/sec execute 396 sec latency 3144.721 ms 2023-09-23T04:56:01.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:01 smithi118 ceph-mon[134297]: pgmap v301: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2023-09-23T04:56:01.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 509810 4.49 MB/sec execute 397 sec latency 4086.510 ms 2023-09-23T04:56:01.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:01 smithi069 ceph-mon[133931]: pgmap v301: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2023-09-23T04:56:02.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:01 smithi005 ceph-mon[97638]: pgmap v301: 129 pgs: 129 active+clean; 378 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2023-09-23T04:56:02.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 510424 4.48 MB/sec execute 398 sec latency 62.280 ms 2023-09-23T04:56:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:02 smithi005 ceph-mon[97638]: pgmap v302: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2023-09-23T04:56:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:02 smithi118 ceph-mon[134297]: pgmap v302: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2023-09-23T04:56:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:02 smithi069 ceph-mon[133931]: pgmap v302: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2023-09-23T04:56:03.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 511970 4.50 MB/sec execute 399 sec latency 11.209 ms 2023-09-23T04:56:04.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 513272 4.50 MB/sec execute 400 sec latency 49.069 ms 2023-09-23T04:56:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:04 smithi005 ceph-mon[97638]: pgmap v303: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:56:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:04 smithi118 ceph-mon[134297]: pgmap v303: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:56:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:04 smithi069 ceph-mon[133931]: pgmap v303: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:56:05.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 513272 4.49 MB/sec execute 401 sec latency 1012.697 ms 2023-09-23T04:56:06.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 513406 4.48 MB/sec execute 402 sec latency 1947.817 ms 2023-09-23T04:56:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:06 smithi005 ceph-mon[97638]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T04:56:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:06 smithi005 ceph-mon[97638]: pgmap v305: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-09-23T04:56:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:06 smithi118 ceph-mon[134297]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T04:56:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:06 smithi118 ceph-mon[134297]: pgmap v305: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-09-23T04:56:07.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:06 smithi069 ceph-mon[133931]: osdmap e99: 12 total, 12 up, 12 in 2023-09-23T04:56:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:06 smithi069 ceph-mon[133931]: pgmap v305: 129 pgs: 129 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2023-09-23T04:56:07.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 515365 4.50 MB/sec execute 403 sec latency 15.310 ms 2023-09-23T04:56:08.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 515848 4.49 MB/sec execute 404 sec latency 393.714 ms 2023-09-23T04:56:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:08 smithi005 ceph-mon[97638]: pgmap v306: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-09-23T04:56:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:08 smithi118 ceph-mon[134297]: pgmap v306: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-09-23T04:56:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:08 smithi069 ceph-mon[133931]: pgmap v306: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-09-23T04:56:09.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 516816 4.49 MB/sec execute 405 sec latency 354.544 ms 2023-09-23T04:56:10.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 518449 4.50 MB/sec execute 406 sec latency 472.573 ms 2023-09-23T04:56:11.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:11 smithi118 ceph-mon[134297]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T04:56:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:11 smithi118 ceph-mon[134297]: pgmap v308: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2023-09-23T04:56:11.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 519478 4.50 MB/sec execute 407 sec latency 16.524 ms 2023-09-23T04:56:11.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:11 smithi069 ceph-mon[133931]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T04:56:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:11 smithi069 ceph-mon[133931]: pgmap v308: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2023-09-23T04:56:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:11 smithi005 ceph-mon[97638]: osdmap e100: 12 total, 12 up, 12 in 2023-09-23T04:56:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:11 smithi005 ceph-mon[97638]: pgmap v308: 129 pgs: 129 active+clean; 383 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2023-09-23T04:56:12.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 520365 4.50 MB/sec execute 408 sec latency 427.964 ms 2023-09-23T04:56:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:12 smithi005 ceph-mon[97638]: pgmap v309: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2023-09-23T04:56:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:12 smithi118 ceph-mon[134297]: pgmap v309: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2023-09-23T04:56:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:12 smithi069 ceph-mon[133931]: pgmap v309: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2023-09-23T04:56:13.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 520365 4.49 MB/sec execute 409 sec latency 1428.008 ms 2023-09-23T04:56:14.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 520365 4.48 MB/sec execute 410 sec latency 2428.072 ms 2023-09-23T04:56:15.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:14 smithi005 ceph-mon[97638]: pgmap v310: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:56:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:14 smithi118 ceph-mon[134297]: pgmap v310: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:56:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:14 smithi069 ceph-mon[133931]: pgmap v310: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2023-09-23T04:56:15.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 521255 4.48 MB/sec execute 411 sec latency 2959.224 ms 2023-09-23T04:56:16.961 INFO:tasks.workunit.client.0.smithi005.stdout: 1 522749 4.49 MB/sec execute 412 sec latency 16.067 ms 2023-09-23T04:56:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:16 smithi005 ceph-mon[97638]: pgmap v311: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-09-23T04:56:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:16 smithi118 ceph-mon[134297]: pgmap v311: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-09-23T04:56:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:16 smithi069 ceph-mon[133931]: pgmap v311: 129 pgs: 129 active+clean; 389 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-09-23T04:56:17.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523906 4.50 MB/sec execute 413 sec latency 253.002 ms 2023-09-23T04:56:18.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523906 4.49 MB/sec execute 414 sec latency 1253.046 ms 2023-09-23T04:56:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:18 smithi005 ceph-mon[97638]: pgmap v312: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-09-23T04:56:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:18 smithi118 ceph-mon[134297]: pgmap v312: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-09-23T04:56:19.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:18 smithi069 ceph-mon[133931]: pgmap v312: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2023-09-23T04:56:19.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523906 4.47 MB/sec execute 415 sec latency 2253.109 ms 2023-09-23T04:56:20.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523906 4.46 MB/sec execute 416 sec latency 3253.157 ms 2023-09-23T04:56:21.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 524040 4.45 MB/sec execute 417 sec latency 4196.603 ms 2023-09-23T04:56:21.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:21 smithi069 ceph-mon[133931]: pgmap v313: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-09-23T04:56:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:21 smithi005 ceph-mon[97638]: pgmap v313: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-09-23T04:56:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:21 smithi118 ceph-mon[134297]: pgmap v313: 129 pgs: 129 active+clean; 394 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2023-09-23T04:56:22.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 526029 4.48 MB/sec execute 418 sec latency 15.640 ms 2023-09-23T04:56:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:22 smithi005 ceph-mon[97638]: pgmap v314: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-09-23T04:56:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:22 smithi118 ceph-mon[134297]: pgmap v314: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-09-23T04:56:23.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:22 smithi069 ceph-mon[133931]: pgmap v314: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2023-09-23T04:56:23.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:56:23.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 -- 172.21.15.5:0/2900309765 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 msgr2=0x7efcd01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2900309765 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd01014d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efccc009990 tx=0x7efccc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 -- 172.21.15.5:0/2900309765 shutdown_connections 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2900309765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0106210 0x7efcd010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2900309765 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd01014d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2900309765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0101aa0 0x7efcd0105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.729+0000 7efcd7dff700 1 -- 172.21.15.5:0/2900309765 >> 172.21.15.5:0/2900309765 conn(0x7efcd00fc9d0 msgr2=0x7efcd00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.730+0000 7efcd7dff700 1 -- 172.21.15.5:0/2900309765 shutdown_connections 2023-09-23T04:56:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.730+0000 7efcd7dff700 1 -- 172.21.15.5:0/2900309765 wait complete. 2023-09-23T04:56:23.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.731+0000 7efcd7dff700 1 Processor -- start 2023-09-23T04:56:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 -- start start 2023-09-23T04:56:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd0196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 0x7efcd0197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd019b980 0x7efcd019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019c400 con 0x7efcd019b980 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019c570 con 0x7efcd0106210 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd7dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019c730 con 0x7efcd01010f0 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd5b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd0196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd5b9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd0196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49134/0 (socket says 172.21.15.5:49134) 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd5b9b700 1 -- 172.21.15.5:0/2440760249 learned_addr learned my addr 172.21.15.5:0/2440760249 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.732+0000 7efcd539a700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 0x7efcd0197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd639c700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd019b980 0x7efcd019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 -- 172.21.15.5:0/2440760249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 msgr2=0x7efcd0196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd0196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 -- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd019b980 msgr2=0x7efcd019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd019b980 0x7efcd019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 -- 172.21.15.5:0/2440760249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0095f0 con 0x7efcd0106210 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd539a700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 0x7efcd0197520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcc000e9c0 tx=0x7efcc000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcc6ffd700 1 -- 172.21.15.5:0/2440760249 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc000c9c0 con 0x7efcd0106210 2023-09-23T04:56:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcc6ffd700 1 -- 172.21.15.5:0/2440760249 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc0010430 con 0x7efcd0106210 2023-09-23T04:56:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd01a3c90 con 0x7efcd0106210 2023-09-23T04:56:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.733+0000 7efcc6ffd700 1 -- 172.21.15.5:0/2440760249 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0018680 con 0x7efcd0106210 2023-09-23T04:56:23.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.734+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd01a4170 con 0x7efcd0106210 2023-09-23T04:56:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.738+0000 7efcc6ffd700 1 -- 172.21.15.5:0/2440760249 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7efcc000cb20 con 0x7efcd0106210 2023-09-23T04:56:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.739+0000 7efcc6ffd700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 0x7efcbc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.739+0000 7efcc6ffd700 1 -- 172.21.15.5:0/2440760249 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7efcc0014070 con 0x7efcd0106210 2023-09-23T04:56:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.739+0000 7efceb7d3b80 1 Processor -- start 2023-09-23T04:56:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 0x7efcbc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 -- start start 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddbd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dd1b460 con 0x55901ddb2d20 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dd0cc20 con 0x55901ddb2400 2023-09-23T04:56:23.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.740+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dd0a540 con 0x55901ddbdbd0 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 -- 172.21.15.5:0/739993808 learned_addr learned my addr 172.21.15.5:0/739993808 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd539a700 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddbd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 -- 172.21.15.5:0/739993808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 msgr2=0x55901ddb27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 -- 172.21.15.5:0/739993808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 msgr2=0x55901ddbd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddbd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd639c700 1 -- 172.21.15.5:0/739993808 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55901dd688d0 con 0x55901ddbdbd0 2023-09-23T04:56:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.741+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 0x7efcbc0a3bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc0053c0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.742+0000 7efcd639c700 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efcc8011680 tx=0x7efcc8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=860a45ed80c31430 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.742+0000 7efcbbfff700 1 -- 172.21.15.5:0/739993808 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800e040 con 0x55901ddbdbd0 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.742+0000 7efcbbfff700 1 -- 172.21.15.5:0/739993808 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc800f040 con 0x55901ddbdbd0 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.742+0000 7efcbbfff700 1 -- 172.21.15.5:0/739993808 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004850 con 0x55901ddbdbd0 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- 172.21.15.5:0/739993808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 msgr2=0x55901ddbffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efcc8011680 tx=0x7efcc8009920 comp rx=0 tx=0).stop 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- 172.21.15.5:0/739993808 shutdown_connections 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddbffc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/739993808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddbd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- 172.21.15.5:0/739993808 >> 172.21.15.5:0/739993808 conn(0x55901dd27260 msgr2=0x55901ddae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- 172.21.15.5:0/739993808 shutdown_connections 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- 172.21.15.5:0/739993808 wait complete. 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.743+0000 7efceb7d3b80 1 -- start start 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 Processor -- start 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddc7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 0x55901ddc79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddbdbd0 0x55901ddc1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dd05e30 con 0x55901ddbdbd0 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dca2710 con 0x55901ddb2400 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901dca23b0 con 0x55901ddb2d20 2023-09-23T04:56:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.744+0000 7efceb7d3b80 10 client.0 _ll_register_callbacks cb 0x55901daff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd539a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 0x55901ddc79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddc7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd539a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 0x55901ddc79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49154/0 (socket says 172.21.15.5:49154) 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd539a700 1 -- 172.21.15.5:0/2062795995 learned_addr learned my addr 172.21.15.5:0/2062795995 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd639c700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddbdbd0 0x55901ddc1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 -- 172.21.15.5:0/2062795995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 msgr2=0x55901ddc79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 0x55901ddc79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 -- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddbdbd0 msgr2=0x55901ddc1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddbdbd0 0x55901ddc1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 -- 172.21.15.5:0/2062795995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc8011330 con 0x55901ddb2400 2023-09-23T04:56:23.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddc7490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b690 tx=0x7efccc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69722e99e8dbfaec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.745+0000 7efcb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc004ad0 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55901dda6250 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7efccc004c30 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efccc004df0 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efceb7d3b80 10 client.27392 fetch_fsmap learned FSMap version 47 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55901db30de0 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.746+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55901dda6250 con 0x55901ddb2400 2023-09-23T04:56:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7efccc02f030 con 0x55901ddb2400 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 10 client.27392 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 20 client.27392 populate_metadata read hostname 'smithi005' 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901ddd18c0 0x55901ddd3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 4 client.27392 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55901dcfee10 con 0x55901ddd18c0 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.747+0000 7efcd539a700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901ddd18c0 0x55901ddd3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.748+0000 7efcd539a700 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901ddd18c0 0x55901ddd3ce0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd0e63b04331e4a server_cookie=ecd33df50dc80ce1 in_seq=0 out_seq=0 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.748+0000 7efcb97fa700 10 client.27392 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.748+0000 7efcb97fa700 1 -- 172.21.15.5:0/2062795995 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55901dcfee10 con 0x55901ddd18c0 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.748+0000 7efcb97fa700 10 client.27392 handle_command_reply: tid=0 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.748+0000 7efceb7d3b80 1 client.27392 shutdown 2023-09-23T04:56:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 msgr2=0x55901ddc7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddc7490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b690 tx=0x7efccc009ca0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 shutdown_connections 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901ddd18c0 0x55901ddd3ce0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddbdbd0 0x55901ddc1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddc7490 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/2062795995 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddb2d20 0x55901ddc79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efcb97fa700 0 client.27392 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 >> 172.21.15.5:0/2062795995 conn(0x55901dd27260 msgr2=0x55901ddae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 shutdown_connections 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 1 -- 172.21.15.5:0/2062795995 wait complete. 2023-09-23T04:56:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.749+0000 7efceb7d3b80 20 client.27392 trim_cache size 0 max 16384 2023-09-23T04:56:23.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.773+0000 7efceb7d3b80 1 Processor -- start 2023-09-23T04:56:23.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.773+0000 7efceb7d3b80 1 -- start start 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901db86920 con 0x55901ddb2d20 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901ddbda30 con 0x55901ddb2400 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901ddb49a0 con 0x55901ddbdbd0 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd5b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd5b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:51210/0 (socket says 172.21.15.5:51210) 2023-09-23T04:56:23.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49170/0 (socket says 172.21.15.5:49170) 2023-09-23T04:56:23.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd5b9b700 1 -- 172.21.15.5:0/1865188179 learned_addr learned my addr 172.21.15.5:0/1865188179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 -- 172.21.15.5:0/1865188179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 msgr2=0x55901ddb9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 -- 172.21.15.5:0/1865188179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 msgr2=0x55901ddba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.774+0000 7efcd639c700 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcd639c700 1 -- 172.21.15.5:0/1865188179 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55901ddaa480 con 0x55901ddbdbd0 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcd539a700 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcd639c700 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000f80 tx=0x7efcc8006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46be376bc95956bc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcc4bff700 1 -- 172.21.15.5:0/1865188179 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800e050 con 0x55901ddbdbd0 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcc4bff700 1 -- 172.21.15.5:0/1865188179 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc80041c0 con 0x55901ddbdbd0 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.775+0000 7efcc4bff700 1 -- 172.21.15.5:0/1865188179 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8007ab0 con 0x55901ddbdbd0 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 -- 172.21.15.5:0/1865188179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 msgr2=0x55901ddc7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000f80 tx=0x7efcc8006000 comp rx=0 tx=0).stop 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 -- 172.21.15.5:0/1865188179 shutdown_connections 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc7060 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2400 0x55901ddb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/1865188179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2d20 0x55901ddba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 -- 172.21.15.5:0/1865188179 >> 172.21.15.5:0/1865188179 conn(0x55901dd27260 msgr2=0x55901ddb4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 -- 172.21.15.5:0/1865188179 shutdown_connections 2023-09-23T04:56:23.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.776+0000 7efceb7d3b80 1 -- 172.21.15.5:0/1865188179 wait complete. 2023-09-23T04:56:23.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 -- start start 2023-09-23T04:56:23.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 Processor -- start 2023-09-23T04:56:23.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2400 0x55901ddc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901de258a0 con 0x55901ddb2400 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901de25a10 con 0x55901ddb2d20 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.777+0000 7efceb7d3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55901de25bd0 con 0x55901ddbdbd0 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efceb7d3b80 10 client.0 _ll_register_callbacks cb 0x55901daff800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd639c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:51216/0 (socket says 172.21.15.5:51216) 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 -- 172.21.15.5:0/3084756472 learned_addr learned my addr 172.21.15.5:0/3084756472 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2400 0x55901ddc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 -- 172.21.15.5:0/3084756472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 msgr2=0x55901ddc6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 -- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2400 msgr2=0x55901ddc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2400 0x55901ddc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 -- 172.21.15.5:0/3084756472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55901db30de0 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcd539a700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcc00cb1b0 tx=0x7efcc009a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3acae53ed2277d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.778+0000 7efcb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0062060 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcc800f040 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7efcc009e520 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcc00a2760 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 10 client.27398 fetch_fsmap learned FSMap version 47 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7efcd01a4170 con 0x55901ddb2d20 2023-09-23T04:56:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc800f040 con 0x55901ddb2d20 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 47) v1 ==== 7922+0+0 (secure 0 0 0) 0x7efcc009a630 con 0x55901ddb2d20 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 10 client.27398 fetch_fsmap finished waiting for FSMap version 47 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 10 client.27398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 20 client.27398 populate_metadata read hostname 'smithi005' 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901de27480 0x55901de298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 4 client.27398 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.779+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55901dcfeb30 con 0x55901de27480 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901de27480 0x55901de298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efcd5b9b700 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901de27480 0x55901de298a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89f7b73da498aa2 server_cookie=9db79b9e8bb7c649 in_seq=0 out_seq=0 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efcb97fa700 10 client.27398 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efcb97fa700 1 -- 172.21.15.5:0/3084756472 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55901dcfeb30 con 0x55901de27480 2023-09-23T04:56:23.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efcb97fa700 10 client.27398 handle_command_reply: tid=0 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.780+0000 7efceb7d3b80 1 client.27398 shutdown 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 msgr2=0x55901ddc65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcc00cb1b0 tx=0x7efcc009a1c0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 shutdown_connections 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55901de27480 0x55901de298a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55901ddbdbd0 0x55901ddc6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55901ddb2400 0x55901ddc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efcb97fa700 0 client.27398 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 --2- 172.21.15.5:0/3084756472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55901ddb2d20 0x55901ddc65d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 >> 172.21.15.5:0/3084756472 conn(0x55901dd27260 msgr2=0x55901ddb9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 shutdown_connections 2023-09-23T04:56:23.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 1 -- 172.21.15.5:0/3084756472 wait complete. 2023-09-23T04:56:23.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.781+0000 7efceb7d3b80 20 client.27398 trim_cache size 0 max 16384 2023-09-23T04:56:23.785 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:56:23.785 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:56:23.785 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:56:23.786 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 msgr2=0x7efcbc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 0x7efcbc0a3bd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efccc006010 tx=0x7efccc0053c0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 msgr2=0x7efcd0197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 0x7efcd0197520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcc000e9c0 tx=0x7efcc000ecd0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 shutdown_connections 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7efcbc0a1710 0x7efcbc0a3bd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd019b980 0x7efcd019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7efcd01010f0 0x7efcd0196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 --2- 172.21.15.5:0/2440760249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efcd0106210 0x7efcd0197520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 >> 172.21.15.5:0/2440760249 conn(0x7efcd00fc9d0 msgr2=0x7efcd0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 shutdown_connections 2023-09-23T04:56:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:23.782+0000 7efcd7dff700 1 -- 172.21.15.5:0/2440760249 wait complete. 2023-09-23T04:56:23.804 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:56:23.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 527257 4.48 MB/sec execute 419 sec latency 15.680 ms 2023-09-23T04:56:24.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 527456 4.47 MB/sec execute 420 sec latency 861.093 ms 2023-09-23T04:56:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:24 smithi005 ceph-mon[97638]: pgmap v315: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-09-23T04:56:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:24 smithi118 ceph-mon[134297]: pgmap v315: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-09-23T04:56:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:24 smithi069 ceph-mon[133931]: pgmap v315: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2023-09-23T04:56:25.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 527456 4.46 MB/sec execute 421 sec latency 1861.142 ms 2023-09-23T04:56:26.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 527590 4.45 MB/sec execute 422 sec latency 2800.209 ms 2023-09-23T04:56:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:26 smithi005 ceph-mon[97638]: pgmap v316: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2023-09-23T04:56:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:26 smithi118 ceph-mon[134297]: pgmap v316: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2023-09-23T04:56:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:26 smithi069 ceph-mon[133931]: pgmap v316: 129 pgs: 129 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2023-09-23T04:56:27.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 529550 4.47 MB/sec execute 423 sec latency 14.564 ms 2023-09-23T04:56:28.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 530813 4.47 MB/sec execute 424 sec latency 14.923 ms 2023-09-23T04:56:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:28 smithi005 ceph-mon[97638]: pgmap v317: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2023-09-23T04:56:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:28 smithi118 ceph-mon[134297]: pgmap v317: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2023-09-23T04:56:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:28 smithi069 ceph-mon[133931]: pgmap v317: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2023-09-23T04:56:29.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 530999 4.46 MB/sec execute 425 sec latency 862.048 ms 2023-09-23T04:56:30.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 532837 4.48 MB/sec execute 426 sec latency 980.420 ms 2023-09-23T04:56:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:31 smithi118 ceph-mon[134297]: pgmap v318: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2023-09-23T04:56:31.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 533956 4.48 MB/sec execute 427 sec latency 22.869 ms 2023-09-23T04:56:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:31 smithi069 ceph-mon[133931]: pgmap v318: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2023-09-23T04:56:32.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:31 smithi005 ceph-mon[97638]: pgmap v318: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2023-09-23T04:56:32.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 534579 4.47 MB/sec execute 428 sec latency 633.813 ms 2023-09-23T04:56:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:32 smithi005 ceph-mon[97638]: pgmap v319: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2023-09-23T04:56:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:32 smithi118 ceph-mon[134297]: pgmap v319: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2023-09-23T04:56:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:32 smithi069 ceph-mon[133931]: pgmap v319: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2023-09-23T04:56:33.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 534579 4.46 MB/sec execute 429 sec latency 1633.864 ms 2023-09-23T04:56:34.962 INFO:tasks.workunit.client.0.smithi005.stdout: 1 534579 4.45 MB/sec execute 430 sec latency 2633.922 ms 2023-09-23T04:56:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:34 smithi005 ceph-mon[97638]: pgmap v320: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-23T04:56:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:34 smithi118 ceph-mon[134297]: pgmap v320: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-23T04:56:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:34 smithi069 ceph-mon[133931]: pgmap v320: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.7 MiB/s wr, 138 op/s 2023-09-23T04:56:35.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 535449 4.45 MB/sec execute 431 sec latency 3158.111 ms 2023-09-23T04:56:36.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 536865 4.46 MB/sec execute 432 sec latency 20.245 ms 2023-09-23T04:56:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:56:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:56:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:56:37.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538122 4.47 MB/sec execute 433 sec latency 21.216 ms 2023-09-23T04:56:37.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:37 smithi069 ceph-mon[133931]: pgmap v321: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:38.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:37 smithi005 ceph-mon[97638]: pgmap v321: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:38.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:37 smithi118 ceph-mon[134297]: pgmap v321: 129 pgs: 129 active+clean; 407 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:38.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538122 4.46 MB/sec execute 434 sec latency 1003.208 ms 2023-09-23T04:56:38.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:38 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:38 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:38 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:39.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538122 4.45 MB/sec execute 435 sec latency 2003.594 ms 2023-09-23T04:56:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:39 smithi069 ceph-mon[133931]: pgmap v322: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-09-23T04:56:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:39 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:39 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:39 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:39.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:39 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:39 smithi005 ceph-mon[97638]: pgmap v322: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-09-23T04:56:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:39 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:39 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:39 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:39 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:39 smithi118 ceph-mon[134297]: pgmap v322: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-09-23T04:56:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:39 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:39 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:39 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:39 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538122 4.43 MB/sec execute 436 sec latency 3003.636 ms 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:40.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:40 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:40 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:41.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:40 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:41.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538206 4.42 MB/sec execute 437 sec latency 3950.305 ms 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: pgmap v323: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:56:41.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:56:41.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:56:42.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: pgmap v323: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:56:42.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T04:56:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: pgmap v323: 129 pgs: 129 active+clean; 411 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:56:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:56:42.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 540215 4.45 MB/sec execute 438 sec latency 22.145 ms 2023-09-23T04:56:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:42 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:56:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:42 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:56:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:42 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:56:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:42 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:56:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:42 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T04:56:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:42 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T04:56:43.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 541472 4.45 MB/sec execute 439 sec latency 17.999 ms 2023-09-23T04:56:43.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:43 smithi069 ceph-mon[133931]: pgmap v324: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2023-09-23T04:56:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:43 smithi005 ceph-mon[97638]: pgmap v324: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2023-09-23T04:56:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:43 smithi118 ceph-mon[134297]: pgmap v324: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 157 op/s 2023-09-23T04:56:44.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 541671 4.44 MB/sec execute 440 sec latency 858.074 ms 2023-09-23T04:56:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:44 smithi005 ceph-mon[97638]: pgmap v325: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2023-09-23T04:56:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:44 smithi118 ceph-mon[134297]: pgmap v325: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2023-09-23T04:56:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:44 smithi069 ceph-mon[133931]: pgmap v325: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2023-09-23T04:56:45.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 541671 4.43 MB/sec execute 441 sec latency 1858.138 ms 2023-09-23T04:56:46.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 541757 4.42 MB/sec execute 442 sec latency 2802.671 ms 2023-09-23T04:56:47.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:46 smithi005 ceph-mon[97638]: pgmap v326: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2023-09-23T04:56:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:46 smithi118 ceph-mon[134297]: pgmap v326: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2023-09-23T04:56:47.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:46 smithi069 ceph-mon[133931]: pgmap v326: 129 pgs: 129 active+clean; 412 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2023-09-23T04:56:47.963 INFO:tasks.workunit.client.0.smithi005.stdout: 1 543757 4.44 MB/sec execute 443 sec latency 15.877 ms 2023-09-23T04:56:48.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 545013 4.44 MB/sec execute 444 sec latency 16.913 ms 2023-09-23T04:56:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:48 smithi005 ceph-mon[97638]: pgmap v327: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2023-09-23T04:56:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:48 smithi118 ceph-mon[134297]: pgmap v327: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2023-09-23T04:56:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:48 smithi069 ceph-mon[133931]: pgmap v327: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2023-09-23T04:56:49.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 545212 4.43 MB/sec execute 445 sec latency 851.801 ms 2023-09-23T04:56:50.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 547177 4.45 MB/sec execute 446 sec latency 966.814 ms 2023-09-23T04:56:51.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 547689 4.44 MB/sec execute 447 sec latency 80.586 ms 2023-09-23T04:56:51.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:51 smithi069 ceph-mon[133931]: pgmap v328: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2023-09-23T04:56:52.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:51 smithi005 ceph-mon[97638]: pgmap v328: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2023-09-23T04:56:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:51 smithi118 ceph-mon[134297]: pgmap v328: 129 pgs: 129 active+clean; 417 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 92 op/s 2023-09-23T04:56:52.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 548762 4.44 MB/sec execute 448 sec latency 178.100 ms 2023-09-23T04:56:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:52 smithi005 ceph-mon[97638]: pgmap v329: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 185 op/s 2023-09-23T04:56:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:52 smithi118 ceph-mon[134297]: pgmap v329: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 185 op/s 2023-09-23T04:56:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:52 smithi069 ceph-mon[133931]: pgmap v329: 129 pgs: 129 active+clean; 420 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 185 op/s 2023-09-23T04:56:53.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:56:53.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 548762 4.43 MB/sec execute 449 sec latency 1178.155 ms 2023-09-23T04:56:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/1496539502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8106f30 msgr2=0x7ff7e8107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/1496539502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8106f30 0x7ff7e8107310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40099c0 tx=0x7ff7e4009cd0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/1496539502 shutdown_connections 2023-09-23T04:56:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/1496539502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/1496539502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8106f30 0x7ff7e8107310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/1496539502 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8068e10 0x7ff7e8069290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/1496539502 >> 172.21.15.5:0/1496539502 conn(0x7ff7e8077ca0 msgr2=0x7ff7e80780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/1496539502 shutdown_connections 2023-09-23T04:56:54.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.040+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/1496539502 wait complete. 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.041+0000 7ff7ef9ff700 1 Processor -- start 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 -- start start 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8068e10 0x7ff7e8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e819e110 con 0x7ff7e80697d0 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e819e280 con 0x7ff7e8068e10 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ef9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e819e440 con 0x7ff7e8106f30 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ecf9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7edf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ecf9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43426/0 (socket says 172.21.15.5:43426) 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8068e10 0x7ff7e8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/3350980273 learned_addr learned my addr 172.21.15.5:0/3350980273 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:54.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.042+0000 7ff7edf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37656/0 (socket says 172.21.15.5:37656) 2023-09-23T04:56:54.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/3350980273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 msgr2=0x7ff7e819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/3350980273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8068e10 msgr2=0x7ff7e8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8068e10 0x7ff7e8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/3350980273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0009c40 con 0x7ff7e80697d0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7edf9c700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9a0 tx=0x7ff7d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7de7fc700 1 -- 172.21.15.5:0/3350980273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8009790 con 0x7ff7e80697d0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e40095f0 con 0x7ff7e80697d0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7de7fc700 1 -- 172.21.15.5:0/3350980273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7d8010430 con 0x7ff7e80697d0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.043+0000 7ff7de7fc700 1 -- 172.21.15.5:0/3350980273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8018680 con 0x7ff7e80697d0 2023-09-23T04:56:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.044+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e819e910 con 0x7ff7e80697d0 2023-09-23T04:56:54.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.047+0000 7ff7de7fc700 1 -- 172.21.15.5:0/3350980273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff7d80105a0 con 0x7ff7e80697d0 2023-09-23T04:56:54.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.048+0000 7ff7de7fc700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 0x7ff7d40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.048+0000 7ff7de7fc700 1 -- 172.21.15.5:0/3350980273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff7d8014070 con 0x7ff7e80697d0 2023-09-23T04:56:54.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.049+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 0x7ff7d40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.049+0000 7ff80323eb80 1 Processor -- start 2023-09-23T04:56:54.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.049+0000 7ff80323eb80 1 -- start start 2023-09-23T04:56:54.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.049+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 0x7ff7d40a3bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8078480 tx=0x7ff7e401a040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43d20 0x55d773f4e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 0x55d773f50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773eac460 con 0x55d773f43d20 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e9dc20 con 0x55d773f4ebd0 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff80323eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e9b540 con 0x55d773f43400 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37670/0 (socket says 172.21.15.5:37670) 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 -- 172.21.15.5:0/3056475088 learned_addr learned my addr 172.21.15.5:0/3056475088 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43d20 0x55d773f4e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7edf9c700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 0x55d773f50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 -- 172.21.15.5:0/3056475088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 msgr2=0x55d773f50fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 0x55d773f50fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 -- 172.21.15.5:0/3056475088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43d20 msgr2=0x55d773f4e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43d20 0x55d773f4e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7ed79b700 1 -- 172.21.15.5:0/3056475088 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773ef98d0 con 0x55d773f43400 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.050+0000 7ff7edf9c700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 0x55d773f50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:56:54.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.051+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004800 tx=0x7ff7e401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42b7b519cac7b0e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.051+0000 7ff7d37fe700 1 -- 172.21.15.5:0/3056475088 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e402a070 con 0x55d773f43400 2023-09-23T04:56:54.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.051+0000 7ff7d37fe700 1 -- 172.21.15.5:0/3056475088 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e401e040 con 0x55d773f43400 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.051+0000 7ff7d37fe700 1 -- 172.21.15.5:0/3056475088 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400f840 con 0x55d773f43400 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- 172.21.15.5:0/3056475088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 msgr2=0x55d773f437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004800 tx=0x7ff7e401f000 comp rx=0 tx=0).stop 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- 172.21.15.5:0/3056475088 shutdown_connections 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f4ebd0 0x55d773f50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f437e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 --2- 172.21.15.5:0/3056475088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43d20 0x55d773f4e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- 172.21.15.5:0/3056475088 >> 172.21.15.5:0/3056475088 conn(0x55d773eb8260 msgr2=0x55d773f3f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- 172.21.15.5:0/3056475088 shutdown_connections 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- 172.21.15.5:0/3056475088 wait complete. 2023-09-23T04:56:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.052+0000 7ff80323eb80 1 -- start start 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.053+0000 7ff80323eb80 1 Processor -- start 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773f4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f4be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e96e30 con 0x55d773f4ebd0 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e33710 con 0x55d773f43d20 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773e333b0 con 0x55d773f43400 2023-09-23T04:56:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff80323eb80 10 client.0 _ll_register_callbacks cb 0x55d773c90800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37672/0 (socket says 172.21.15.5:37672) 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff7ed79b700 1 -- 172.21.15.5:0/2920782388 learned_addr learned my addr 172.21.15.5:0/2920782388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773f4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.054+0000 7ff7edf9c700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f4be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 -- 172.21.15.5:0/2920782388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 msgr2=0x55d773f4b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773f4b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 -- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 msgr2=0x55d773f4be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f4be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 -- 172.21.15.5:0/2920782388 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e810e640 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773f4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40098e0 tx=0x7ff7e400fdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5effaabcca4779ad server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e402a040 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d773f37250 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff7e400f870 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e403f830 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.055+0000 7ff80323eb80 10 client.27466 fetch_fsmap learned FSMap version 49 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d773cc1de0 con 0x55d773f43400 2023-09-23T04:56:54.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d773f37250 con 0x55d773f43400 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 49) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff7e4036070 con 0x55d773f43400 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 10 client.27466 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 20 client.27466 populate_metadata read hostname 'smithi005' 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773f5e0b0 0x55d773f604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 4 client.27466 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d773e8fe10 con 0x55d773f5e0b0 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.056+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773f5e0b0 0x55d773f604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.057+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773f5e0b0 0x55d773f604a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ec571828e43310 server_cookie=f589a0ec672557dc in_seq=0 out_seq=0 2023-09-23T04:56:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.057+0000 7ff7d0ff9700 10 client.27466 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:54.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/2920782388 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d773e8fe10 con 0x55d773f5e0b0 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff7d0ff9700 10 client.27466 handle_command_reply: tid=0 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 client.27466 shutdown 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 msgr2=0x55d773f4e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40098e0 tx=0x7ff7e400fdb0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 shutdown_connections 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773f5e0b0 0x55d773f604a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f4be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43400 0x55d773f4e5b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff7d0ff9700 0 client.27466 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 --2- 172.21.15.5:0/2920782388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773f4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 >> 172.21.15.5:0/2920782388 conn(0x55d773eb8260 msgr2=0x55d773f3f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 shutdown_connections 2023-09-23T04:56:54.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.058+0000 7ff80323eb80 1 -- 172.21.15.5:0/2920782388 wait complete. 2023-09-23T04:56:54.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.059+0000 7ff80323eb80 20 client.27466 trim_cache size 0 max 16384 2023-09-23T04:56:54.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.076+0000 7ff80323eb80 1 Processor -- start 2023-09-23T04:56:54.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.076+0000 7ff80323eb80 1 -- start start 2023-09-23T04:56:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43400 0x55d773f556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773d17920 con 0x55d773f4ebd0 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773f405b0 con 0x55d773f43400 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff80323eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773f37140 con 0x55d773f43d20 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7edf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7edf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f48ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43458/0 (socket says 172.21.15.5:43458) 2023-09-23T04:56:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37686/0 (socket says 172.21.15.5:37686) 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7edf9c700 1 -- 172.21.15.5:0/1648932530 learned_addr learned my addr 172.21.15.5:0/1648932530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43400 0x55d773f556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/1648932530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43400 msgr2=0x55d773f556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43400 0x55d773f556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/1648932530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 msgr2=0x55d773f48ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f48ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.077+0000 7ff7ecf9a700 1 -- 172.21.15.5:0/1648932530 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d773caac60 con 0x55d773f43d20 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80cac10 tx=0x7ff7d8010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149978437ce7a4cb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff7d3fff700 1 -- 172.21.15.5:0/1648932530 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d809e580 con 0x55d773f43d20 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff7d3fff700 1 -- 172.21.15.5:0/1648932530 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7d80a27c0 con 0x55d773f43d20 2023-09-23T04:56:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff7d3fff700 1 -- 172.21.15.5:0/1648932530 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8099920 con 0x55d773f43d20 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff80323eb80 1 -- 172.21.15.5:0/1648932530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 msgr2=0x55d773f45ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.078+0000 7ff80323eb80 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80cac10 tx=0x7ff7d8010c80 comp rx=0 tx=0).stop 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 -- 172.21.15.5:0/1648932530 shutdown_connections 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f4ebd0 0x55d773f48ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43400 0x55d773f556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 --2- 172.21.15.5:0/1648932530 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f43d20 0x55d773f45ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 -- 172.21.15.5:0/1648932530 >> 172.21.15.5:0/1648932530 conn(0x55d773eb8260 msgr2=0x55d773f453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 -- 172.21.15.5:0/1648932530 shutdown_connections 2023-09-23T04:56:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 -- 172.21.15.5:0/1648932530 wait complete. 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 -- start start 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.079+0000 7ff80323eb80 1 Processor -- start 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773fab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f4ebd0 0x55d773fa5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773fa64b0 con 0x55d773f43400 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773fa6620 con 0x55d773f43d20 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d773fa67e0 con 0x55d773f4ebd0 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff80323eb80 10 client.0 _ll_register_callbacks cb 0x7ff7d80967f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff7ed79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43470/0 (socket says 172.21.15.5:43470) 2023-09-23T04:56:54.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff7ed79b700 1 -- 172.21.15.5:0/112674333 learned_addr learned my addr 172.21.15.5:0/112674333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773fab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.080+0000 7ff7edf9c700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f4ebd0 0x55d773fa5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 -- 172.21.15.5:0/112674333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f4ebd0 msgr2=0x55d773fa5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f4ebd0 0x55d773fa5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 -- 172.21.15.5:0/112674333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 msgr2=0x55d773fab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773fab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 -- 172.21.15.5:0/112674333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e819e910 con 0x55d773f43400 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7ed79b700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004800 tx=0x7ff7e4024b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b180422ff117556 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e402a030 con 0x55d773f43400 2023-09-23T04:56:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7e401e040 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.081+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff7e402f670 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e400be50 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 10 client.17427 fetch_fsmap learned FSMap version 49 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d773ce8250 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e401e040 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 49) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff7e4036050 con 0x55d773f43400 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 10 client.17427 fetch_fsmap finished waiting for FSMap version 49 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 20 client.17427 populate_metadata read hostname 'smithi005' 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773fb2e30 0x55d773fb5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 4 client.17427 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d773e8fb30 con 0x55d773fb2e30 2023-09-23T04:56:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.082+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773fb2e30 0x55d773fb5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:56:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff7d0ff9700 10 client.17427 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff7ecf9a700 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773fb2e30 0x55d773fb5250 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92db4f922d1f1862 server_cookie=a05194291668807d in_seq=0 out_seq=0 2023-09-23T04:56:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff7d0ff9700 1 -- 172.21.15.5:0/112674333 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d773e8fb30 con 0x55d773fb2e30 2023-09-23T04:56:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff7d0ff9700 10 client.17427 handle_command_reply: tid=0 2023-09-23T04:56:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff80323eb80 1 client.17427 shutdown 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 msgr2=0x55d773fab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.083+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4004800 tx=0x7ff7e4024b00 comp rx=0 tx=0).stop 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 shutdown_connections 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d773fb2e30 0x55d773fb5250 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d773f4ebd0 0x55d773fa5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d773f43400 0x55d773fab430 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff7d0ff9700 0 client.17427 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 --2- 172.21.15.5:0/112674333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d773f43d20 0x55d773fab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 >> 172.21.15.5:0/112674333 conn(0x55d773eb8260 msgr2=0x55d773f45c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 shutdown_connections 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 1 -- 172.21.15.5:0/112674333 wait complete. 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.084+0000 7ff80323eb80 20 client.17427 trim_cache size 0 max 16384 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:56:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:56:54.091 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 msgr2=0x7ff7d40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 0x7ff7d40a3bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8078480 tx=0x7ff7e401a040 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 msgr2=0x7ff7e8199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9a0 tx=0x7ff7d800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 shutdown_connections 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff7d40a1710 0x7ff7d40a3bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff7e8106f30 0x7ff7e819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7e8068e10 0x7ff7e8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 --2- 172.21.15.5:0/3350980273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e80697d0 0x7ff7e8199780 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 >> 172.21.15.5:0/3350980273 conn(0x7ff7e8077ca0 msgr2=0x7ff7e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:56:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 shutdown_connections 2023-09-23T04:56:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:56:54.085+0000 7ff7ef9ff700 1 -- 172.21.15.5:0/3350980273 wait complete. 2023-09-23T04:56:54.111 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:56:54.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 548762 4.42 MB/sec execute 450 sec latency 2178.198 ms 2023-09-23T04:56:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:54 smithi005 ceph-mon[97638]: pgmap v330: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-09-23T04:56:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:54 smithi118 ceph-mon[134297]: pgmap v330: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-09-23T04:56:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:54 smithi069 ceph-mon[133931]: pgmap v330: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-09-23T04:56:55.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 549827 4.43 MB/sec execute 451 sec latency 2695.075 ms 2023-09-23T04:56:56.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 551138 4.44 MB/sec execute 452 sec latency 12.180 ms 2023-09-23T04:56:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:56 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:56 smithi005 ceph-mon[97638]: pgmap v331: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2023-09-23T04:56:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:56 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:56 smithi118 ceph-mon[134297]: pgmap v331: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2023-09-23T04:56:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:56 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:56:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:56 smithi069 ceph-mon[133931]: pgmap v331: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 185 op/s 2023-09-23T04:56:57.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 552305 4.44 MB/sec execute 453 sec latency 118.906 ms 2023-09-23T04:56:58.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 552305 4.43 MB/sec execute 454 sec latency 1118.973 ms 2023-09-23T04:56:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:56:58 smithi005 ceph-mon[97638]: pgmap v332: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-09-23T04:56:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:56:58 smithi118 ceph-mon[134297]: pgmap v332: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-09-23T04:56:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:56:58 smithi069 ceph-mon[133931]: pgmap v332: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 231 op/s 2023-09-23T04:56:59.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 552305 4.42 MB/sec execute 455 sec latency 2119.034 ms 2023-09-23T04:57:00.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 552305 4.41 MB/sec execute 456 sec latency 3119.095 ms 2023-09-23T04:57:01.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:01 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:01.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:01 smithi005 ceph-mon[97638]: pgmap v333: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 177 op/s 2023-09-23T04:57:01.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:01 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:01.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:01 smithi118 ceph-mon[134297]: pgmap v333: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 177 op/s 2023-09-23T04:57:01.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:01 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:01.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:01 smithi069 ceph-mon[133931]: pgmap v333: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 177 op/s 2023-09-23T04:57:01.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 552423 4.40 MB/sec execute 457 sec latency 4053.347 ms 2023-09-23T04:57:02.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 553016 4.39 MB/sec execute 458 sec latency 60.526 ms 2023-09-23T04:57:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:02 smithi005 ceph-mon[97638]: pgmap v334: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 194 op/s 2023-09-23T04:57:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:02 smithi118 ceph-mon[134297]: pgmap v334: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 194 op/s 2023-09-23T04:57:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:02 smithi069 ceph-mon[133931]: pgmap v334: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 194 op/s 2023-09-23T04:57:03.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 554541 4.41 MB/sec execute 459 sec latency 11.866 ms 2023-09-23T04:57:04.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555852 4.41 MB/sec execute 460 sec latency 16.988 ms 2023-09-23T04:57:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:04 smithi005 ceph-mon[97638]: pgmap v335: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2023-09-23T04:57:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:04 smithi118 ceph-mon[134297]: pgmap v335: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2023-09-23T04:57:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:04 smithi069 ceph-mon[133931]: pgmap v335: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2023-09-23T04:57:05.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555852 4.40 MB/sec execute 461 sec latency 1004.544 ms 2023-09-23T04:57:06.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555986 4.39 MB/sec execute 462 sec latency 1937.579 ms 2023-09-23T04:57:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:06 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:06 smithi005 ceph-mon[97638]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T04:57:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:06 smithi005 ceph-mon[97638]: pgmap v337: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 743 KiB/s wr, 112 op/s 2023-09-23T04:57:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:06 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:06 smithi118 ceph-mon[134297]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T04:57:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:06 smithi118 ceph-mon[134297]: pgmap v337: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 743 KiB/s wr, 112 op/s 2023-09-23T04:57:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:06 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:06 smithi069 ceph-mon[133931]: osdmap e101: 12 total, 12 up, 12 in 2023-09-23T04:57:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:06 smithi069 ceph-mon[133931]: pgmap v337: 129 pgs: 129 active+clean; 426 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 743 KiB/s wr, 112 op/s 2023-09-23T04:57:07.964 INFO:tasks.workunit.client.0.smithi005.stdout: 1 557941 4.41 MB/sec execute 463 sec latency 14.165 ms 2023-09-23T04:57:08.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 558957 4.41 MB/sec execute 464 sec latency 37.857 ms 2023-09-23T04:57:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:08 smithi005 ceph-mon[97638]: pgmap v338: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 875 KiB/s wr, 120 op/s 2023-09-23T04:57:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:08 smithi118 ceph-mon[134297]: pgmap v338: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 875 KiB/s wr, 120 op/s 2023-09-23T04:57:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:08 smithi069 ceph-mon[133931]: pgmap v338: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 875 KiB/s wr, 120 op/s 2023-09-23T04:57:09.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 559397 4.41 MB/sec execute 465 sec latency 694.619 ms 2023-09-23T04:57:10.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 559397 4.40 MB/sec execute 466 sec latency 1694.670 ms 2023-09-23T04:57:11.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 559531 4.39 MB/sec execute 467 sec latency 2631.079 ms 2023-09-23T04:57:11.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:11 smithi069 ceph-mon[133931]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T04:57:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:11 smithi005 ceph-mon[97638]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T04:57:12.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:11 smithi118 ceph-mon[134297]: osdmap e102: 12 total, 12 up, 12 in 2023-09-23T04:57:12.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 561504 4.41 MB/sec execute 468 sec latency 14.022 ms 2023-09-23T04:57:12.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:12 smithi069 ceph-mon[133931]: pgmap v340: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 965 KiB/s wr, 124 op/s 2023-09-23T04:57:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:12 smithi005 ceph-mon[97638]: pgmap v340: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 965 KiB/s wr, 124 op/s 2023-09-23T04:57:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:12 smithi118 ceph-mon[134297]: pgmap v340: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 965 KiB/s wr, 124 op/s 2023-09-23T04:57:13.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 562760 4.41 MB/sec execute 469 sec latency 15.459 ms 2023-09-23T04:57:14.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 562946 4.40 MB/sec execute 470 sec latency 845.027 ms 2023-09-23T04:57:14.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:14 smithi069 ceph-mon[133931]: pgmap v341: 129 pgs: 129 active+clean; 433 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-09-23T04:57:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:14 smithi005 ceph-mon[97638]: pgmap v341: 129 pgs: 129 active+clean; 433 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-09-23T04:57:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:14 smithi118 ceph-mon[134297]: pgmap v341: 129 pgs: 129 active+clean; 433 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 136 op/s 2023-09-23T04:57:15.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 564013 4.40 MB/sec execute 471 sec latency 1362.462 ms 2023-09-23T04:57:16.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 565403 4.41 MB/sec execute 472 sec latency 13.686 ms 2023-09-23T04:57:16.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:16 smithi069 ceph-mon[133931]: pgmap v342: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-09-23T04:57:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:16 smithi005 ceph-mon[97638]: pgmap v342: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-09-23T04:57:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:16 smithi118 ceph-mon[134297]: pgmap v342: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-09-23T04:57:17.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566486 4.41 MB/sec execute 473 sec latency 307.896 ms 2023-09-23T04:57:18.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566486 4.40 MB/sec execute 474 sec latency 1307.938 ms 2023-09-23T04:57:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:18 smithi069 ceph-mon[133931]: pgmap v343: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-09-23T04:57:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:18 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:18 smithi005 ceph-mon[97638]: pgmap v343: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-09-23T04:57:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:18 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:18 smithi118 ceph-mon[134297]: pgmap v343: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2023-09-23T04:57:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:18 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:19.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566486 4.39 MB/sec execute 475 sec latency 2307.982 ms 2023-09-23T04:57:20.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566486 4.38 MB/sec execute 476 sec latency 3308.027 ms 2023-09-23T04:57:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:20 smithi069 ceph-mon[133931]: pgmap v344: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-09-23T04:57:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:20 smithi005 ceph-mon[97638]: pgmap v344: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-09-23T04:57:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:20 smithi118 ceph-mon[134297]: pgmap v344: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2023-09-23T04:57:21.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566486 4.37 MB/sec execute 477 sec latency 4308.122 ms 2023-09-23T04:57:22.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 567776 4.38 MB/sec execute 478 sec latency 4469.513 ms 2023-09-23T04:57:22.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:22 smithi069 ceph-mon[133931]: pgmap v345: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-09-23T04:57:23.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:22 smithi005 ceph-mon[97638]: pgmap v345: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-09-23T04:57:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:22 smithi118 ceph-mon[134297]: pgmap v345: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 133 op/s 2023-09-23T04:57:23.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 568790 4.38 MB/sec execute 479 sec latency 23.562 ms 2023-09-23T04:57:24.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.389+0000 7f74b11ff700 1 -- 172.21.15.5:0/738261847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac069a30 msgr2=0x7f74ac105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 --2- 172.21.15.5:0/738261847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac069a30 0x7f74ac105af0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f74a00098f0 tx=0x7f74a0009c00 comp rx=0 tx=0).stop 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 -- 172.21.15.5:0/738261847 shutdown_connections 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 --2- 172.21.15.5:0/738261847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 0x7f74ac10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 --2- 172.21.15.5:0/738261847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 --2- 172.21.15.5:0/738261847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac069a30 0x7f74ac105af0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 -- 172.21.15.5:0/738261847 >> 172.21.15.5:0/738261847 conn(0x7f74ac077ca0 msgr2=0x7f74ac0780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 -- 172.21.15.5:0/738261847 shutdown_connections 2023-09-23T04:57:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.390+0000 7f74b11ff700 1 -- 172.21.15.5:0/738261847 wait complete. 2023-09-23T04:57:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.391+0000 7f74b11ff700 1 Processor -- start 2023-09-23T04:57:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 -- start start 2023-09-23T04:57:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 0x7f74ac199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac19e630 con 0x7f74ac106220 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac19e7a0 con 0x7f74ac069080 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.392+0000 7f74b11ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac19e960 con 0x7f74ac19dbb0 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:51444/0 (socket says 172.21.15.5:51444) 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55232/0 (socket says 172.21.15.5:55232) 2023-09-23T04:57:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 -- 172.21.15.5:0/3172902948 learned_addr learned my addr 172.21.15.5:0/3172902948 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74aa59c700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 0x7f74ac199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 -- 172.21.15.5:0/3172902948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 msgr2=0x7f74ac199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 -- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 msgr2=0x7f74ac199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 0x7f74ac199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 -- 172.21.15.5:0/3172902948 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a0009520 con 0x7f74ac19dbb0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.393+0000 7f74ab59e700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74a400efc0 tx=0x7f74a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.394+0000 7f7493fff700 1 -- 172.21.15.5:0/3172902948 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4018070 con 0x7f74ac19dbb0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.394+0000 7f7493fff700 1 -- 172.21.15.5:0/3172902948 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a400f040 con 0x7f74ac19dbb0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.394+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac1a5e60 con 0x7f74ac19dbb0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.394+0000 7f7493fff700 1 -- 172.21.15.5:0/3172902948 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400cc20 con 0x7f74ac19dbb0 2023-09-23T04:57:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.394+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac1a62e0 con 0x7f74ac19dbb0 2023-09-23T04:57:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.399+0000 7f7493fff700 1 -- 172.21.15.5:0/3172902948 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f74a4003a00 con 0x7f74ac19dbb0 2023-09-23T04:57:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.400+0000 7f74c4ba0b80 1 Processor -- start 2023-09-23T04:57:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 -- start start 2023-09-23T04:57:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ccbd0 0x55ac557cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5572a460 con 0x55ac557ccbd0 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5571bc20 con 0x55ac557c1d20 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.401+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac55719540 con 0x55ac557c1400 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55246/0 (socket says 172.21.15.5:55246) 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 -- 172.21.15.5:0/1342978564 learned_addr learned my addr 172.21.15.5:0/1342978564 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ccbd0 0x55ac557cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 -- 172.21.15.5:0/1342978564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 msgr2=0x55ac557c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 -- 172.21.15.5:0/1342978564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ccbd0 msgr2=0x55ac557cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ccbd0 0x55ac557cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aad9d700 1 -- 172.21.15.5:0/1342978564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac557778d0 con 0x55ac557c1d20 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.402+0000 7f7493fff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 0x7f74940a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f749c009d60 tx=0x7f749c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a5685a7239b21 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f7493fff700 1 -- 172.21.15.5:0/3172902948 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f74a40cd060 con 0x7f74ac19dbb0 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f74aa59c700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 0x7f74940a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f74917fa700 1 -- 172.21.15.5:0/1342978564 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c019630 con 0x55ac557c1d20 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f74917fa700 1 -- 172.21.15.5:0/1342978564 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f749c019c50 con 0x55ac557c1d20 2023-09-23T04:57:24.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.403+0000 7f74917fa700 1 -- 172.21.15.5:0/1342978564 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00eb70 con 0x55ac557c1d20 2023-09-23T04:57:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1342978564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 msgr2=0x55ac557cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74aa59c700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 0x7f74940a3c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009fd0 tx=0x7f74a0005dc0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f749c009d60 tx=0x7f749c011720 comp rx=0 tx=0).stop 2023-09-23T04:57:24.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1342978564 shutdown_connections 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ccbd0 0x55ac557cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1342978564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1d20 0x55ac557cc690 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f749c009d60 tx=0x7f749c011720 comp rx=0 tx=0).stop 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1342978564 >> 172.21.15.5:0/1342978564 conn(0x55ac55736260 msgr2=0x55ac557bd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1342978564 shutdown_connections 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.404+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1342978564 wait complete. 2023-09-23T04:57:24.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.405+0000 7f74c4ba0b80 1 -- start start 2023-09-23T04:57:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.405+0000 7f74c4ba0b80 1 Processor -- start 2023-09-23T04:57:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac55714e30 con 0x55ac557ca550 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac556b1710 con 0x55ac557c1400 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac556b13b0 con 0x55ac557ccbd0 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74c4ba0b80 10 client.0 _ll_register_callbacks cb 0x55ac5550e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74aa59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.406+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55248/0 (socket says 172.21.15.5:55248) 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 -- 172.21.15.5:0/88116436 learned_addr learned my addr 172.21.15.5:0/88116436 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74ab59e700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 -- 172.21.15.5:0/88116436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 msgr2=0x55ac557ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 -- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 msgr2=0x55ac557d8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 -- 172.21.15.5:0/88116436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c0099c0 con 0x55ac557c1400 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f74aad9d700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f749c006010 tx=0x7f749c019ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc2ccfcf53578bb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.407+0000 7f748eff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c005450 con 0x55ac557c1400 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f749c019dc0 con 0x55ac557c1400 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac557b5250 con 0x55ac557c1400 2023-09-23T04:57:24.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749c0055b0 con 0x55ac557c1400 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f74c4ba0b80 10 client.27428 fetch_fsmap learned FSMap version 53 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55ac5553fde0 con 0x55ac557c1400 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac557b5250 con 0x55ac557c1400 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.408+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 53) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f749c029020 con 0x55ac557c1400 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 10 client.27428 fetch_fsmap finished waiting for FSMap version 53 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 20 client.27428 populate_metadata read hostname 'smithi005' 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac557dcb00 0x55ac557def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 4 client.27428 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:57:24.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac5570de10 con 0x55ac557dcb00 2023-09-23T04:57:24.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.409+0000 7f74aa59c700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac557dcb00 0x55ac557def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.410+0000 7f74aa59c700 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac557dcb00 0x55ac557def20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4193ddc69f3bf8be server_cookie=fa03125f574e00b1 in_seq=0 out_seq=0 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.410+0000 7f748eff5700 10 client.27428 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.410+0000 7f748eff5700 1 -- 172.21.15.5:0/88116436 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac5570de10 con 0x55ac557dcb00 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.410+0000 7f748eff5700 10 client.27428 handle_command_reply: tid=0 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.410+0000 7f74c4ba0b80 1 client.27428 shutdown 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 msgr2=0x55ac557c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f749c006010 tx=0x7f749c019ae0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 shutdown_connections 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac557dcb00 0x55ac557def20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557c9ad0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/88116436 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 >> 172.21.15.5:0/88116436 conn(0x55ac55736260 msgr2=0x55ac557bd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f748eff5700 0 client.27428 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 shutdown_connections 2023-09-23T04:57:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/88116436 wait complete. 2023-09-23T04:57:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.411+0000 7f74c4ba0b80 20 client.27428 trim_cache size 0 max 16384 2023-09-23T04:57:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 Processor -- start 2023-09-23T04:57:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 -- start start 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac55595920 con 0x55ac557ca550 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.439+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac557c8070 con 0x55ac557c1400 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac557c58e0 con 0x55ac557ccbd0 2023-09-23T04:57:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55252/0 (socket says 172.21.15.5:55252) 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 -- 172.21.15.5:0/2558917555 learned_addr learned my addr 172.21.15.5:0/2558917555 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74ab59e700 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aa59c700 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 -- 172.21.15.5:0/2558917555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 msgr2=0x55ac557c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 -- 172.21.15.5:0/2558917555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 msgr2=0x55ac557c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 -- 172.21.15.5:0/2558917555 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac55528c60 con 0x55ac557c1400 2023-09-23T04:57:24.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.440+0000 7f74aad9d700 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f749c019c20 tx=0x7f749c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ca42870c861e47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.441+0000 7f7491ffb700 1 -- 172.21.15.5:0/2558917555 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c021360 con 0x55ac557c1400 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.441+0000 7f7491ffb700 1 -- 172.21.15.5:0/2558917555 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f749c005710 con 0x55ac557c1400 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.441+0000 7f7491ffb700 1 -- 172.21.15.5:0/2558917555 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0038e0 con 0x55ac557c1400 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.441+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/2558917555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 msgr2=0x55ac557d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.441+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f749c019c20 tx=0x7f749c001e50 comp rx=0 tx=0).stop 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/2558917555 shutdown_connections 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557ccbd0 0x55ac557c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557c1400 0x55ac557d8700 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/2558917555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/2558917555 >> 172.21.15.5:0/2558917555 conn(0x55ac55736260 msgr2=0x55ac557c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/2558917555 shutdown_connections 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/2558917555 wait complete. 2023-09-23T04:57:24.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.442+0000 7f74c4ba0b80 1 -- start start 2023-09-23T04:57:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 Processor -- start 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 0x55ac5582f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5582feb0 con 0x55ac557ca550 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5582fff0 con 0x55ac557ccbd0 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac55830130 con 0x55ac557c1400 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.443+0000 7f74c4ba0b80 10 client.0 _ll_register_callbacks cb 0x7f749c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:51472/0 (socket says 172.21.15.5:51472) 2023-09-23T04:57:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 -- 172.21.15.5:0/1184002291 learned_addr learned my addr 172.21.15.5:0/1184002291 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74ab59e700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 0x55ac5582f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 -- 172.21.15.5:0/1184002291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 msgr2=0x55ac5582f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 0x55ac5582f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 -- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 msgr2=0x55ac557d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 -- 172.21.15.5:0/1184002291 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac5553fde0 con 0x55ac557c1400 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74ab59e700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 0x55ac5582f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f74aad9d700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f749c009350 tx=0x7f749c019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3879e0c0b13e7805 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.444+0000 7f748eff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:57:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c005450 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f749c019c50 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f749c00e460 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749c0055b0 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f74c4ba0b80 10 client.27490 fetch_fsmap learned FSMap version 53 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55ac55566250 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.445+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f749c019c50 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 53) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f749c029070 con 0x55ac557c1400 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 10 client.27490 fetch_fsmap finished waiting for FSMap version 53 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 10 client.27490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 20 client.27490 populate_metadata read hostname 'smithi005' 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac558313c0 0x55ac558337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 4 client.27490 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:57:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac5570db30 con 0x55ac558313c0 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.446+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac558313c0 0x55ac558337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.447+0000 7f748eff5700 10 client.27490 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.447+0000 7f74aa59c700 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac558313c0 0x55ac558337e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5570dd29f3cdca61 server_cookie=801283667e997c65 in_seq=0 out_seq=0 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.447+0000 7f748eff5700 1 -- 172.21.15.5:0/1184002291 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ac5570db30 con 0x55ac558313c0 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.447+0000 7f748eff5700 10 client.27490 handle_command_reply: tid=0 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.447+0000 7f74c4ba0b80 1 client.27490 shutdown 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 msgr2=0x55ac557d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f749c009350 tx=0x7f749c019740 comp rx=0 tx=0).stop 2023-09-23T04:57:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 shutdown_connections 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ac558313c0 0x55ac558337e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ac557ccbd0 0x55ac5582f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f748eff5700 0 client.27490 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ac557c1400 0x55ac557d1470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 --2- 172.21.15.5:0/1184002291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac557ca550 0x55ac557d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 >> 172.21.15.5:0/1184002291 conn(0x55ac55736260 msgr2=0x55ac557bd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 shutdown_connections 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 1 -- 172.21.15.5:0/1184002291 wait complete. 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.448+0000 7f74c4ba0b80 20 client.27490 trim_cache size 0 max 16384 2023-09-23T04:57:24.451 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:57:24.452 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:57:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 msgr2=0x7f74940a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 0x7f74940a3c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009fd0 tx=0x7f74a0005dc0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 msgr2=0x7f74ac19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74a400efc0 tx=0x7f74a400c3a0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 shutdown_connections 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f74940a1790 0x7f74940a3c50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f74ac19dbb0 0x7f74ac19e030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ac069080 0x7f74ac199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 --2- 172.21.15.5:0/3172902948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74ac106220 0x7f74ac199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 >> 172.21.15.5:0/3172902948 conn(0x7f74ac077ca0 msgr2=0x7f74ac10c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 shutdown_connections 2023-09-23T04:57:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:24.449+0000 7f74b11ff700 1 -- 172.21.15.5:0/3172902948 wait complete. 2023-09-23T04:57:24.471 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:57:24.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:24 smithi005 ceph-mon[97638]: pgmap v346: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-09-23T04:57:24.965 INFO:tasks.workunit.client.0.smithi005.stdout: 1 570036 4.39 MB/sec execute 480 sec latency 37.388 ms 2023-09-23T04:57:24.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:24 smithi069 ceph-mon[133931]: pgmap v346: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-09-23T04:57:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:24 smithi118 ceph-mon[134297]: pgmap v346: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2023-09-23T04:57:25.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 570036 4.38 MB/sec execute 481 sec latency 1037.431 ms 2023-09-23T04:57:26.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 570170 4.37 MB/sec execute 482 sec latency 1972.955 ms 2023-09-23T04:57:26.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:26 smithi069 ceph-mon[133931]: pgmap v347: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2023-09-23T04:57:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:26 smithi005 ceph-mon[97638]: pgmap v347: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2023-09-23T04:57:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:26 smithi118 ceph-mon[134297]: pgmap v347: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 2.2 MiB/s wr, 124 op/s 2023-09-23T04:57:27.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 572113 4.39 MB/sec execute 483 sec latency 16.149 ms 2023-09-23T04:57:28.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 573380 4.39 MB/sec execute 484 sec latency 18.377 ms 2023-09-23T04:57:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:28 smithi069 ceph-mon[133931]: pgmap v348: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2023-09-23T04:57:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:28 smithi005 ceph-mon[97638]: pgmap v348: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2023-09-23T04:57:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:28 smithi118 ceph-mon[134297]: pgmap v348: 129 pgs: 129 active+clean; 454 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.1 MiB/s wr, 118 op/s 2023-09-23T04:57:29.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 573579 4.38 MB/sec execute 485 sec latency 852.529 ms 2023-09-23T04:57:30.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 575439 4.40 MB/sec execute 486 sec latency 965.062 ms 2023-09-23T04:57:30.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:30 smithi069 ceph-mon[133931]: pgmap v349: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-09-23T04:57:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:30 smithi005 ceph-mon[97638]: pgmap v349: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-09-23T04:57:31.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:30 smithi118 ceph-mon[134297]: pgmap v349: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-09-23T04:57:31.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 576492 4.40 MB/sec execute 487 sec latency 20.590 ms 2023-09-23T04:57:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:31 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:32.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:31 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:32.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:31 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:32.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 577126 4.39 MB/sec execute 488 sec latency 574.221 ms 2023-09-23T04:57:32.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:32 smithi069 ceph-mon[133931]: pgmap v350: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-09-23T04:57:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:32 smithi005 ceph-mon[97638]: pgmap v350: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-09-23T04:57:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:32 smithi118 ceph-mon[134297]: pgmap v350: 129 pgs: 129 active+clean; 447 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2023-09-23T04:57:33.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 577126 4.38 MB/sec execute 489 sec latency 1574.272 ms 2023-09-23T04:57:34.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 577126 4.38 MB/sec execute 490 sec latency 2574.335 ms 2023-09-23T04:57:34.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:34 smithi069 ceph-mon[133931]: pgmap v351: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2023-09-23T04:57:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:34 smithi005 ceph-mon[97638]: pgmap v351: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2023-09-23T04:57:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:34 smithi118 ceph-mon[134297]: pgmap v351: 129 pgs: 129 active+clean; 453 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 171 op/s 2023-09-23T04:57:35.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 578042 4.37 MB/sec execute 491 sec latency 3096.997 ms 2023-09-23T04:57:36.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 579449 4.39 MB/sec execute 492 sec latency 16.294 ms 2023-09-23T04:57:36.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:36 smithi069 ceph-mon[133931]: pgmap v352: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 157 op/s 2023-09-23T04:57:36.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:36 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:36 smithi005 ceph-mon[97638]: pgmap v352: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 157 op/s 2023-09-23T04:57:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:36 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:36 smithi118 ceph-mon[134297]: pgmap v352: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 157 op/s 2023-09-23T04:57:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:36 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:57:37.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 580669 4.39 MB/sec execute 493 sec latency 175.129 ms 2023-09-23T04:57:38.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 580669 4.38 MB/sec execute 494 sec latency 1175.176 ms 2023-09-23T04:57:38.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:38 smithi069 ceph-mon[133931]: pgmap v353: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-09-23T04:57:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:38 smithi005 ceph-mon[97638]: pgmap v353: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-09-23T04:57:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:38 smithi118 ceph-mon[134297]: pgmap v353: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-09-23T04:57:39.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 580669 4.37 MB/sec execute 495 sec latency 2175.244 ms 2023-09-23T04:57:40.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 580669 4.36 MB/sec execute 496 sec latency 3175.300 ms 2023-09-23T04:57:40.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:40 smithi069 ceph-mon[133931]: pgmap v354: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2023-09-23T04:57:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:40 smithi005 ceph-mon[97638]: pgmap v354: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2023-09-23T04:57:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:40 smithi118 ceph-mon[134297]: pgmap v354: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2023-09-23T04:57:41.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 580741 4.35 MB/sec execute 497 sec latency 4113.770 ms 2023-09-23T04:57:41.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:41 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:57:42.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:41 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:57:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:41 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:57:42.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 582610 4.37 MB/sec execute 498 sec latency 19.379 ms 2023-09-23T04:57:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:42 smithi069 ceph-mon[133931]: pgmap v355: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-09-23T04:57:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:42 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:42 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:42 smithi005 ceph-mon[97638]: pgmap v355: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-09-23T04:57:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:42 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:43.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:42 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:42 smithi118 ceph-mon[134297]: pgmap v355: 129 pgs: 129 active+clean; 459 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2023-09-23T04:57:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:42 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:43.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:42 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:43.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 583506 4.36 MB/sec execute 499 sec latency 30.209 ms 2023-09-23T04:57:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:57:44.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:57:44.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:57:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:57:44.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:44.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:57:44.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:57:44.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:57:44.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 584222 4.36 MB/sec execute 500 sec latency 565.014 ms 2023-09-23T04:57:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:44 smithi005 ceph-mon[97638]: pgmap v356: 129 pgs: 129 active+clean; 460 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:57:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:44 smithi118 ceph-mon[134297]: pgmap v356: 129 pgs: 129 active+clean; 460 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:57:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:44 smithi069 ceph-mon[133931]: pgmap v356: 129 pgs: 129 active+clean; 460 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-09-23T04:57:45.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 584222 4.36 MB/sec execute 501 sec latency 1565.057 ms 2023-09-23T04:57:46.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 584376 4.35 MB/sec execute 502 sec latency 2495.691 ms 2023-09-23T04:57:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:46 smithi005 ceph-mon[97638]: pgmap v357: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 843 KiB/s wr, 85 op/s 2023-09-23T04:57:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:46 smithi118 ceph-mon[134297]: pgmap v357: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 843 KiB/s wr, 85 op/s 2023-09-23T04:57:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:46 smithi069 ceph-mon[133931]: pgmap v357: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 843 KiB/s wr, 85 op/s 2023-09-23T04:57:47.966 INFO:tasks.workunit.client.0.smithi005.stdout: 1 586212 4.36 MB/sec execute 503 sec latency 93.573 ms 2023-09-23T04:57:48.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 587279 4.36 MB/sec execute 504 sec latency 13.394 ms 2023-09-23T04:57:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:48 smithi005 ceph-mon[97638]: pgmap v358: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 703 KiB/s wr, 70 op/s 2023-09-23T04:57:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:48 smithi118 ceph-mon[134297]: pgmap v358: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 703 KiB/s wr, 70 op/s 2023-09-23T04:57:49.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:48 smithi069 ceph-mon[133931]: pgmap v358: 129 pgs: 129 active+clean; 461 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 703 KiB/s wr, 70 op/s 2023-09-23T04:57:49.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 587763 4.36 MB/sec execute 505 sec latency 705.832 ms 2023-09-23T04:57:50.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 589696 4.37 MB/sec execute 506 sec latency 816.641 ms 2023-09-23T04:57:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:50 smithi005 ceph-mon[97638]: pgmap v359: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-09-23T04:57:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:50 smithi118 ceph-mon[134297]: pgmap v359: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-09-23T04:57:51.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:50 smithi069 ceph-mon[133931]: pgmap v359: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2023-09-23T04:57:51.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 590724 4.37 MB/sec execute 507 sec latency 13.640 ms 2023-09-23T04:57:52.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 591312 4.37 MB/sec execute 508 sec latency 634.850 ms 2023-09-23T04:57:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:52 smithi005 ceph-mon[97638]: pgmap v360: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 625 KiB/s wr, 70 op/s 2023-09-23T04:57:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:52 smithi118 ceph-mon[134297]: pgmap v360: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 625 KiB/s wr, 70 op/s 2023-09-23T04:57:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:52 smithi069 ceph-mon[133931]: pgmap v360: 129 pgs: 129 active+clean; 466 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 625 KiB/s wr, 70 op/s 2023-09-23T04:57:53.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 591312 4.36 MB/sec execute 509 sec latency 1634.910 ms 2023-09-23T04:57:54.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 -- 172.21.15.5:0/3273278038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103cc0 msgr2=0x7fca94107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 --2- 172.21.15.5:0/3273278038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103cc0 0x7fca94107d00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca880098f0 tx=0x7fca88009c00 comp rx=0 tx=0).stop 2023-09-23T04:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 -- 172.21.15.5:0/3273278038 shutdown_connections 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 --2- 172.21.15.5:0/3273278038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca94108430 0x7fca9410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 --2- 172.21.15.5:0/3273278038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94103310 0x7fca941036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 --2- 172.21.15.5:0/3273278038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103cc0 0x7fca94107d00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 -- 172.21.15.5:0/3273278038 >> 172.21.15.5:0/3273278038 conn(0x7fca940feb50 msgr2=0x7fca94100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 -- 172.21.15.5:0/3273278038 shutdown_connections 2023-09-23T04:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.702+0000 7fca997ff700 1 -- 172.21.15.5:0/3273278038 wait complete. 2023-09-23T04:57:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.703+0000 7fca997ff700 1 Processor -- start 2023-09-23T04:57:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.703+0000 7fca997ff700 1 -- start start 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108430 0x7fca94199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca9419dcf0 0x7fca9419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419e770 con 0x7fca9419dcf0 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419e8e0 con 0x7fca94108430 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca997ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419eaa0 con 0x7fca94103310 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59924/0 (socket says 172.21.15.5:59924) 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 -- 172.21.15.5:0/4271095336 learned_addr learned my addr 172.21.15.5:0/4271095336 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca937fe700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca9419dcf0 0x7fca9419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca927fc700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108430 0x7fca94199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 -- 172.21.15.5:0/4271095336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108430 msgr2=0x7fca94199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108430 0x7fca94199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 -- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca9419dcf0 msgr2=0x7fca9419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca9419dcf0 0x7fca9419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.704+0000 7fca92ffd700 1 -- 172.21.15.5:0/4271095336 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88009520 con 0x7fca94103310 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca92ffd700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca8400e9a0 tx=0x7fca8400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca7bfff700 1 -- 172.21.15.5:0/4271095336 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84009790 con 0x7fca94103310 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca7bfff700 1 -- 172.21.15.5:0/4271095336 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca84010430 con 0x7fca94103310 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca7bfff700 1 -- 172.21.15.5:0/4271095336 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84018680 con 0x7fca94103310 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca941a5fa0 con 0x7fca94103310 2023-09-23T04:57:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.705+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca941a6370 con 0x7fca94103310 2023-09-23T04:57:54.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.709+0000 7fca7bfff700 1 -- 172.21.15.5:0/4271095336 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fca840105a0 con 0x7fca94103310 2023-09-23T04:57:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.710+0000 7fca7bfff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 0x7fca7c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.711+0000 7fca7bfff700 1 -- 172.21.15.5:0/4271095336 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fca84014070 con 0x7fca94103310 2023-09-23T04:57:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.711+0000 7fca927fc700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 0x7fca7c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.711+0000 7fcaad08bb80 1 Processor -- start 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.711+0000 7fcaad08bb80 1 -- start start 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.711+0000 7fca927fc700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 0x7fca7c0a3c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca88009fd0 tx=0x7fca88019040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fd0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fde7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2f31460 con 0x5630f2fdc3e0 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2f22c20 con 0x5630f2fd0c10 2023-09-23T04:57:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2f20540 con 0x5630f2fd1530 2023-09-23T04:57:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fca927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fca927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59938/0 (socket says 172.21.15.5:59938) 2023-09-23T04:57:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fca927fc700 1 -- 172.21.15.5:0/3320569351 learned_addr learned my addr 172.21.15.5:0/3320569351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fca92ffd700 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fd0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.712+0000 7fca937fe700 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fde7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 -- 172.21.15.5:0/3320569351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 msgr2=0x5630f2fd0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fd0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 -- 172.21.15.5:0/3320569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 msgr2=0x5630f2fde7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fde7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 -- 172.21.15.5:0/3320569351 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630f2f7e8d0 con 0x5630f2fd1530 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca927fc700 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca880094b0 tx=0x7fca8801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c7eeb510f1cd412 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca797fa700 1 -- 172.21.15.5:0/3320569351 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800f760 con 0x5630f2fd1530 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca797fa700 1 -- 172.21.15.5:0/3320569351 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8801d040 con 0x5630f2fd1530 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.713+0000 7fca797fa700 1 -- 172.21.15.5:0/3320569351 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88005610 con 0x5630f2fd1530 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 -- 172.21.15.5:0/3320569351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 msgr2=0x5630f2fdbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca880094b0 tx=0x7fca8801e000 comp rx=0 tx=0).stop 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 -- 172.21.15.5:0/3320569351 shutdown_connections 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fde7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fd0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/3320569351 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fdbea0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 -- 172.21.15.5:0/3320569351 >> 172.21.15.5:0/3320569351 conn(0x5630f2f3d260 msgr2=0x5630f2fc4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 -- 172.21.15.5:0/3320569351 shutdown_connections 2023-09-23T04:57:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.714+0000 7fcaad08bb80 1 -- 172.21.15.5:0/3320569351 wait complete. 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.715+0000 7fcaad08bb80 1 -- start start 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.715+0000 7fcaad08bb80 1 Processor -- start 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd1530 0x5630f2fd90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2f1be30 con 0x5630f2fd1530 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2eb8710 con 0x5630f2fdc3e0 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2eb83b0 con 0x5630f2fd0c10 2023-09-23T04:57:54.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fcaad08bb80 10 client.0 _ll_register_callbacks cb 0x5630f2d15800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59954/0 (socket says 172.21.15.5:59954) 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fca92ffd700 1 -- 172.21.15.5:0/94639874 learned_addr learned my addr 172.21.15.5:0/94639874 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fca927fc700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd1530 0x5630f2fd90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.716+0000 7fca937fe700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 -- 172.21.15.5:0/94639874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 msgr2=0x5630f2fd97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 -- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd1530 msgr2=0x5630f2fd90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd1530 0x5630f2fd90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 -- 172.21.15.5:0/94639874 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca94110820 con 0x5630f2fd0c10 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca937fe700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:57:54.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca92ffd700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fca84096870 tx=0x7fca84099dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b06de4753c7006 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840a2700 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fca8409a5d0 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630f2fbc250 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8409a790 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fcaad08bb80 10 client.27502 fetch_fsmap learned FSMap version 55 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.717+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5630f2d46de0 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630f2fbc250 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 55) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fca840a2270 con 0x5630f2fd0c10 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 10 client.27502 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T04:57:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 10 client.27502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:57:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 20 client.27502 populate_metadata read hostname 'smithi005' 2023-09-23T04:57:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f2febed0 0x5630f2fee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 4 client.27502 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:57:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630f2f14e10 con 0x5630f2febed0 2023-09-23T04:57:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.718+0000 7fca927fc700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f2febed0 0x5630f2fee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.719+0000 7fca927fc700 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f2febed0 0x5630f2fee2c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84c188fb69393dc server_cookie=212c7066545d54cf in_seq=0 out_seq=0 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.719+0000 7fca76ff5700 10 client.27502 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.719+0000 7fca76ff5700 1 -- 172.21.15.5:0/94639874 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5630f2f14e10 con 0x5630f2febed0 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.719+0000 7fca76ff5700 10 client.27502 handle_command_reply: tid=0 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.719+0000 7fcaad08bb80 1 client.27502 shutdown 2023-09-23T04:57:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 msgr2=0x5630f2fdbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fca84096870 tx=0x7fca84099dc0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 shutdown_connections 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f2febed0 0x5630f2fee2c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd0c10 0x5630f2fdbe30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fca76ff5700 0 client.27502 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/94639874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd1530 0x5630f2fd90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 >> 172.21.15.5:0/94639874 conn(0x5630f2f3d260 msgr2=0x5630f2fc4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 shutdown_connections 2023-09-23T04:57:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 1 -- 172.21.15.5:0/94639874 wait complete. 2023-09-23T04:57:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.720+0000 7fcaad08bb80 20 client.27502 trim_cache size 0 max 16384 2023-09-23T04:57:54.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.740+0000 7fcaad08bb80 1 Processor -- start 2023-09-23T04:57:54.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.740+0000 7fcaad08bb80 1 -- start start 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd0c10 0x5630f2fd88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 0x5630f2fdb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2d9c920 con 0x5630f2fd0c10 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2fc35b0 con 0x5630f2fd1530 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2fd4f80 con 0x5630f2fdc3e0 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd0c10 0x5630f2fd88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59968/0 (socket says 172.21.15.5:59968) 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 -- 172.21.15.5:0/1520935407 learned_addr learned my addr 172.21.15.5:0/1520935407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca927fc700 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 0x5630f2fdb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 -- 172.21.15.5:0/1520935407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 msgr2=0x5630f2fdb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 0x5630f2fdb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 -- 172.21.15.5:0/1520935407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd0c10 msgr2=0x5630f2fd88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd0c10 0x5630f2fd88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.741+0000 7fca937fe700 1 -- 172.21.15.5:0/1520935407 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630f2d2fc60 con 0x5630f2fdc3e0 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.742+0000 7fca927fc700 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 0x5630f2fdb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.742+0000 7fca937fe700 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0095d0 tx=0x7fca8c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b1b0d6c0dd831d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.742+0000 7fca79ffb700 1 -- 172.21.15.5:0/1520935407 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00e040 con 0x5630f2fdc3e0 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.742+0000 7fca79ffb700 1 -- 172.21.15.5:0/1520935407 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8c00f040 con 0x5630f2fdc3e0 2023-09-23T04:57:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.742+0000 7fca79ffb700 1 -- 172.21.15.5:0/1520935407 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c016a00 con 0x5630f2fdc3e0 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1520935407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 msgr2=0x5630f2fd63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0095d0 tx=0x7fca8c00b040 comp rx=0 tx=0).stop 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1520935407 shutdown_connections 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fdc3e0 0x5630f2fd63a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fd0c10 0x5630f2fd88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1520935407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd1530 0x5630f2fdb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1520935407 >> 172.21.15.5:0/1520935407 conn(0x5630f2f3d260 msgr2=0x5630f2fd4440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1520935407 shutdown_connections 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1520935407 wait complete. 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.743+0000 7fcaad08bb80 1 -- start start 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.744+0000 7fcaad08bb80 1 Processor -- start 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.744+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fe2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.744+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fcaad08bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fe1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2fe1970 con 0x5630f2fdc3e0 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2fe1ae0 con 0x5630f2fd0c10 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fcaad08bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630f2fe1ca0 con 0x5630f2fd1530 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fcaad08bb80 10 client.0 _ll_register_callbacks cb 0x7fca84099eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:59984/0 (socket says 172.21.15.5:59984) 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 -- 172.21.15.5:0/1525651985 learned_addr learned my addr 172.21.15.5:0/1525651985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca92ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fe2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca937fe700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fe1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 -- 172.21.15.5:0/1525651985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 msgr2=0x5630f2fe2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fe2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 -- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 msgr2=0x5630f2fe1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fe1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 -- 172.21.15.5:0/1525651985 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c009280 con 0x5630f2fd1530 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca937fe700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fe1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca927fc700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca880098f0 tx=0x7fca88009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b433bf8e11fdcf14 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.745+0000 7fca76ff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:57:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800f900 con 0x5630f2fd1530 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca8801d040 con 0x5630f2fd1530 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fca88005260 con 0x5630f2fd1530 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca88005420 con 0x5630f2fd1530 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fcaad08bb80 10 client.27508 fetch_fsmap learned FSMap version 55 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5630f2d6d250 con 0x5630f2fd1530 2023-09-23T04:57:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.746+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8801d040 con 0x5630f2fd1530 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 55) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fca88039040 con 0x5630f2fd1530 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 10 client.27508 fetch_fsmap finished waiting for FSMap version 55 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 20 client.27508 populate_metadata read hostname 'smithi005' 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f3040c30 0x5630f3043050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 4 client.27508 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:57:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630f2f14b30 con 0x5630f3040c30 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.747+0000 7fca92ffd700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f3040c30 0x5630f3043050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fca76ff5700 10 client.27508 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fca92ffd700 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f3040c30 0x5630f3043050 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75a3d672d858647 server_cookie=8e4e3392a7013d5c in_seq=0 out_seq=0 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fca76ff5700 1 -- 172.21.15.5:0/1525651985 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5630f2f14b30 con 0x5630f3040c30 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fca76ff5700 10 client.27508 handle_command_reply: tid=0 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 client.27508 shutdown 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 msgr2=0x5630f2fe0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca880098f0 tx=0x7fca88009840 comp rx=0 tx=0).stop 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 shutdown_connections 2023-09-23T04:57:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5630f3040c30 0x5630f3043050 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fca76ff5700 0 client.27508 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f2fdc3e0 0x5630f2fe1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630f2fd0c10 0x5630f2fe2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 --2- 172.21.15.5:0/1525651985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5630f2fd1530 0x5630f2fe0ef0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 >> 172.21.15.5:0/1525651985 conn(0x5630f2f3d260 msgr2=0x5630f2fc39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 shutdown_connections 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.748+0000 7fcaad08bb80 1 -- 172.21.15.5:0/1525651985 wait complete. 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fcaad08bb80 20 client.27508 trim_cache size 0 max 16384 2023-09-23T04:57:54.754 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "fffc4514-91ea-48e3-9618-9c9e45f8c570": { 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fffc4514-91ea-48e3-9618-9c9e45f8c570", 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:57:54.755 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:57:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 msgr2=0x7fca7c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 0x7fca7c0a3c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca88009fd0 tx=0x7fca88019040 comp rx=0 tx=0).stop 2023-09-23T04:57:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 msgr2=0x7fca94199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:57:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca8400e9a0 tx=0x7fca8400ecb0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 shutdown_connections 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fca7c0a1790 0x7fca7c0a3c50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca9419dcf0 0x7fca9419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fca94103310 0x7fca94199350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 --2- 172.21.15.5:0/4271095336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca94108430 0x7fca94199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.749+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 >> 172.21.15.5:0/4271095336 conn(0x7fca940feb50 msgr2=0x7fca940ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.750+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 shutdown_connections 2023-09-23T04:57:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:57:54.750+0000 7fca997ff700 1 -- 172.21.15.5:0/4271095336 wait complete. 2023-09-23T04:57:54.772 INFO:tasks.cephfs.filesystem:scrub status for tag:fffc4514-91ea-48e3-9618-9c9e45f8c570 - {'path': '/', 'tag': 'fffc4514-91ea-48e3-9618-9c9e45f8c570', 'options': 'recursive,force'} 2023-09-23T04:57:54.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:54 smithi005 ceph-mon[97638]: pgmap v361: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-09-23T04:57:54.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 591312 4.35 MB/sec execute 510 sec latency 2634.955 ms 2023-09-23T04:57:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:54 smithi118 ceph-mon[134297]: pgmap v361: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-09-23T04:57:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:54 smithi069 ceph-mon[133931]: pgmap v361: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2023-09-23T04:57:55.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 592223 4.35 MB/sec execute 511 sec latency 3169.094 ms 2023-09-23T04:57:56.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 593612 4.36 MB/sec execute 512 sec latency 21.468 ms 2023-09-23T04:57:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:56 smithi005 ceph-mon[97638]: pgmap v362: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-09-23T04:57:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:56 smithi118 ceph-mon[134297]: pgmap v362: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-09-23T04:57:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:56 smithi069 ceph-mon[133931]: pgmap v362: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2023-09-23T04:57:57.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594396 4.36 MB/sec execute 513 sec latency 412.602 ms 2023-09-23T04:57:58.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594856 4.36 MB/sec execute 514 sec latency 614.503 ms 2023-09-23T04:57:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:57:58 smithi005 ceph-mon[97638]: pgmap v363: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-09-23T04:57:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:57:58 smithi118 ceph-mon[134297]: pgmap v363: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-09-23T04:57:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:57:58 smithi069 ceph-mon[133931]: pgmap v363: 129 pgs: 129 active+clean; 473 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2023-09-23T04:57:59.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594856 4.35 MB/sec execute 515 sec latency 1614.551 ms 2023-09-23T04:58:00.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594856 4.34 MB/sec execute 516 sec latency 2614.616 ms 2023-09-23T04:58:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:00 smithi005 ceph-mon[97638]: pgmap v364: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 158 op/s 2023-09-23T04:58:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:00 smithi118 ceph-mon[134297]: pgmap v364: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 158 op/s 2023-09-23T04:58:01.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:00 smithi069 ceph-mon[133931]: pgmap v364: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 158 op/s 2023-09-23T04:58:01.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594940 4.33 MB/sec execute 517 sec latency 3549.320 ms 2023-09-23T04:58:02.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 595456 4.33 MB/sec execute 518 sec latency 103.351 ms 2023-09-23T04:58:03.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:02 smithi005 ceph-mon[97638]: pgmap v365: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2023-09-23T04:58:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:02 smithi118 ceph-mon[134297]: pgmap v365: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2023-09-23T04:58:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:02 smithi069 ceph-mon[133931]: pgmap v365: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2023-09-23T04:58:03.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 596950 4.34 MB/sec execute 519 sec latency 68.233 ms 2023-09-23T04:58:04.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:03 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:03 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:04.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:03 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:04.967 INFO:tasks.workunit.client.0.smithi005.stdout: 1 598137 4.34 MB/sec execute 520 sec latency 16.903 ms 2023-09-23T04:58:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: pgmap v366: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2023-09-23T04:58:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:04 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: pgmap v366: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2023-09-23T04:58:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:04 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: pgmap v366: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2023-09-23T04:58:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:04 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:05.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 598403 4.34 MB/sec execute 521 sec latency 792.712 ms 2023-09-23T04:58:06.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 598537 4.33 MB/sec execute 522 sec latency 1726.874 ms 2023-09-23T04:58:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:06 smithi005 ceph-mon[97638]: pgmap v367: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2023-09-23T04:58:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:06 smithi005 ceph-mon[97638]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T04:58:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:06 smithi118 ceph-mon[134297]: pgmap v367: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2023-09-23T04:58:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:06 smithi118 ceph-mon[134297]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T04:58:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:06 smithi069 ceph-mon[133931]: pgmap v367: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 139 op/s 2023-09-23T04:58:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:06 smithi069 ceph-mon[133931]: osdmap e103: 12 total, 12 up, 12 in 2023-09-23T04:58:07.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 600478 4.34 MB/sec execute 523 sec latency 16.956 ms 2023-09-23T04:58:08.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:07 smithi005 ceph-mon[97638]: pgmap v369: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-09-23T04:58:08.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:07 smithi118 ceph-mon[134297]: pgmap v369: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-09-23T04:58:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:07 smithi069 ceph-mon[133931]: pgmap v369: 129 pgs: 129 active+clean; 480 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2023-09-23T04:58:08.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 601513 4.34 MB/sec execute 524 sec latency 31.537 ms 2023-09-23T04:58:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:08 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:09.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:08 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:08 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:58:09.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 601947 4.34 MB/sec execute 525 sec latency 734.357 ms 2023-09-23T04:58:10.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:09 smithi005 ceph-mon[97638]: pgmap v370: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2023-09-23T04:58:10.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:09 smithi118 ceph-mon[134297]: pgmap v370: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2023-09-23T04:58:10.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:09 smithi069 ceph-mon[133931]: pgmap v370: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2023-09-23T04:58:10.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 602836 4.34 MB/sec execute 526 sec latency 1251.121 ms 2023-09-23T04:58:11.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:10 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:58:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:10 smithi005 ceph-mon[97638]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T04:58:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:10 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:58:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:10 smithi118 ceph-mon[134297]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T04:58:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:10 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:58:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:10 smithi069 ceph-mon[133931]: osdmap e104: 12 total, 12 up, 12 in 2023-09-23T04:58:11.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 604354 4.35 MB/sec execute 527 sec latency 11.393 ms 2023-09-23T04:58:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:11 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:58:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:11 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:58:12.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:11 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:58:12.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:11 smithi005 ceph-mon[97638]: scrub summary: idle 2023-09-23T04:58:12.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:11 smithi005 ceph-mon[97638]: pgmap v372: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-09-23T04:58:12.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:11 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:58:12.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:11 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:58:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:11 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:58:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:11 smithi118 ceph-mon[134297]: scrub summary: idle 2023-09-23T04:58:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:11 smithi118 ceph-mon[134297]: pgmap v372: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-09-23T04:58:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:11 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:58:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:11 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:58:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:11 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:58:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:11 smithi069 ceph-mon[133931]: scrub summary: idle 2023-09-23T04:58:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:11 smithi069 ceph-mon[133931]: pgmap v372: 129 pgs: 129 active+clean; 484 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2023-09-23T04:58:12.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605495 4.35 MB/sec execute 528 sec latency 215.931 ms 2023-09-23T04:58:13.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605495 4.34 MB/sec execute 529 sec latency 1216.054 ms 2023-09-23T04:58:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:13 smithi005 ceph-mon[97638]: pgmap v373: 129 pgs: 129 active+clean; 490 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-09-23T04:58:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:13 smithi118 ceph-mon[134297]: pgmap v373: 129 pgs: 129 active+clean; 490 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-09-23T04:58:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:13 smithi069 ceph-mon[133931]: pgmap v373: 129 pgs: 129 active+clean; 490 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-09-23T04:58:14.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605495 4.33 MB/sec execute 530 sec latency 2216.238 ms 2023-09-23T04:58:15.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605495 4.33 MB/sec execute 531 sec latency 3216.316 ms 2023-09-23T04:58:16.968 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605580 4.32 MB/sec execute 532 sec latency 4165.008 ms 2023-09-23T04:58:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:16 smithi069 ceph-mon[133931]: pgmap v374: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2023-09-23T04:58:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:16 smithi005 ceph-mon[97638]: pgmap v374: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2023-09-23T04:58:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:16 smithi118 ceph-mon[134297]: pgmap v374: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2023-09-23T04:58:17.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 607539 4.33 MB/sec execute 533 sec latency 16.517 ms 2023-09-23T04:58:18.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 608511 4.33 MB/sec execute 534 sec latency 16.178 ms 2023-09-23T04:58:18.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:18 smithi069 ceph-mon[133931]: pgmap v375: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2023-09-23T04:58:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:18 smithi005 ceph-mon[97638]: pgmap v375: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2023-09-23T04:58:19.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:18 smithi118 ceph-mon[134297]: pgmap v375: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2023-09-23T04:58:19.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 609036 4.33 MB/sec execute 535 sec latency 655.614 ms 2023-09-23T04:58:20.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 609036 4.32 MB/sec execute 536 sec latency 1655.749 ms 2023-09-23T04:58:20.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:20 smithi069 ceph-mon[133931]: pgmap v376: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2023-09-23T04:58:21.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:20 smithi005 ceph-mon[97638]: pgmap v376: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2023-09-23T04:58:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:20 smithi118 ceph-mon[134297]: pgmap v376: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.7 MiB/s wr, 124 op/s 2023-09-23T04:58:21.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 609170 4.31 MB/sec execute 537 sec latency 2593.322 ms 2023-09-23T04:58:22.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 611123 4.33 MB/sec execute 538 sec latency 17.002 ms 2023-09-23T04:58:22.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:22 smithi069 ceph-mon[133931]: pgmap v377: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2023-09-23T04:58:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:22 smithi005 ceph-mon[97638]: pgmap v377: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2023-09-23T04:58:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:22 smithi118 ceph-mon[134297]: pgmap v377: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2023-09-23T04:58:23.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 612304 4.33 MB/sec execute 539 sec latency 45.710 ms 2023-09-23T04:58:24.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:58:24.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 612586 4.32 MB/sec execute 540 sec latency 775.949 ms 2023-09-23T04:58:24.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 -- 172.21.15.5:0/2497696644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 msgr2=0x7f77bc1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:24.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2497696644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc1014b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009990 tx=0x7f77ac009ca0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 -- 172.21.15.5:0/2497696644 shutdown_connections 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2497696644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2497696644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc1014b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2497696644 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc101a80 0x7f77bc105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.981+0000 7f77c11ff700 1 -- 172.21.15.5:0/2497696644 >> 172.21.15.5:0/2497696644 conn(0x7f77bc0fc990 msgr2=0x7f77bc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.982+0000 7f77c11ff700 1 -- 172.21.15.5:0/2497696644 shutdown_connections 2023-09-23T04:58:24.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.982+0000 7f77c11ff700 1 -- 172.21.15.5:0/2497696644 wait complete. 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.982+0000 7f77c11ff700 1 Processor -- start 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 -- start start 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc19daf0 0x7f77bc19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc19e570 con 0x7f77bc1010d0 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc19e6e0 con 0x7f77bc1061f0 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77c11ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc19e8a0 con 0x7f77bc19daf0 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77bb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc19daf0 0x7f77bc19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60916/0 (socket says 172.21.15.5:60916) 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77b3fff700 1 -- 172.21.15.5:0/2508813326 learned_addr learned my addr 172.21.15.5:0/2508813326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:24.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.983+0000 7f77bad9d700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 -- 172.21.15.5:0/2508813326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc19daf0 msgr2=0x7f77bc19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc19daf0 0x7f77bc19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 -- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 msgr2=0x7f77bc199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:24.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 -- 172.21.15.5:0/2508813326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0095f0 con 0x7f77bc1061f0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b3fff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f77a400e970 tx=0x7f77a400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b8d99700 1 -- 172.21.15.5:0/2508813326 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4009790 con 0x7f77bc1061f0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b8d99700 1 -- 172.21.15.5:0/2508813326 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a400c7e0 con 0x7f77bc1061f0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc1a5e00 con 0x7f77bc1061f0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77b8d99700 1 -- 172.21.15.5:0/2508813326 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40054e0 con 0x7f77bc1061f0 2023-09-23T04:58:24.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.984+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc1a6280 con 0x7f77bc1061f0 2023-09-23T04:58:24.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.988+0000 7f77b8d99700 1 -- 172.21.15.5:0/2508813326 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f77a400c950 con 0x7f77bc1061f0 2023-09-23T04:58:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.989+0000 7f77d4b1db80 1 Processor -- start 2023-09-23T04:58:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 -- start start 2023-09-23T04:58:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:24 smithi069 ceph-mon[133931]: pgmap v378: 129 pgs: 129 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-09-23T04:58:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba03e460 con 0x55c4ba0d5d20 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba02fc20 con 0x55c4ba0e0bd0 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.990+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba02d540 con 0x55c4ba0d5400 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bb59e700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77b8d99700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 0x7f77a80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40800/0 (socket says 172.21.15.5:40800) 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 -- 172.21.15.5:0/2749885813 learned_addr learned my addr 172.21.15.5:0/2749885813 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bb59e700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 0x7f77a80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77b8d99700 1 -- 172.21.15.5:0/2508813326 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f77a4014070 con 0x7f77bc1061f0 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 -- 172.21.15.5:0/2749885813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 msgr2=0x55c4ba0d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 -- 172.21.15.5:0/2749885813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 msgr2=0x55c4ba0e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 -- 172.21.15.5:0/2749885813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4ba08b8d0 con 0x55c4ba0d5d20 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.991+0000 7f77bad9d700 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f77ac015000 tx=0x7f77ac00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9acd0551aacab6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77bb59e700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 0x7f77a80a3bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009ec0 tx=0x7f77b4004b40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77b17fa700 1 -- 172.21.15.5:0/2749885813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac01d070 con 0x55c4ba0d5d20 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77b17fa700 1 -- 172.21.15.5:0/2749885813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac01e040 con 0x55c4ba0d5d20 2023-09-23T04:58:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77b17fa700 1 -- 172.21.15.5:0/2749885813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac004870 con 0x55c4ba0d5d20 2023-09-23T04:58:24.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 -- 172.21.15.5:0/2749885813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 msgr2=0x55c4ba0e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:24.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f77ac015000 tx=0x7f77ac00b5b0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 -- 172.21.15.5:0/2749885813 shutdown_connections 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/2749885813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e0690 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.992+0000 7f77d4b1db80 1 -- 172.21.15.5:0/2749885813 >> 172.21.15.5:0/2749885813 conn(0x55c4ba04a260 msgr2=0x55c4ba0d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.993+0000 7f77d4b1db80 1 -- 172.21.15.5:0/2749885813 shutdown_connections 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.993+0000 7f77d4b1db80 1 -- 172.21.15.5:0/2749885813 wait complete. 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.993+0000 7f77d4b1db80 1 -- start start 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.993+0000 7f77d4b1db80 1 Processor -- start 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba028e30 con 0x55c4ba0d5400 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b9fc5710 con 0x55c4ba0d5d20 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b9fc53b0 con 0x55c4ba0e0bd0 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77d4b1db80 10 client.0 _ll_register_callbacks cb 0x55c4b9e22800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77bad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60932/0 (socket says 172.21.15.5:60932) 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77b3fff700 1 -- 172.21.15.5:0/901422043 learned_addr learned my addr 172.21.15.5:0/901422043 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77bb59e700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77b3fff700 1 -- 172.21.15.5:0/901422043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0e0bd0 msgr2=0x55c4ba0ddf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.994+0000 7f77b3fff700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0ddf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:24.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77b3fff700 1 -- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5400 msgr2=0x55c4ba0e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77b3fff700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77b3fff700 1 -- 172.21.15.5:0/901422043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc10e5e0 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77b3fff700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77a40969a0 tx=0x7f77a4096910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc0ec8186ce69ff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4005720 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4ba0c9250 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f77a409aba0 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a409a4a0 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77d4b1db80 10 client.27458 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c4b9e53de0 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.995+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a40b6020 con 0x55c4ba0d5d20 2023-09-23T04:58:25.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f77a409e160 con 0x55c4ba0d5d20 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 10 client.27458 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 10 client.27458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 20 client.27458 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba0f0650 0x55c4ba0f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 4 client.27458 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4ba021e10 con 0x55c4ba0f0650 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.996+0000 7f77bad9d700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba0f0650 0x55c4ba0f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77a27fc700 10 client.27458 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:25.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77bad9d700 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba0f0650 0x55c4ba0f2a70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9489e4217fc3fe9 server_cookie=36b35cce1856f8f4 in_seq=0 out_seq=0 2023-09-23T04:58:25.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77a27fc700 1 -- 172.21.15.5:0/901422043 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c4ba021e10 con 0x55c4ba0f0650 2023-09-23T04:58:25.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77a27fc700 10 client.27458 handle_command_reply: tid=0 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77d4b1db80 1 client.27458 shutdown 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 msgr2=0x55c4ba0dd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77a40969a0 tx=0x7f77a4096910 comp rx=0 tx=0).stop 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 shutdown_connections 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.997+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba0f0650 0x55c4ba0f2a70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0ddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/901422043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 >> 172.21.15.5:0/901422043 conn(0x55c4ba04a260 msgr2=0x55c4ba0d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77a27fc700 0 client.27458 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 shutdown_connections 2023-09-23T04:58:25.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 1 -- 172.21.15.5:0/901422043 wait complete. 2023-09-23T04:58:25.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:24.998+0000 7f77d4b1db80 20 client.27458 trim_cache size 0 max 16384 2023-09-23T04:58:25.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.015+0000 7f77d4b1db80 1 Processor -- start 2023-09-23T04:58:25.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.015+0000 7f77d4b1db80 1 -- start start 2023-09-23T04:58:25.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b9ea9920 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0d0580 con 0x55c4ba0d5d20 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0d96c0 con 0x55c4ba0d5400 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40812/0 (socket says 172.21.15.5:40812) 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 -- 172.21.15.5:0/3653059027 learned_addr learned my addr 172.21.15.5:0/3653059027 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77b3fff700 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bad9d700 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 -- 172.21.15.5:0/3653059027 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 msgr2=0x55c4ba0e0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 -- 172.21.15.5:0/3653059027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 msgr2=0x55c4ba0dd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.016+0000 7f77bb59e700 1 -- 172.21.15.5:0/3653059027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b9e3cc60 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.017+0000 7f77bb59e700 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f77b40094e0 tx=0x7f77b4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b415f13b48ed085 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.017+0000 7f77b1ffb700 1 -- 172.21.15.5:0/3653059027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4028070 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.017+0000 7f77b1ffb700 1 -- 172.21.15.5:0/3653059027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b400f040 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.017+0000 7f77b1ffb700 1 -- 172.21.15.5:0/3653059027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b400e040 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- 172.21.15.5:0/3653059027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 msgr2=0x55c4ba0dd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f77b40094e0 tx=0x7f77b4009920 comp rx=0 tx=0).stop 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- 172.21.15.5:0/3653059027 shutdown_connections 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0dd570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/3653059027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0dd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- 172.21.15.5:0/3653059027 >> 172.21.15.5:0/3653059027 conn(0x55c4ba04a260 msgr2=0x55c4ba0d8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- 172.21.15.5:0/3653059027 shutdown_connections 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- 172.21.15.5:0/3653059027 wait complete. 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 -- start start 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.018+0000 7f77d4b1db80 1 Processor -- start 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0e6230 con 0x55c4ba0d5d20 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0e63a0 con 0x55c4ba0e0bd0 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ba0e6560 con 0x55c4ba0d5400 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77d4b1db80 10 client.0 _ll_register_callbacks cb 0x7f77a4099fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42436/0 (socket says 172.21.15.5:42436) 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bad9d700 1 -- 172.21.15.5:0/1631507234 learned_addr learned my addr 172.21.15.5:0/1631507234 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bb59e700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bad9d700 1 -- 172.21.15.5:0/1631507234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 msgr2=0x55c4ba0e5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.019+0000 7f77bad9d700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77bad9d700 1 -- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 msgr2=0x55c4ba0e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77bad9d700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77bad9d700 1 -- 172.21.15.5:0/1631507234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40091c0 con 0x55c4ba0d5400 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77b3fff700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77bad9d700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009350 tx=0x7f77ac0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5680af04dac385c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:58:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac034040 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77ac01e040 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f77ac00b7f0 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77ac01d040 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77d4b1db80 10 client.27526 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.020+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55c4b9e7a250 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac01e040 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f77ac004500 con 0x55c4ba0d5400 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 10 client.27526 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 10 client.27526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 20 client.27526 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba145470 0x55c4ba147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 4 client.27526 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:58:25.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4ba021b30 con 0x55c4ba145470 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.021+0000 7f77b3fff700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba145470 0x55c4ba147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77b3fff700 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba145470 0x55c4ba147890 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bccca0b5496a7fe server_cookie=c1b2c358f7a61841 in_seq=0 out_seq=0 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77a27fc700 10 client.27526 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77a27fc700 1 -- 172.21.15.5:0/1631507234 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c4ba021b30 con 0x55c4ba145470 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77a27fc700 10 client.27526 handle_command_reply: tid=0 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 client.27526 shutdown 2023-09-23T04:58:25.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 msgr2=0x55c4ba0e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009350 tx=0x7f77ac0059d0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 shutdown_connections 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c4ba145470 0x55c4ba147890 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c4ba0e0bd0 0x55c4ba0e5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77a27fc700 0 client.27526 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c4ba0d5400 0x55c4ba0e7480 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 --2- 172.21.15.5:0/1631507234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4ba0d5d20 0x55c4ba0e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 >> 172.21.15.5:0/1631507234 conn(0x55c4ba04a260 msgr2=0x55c4ba0d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 shutdown_connections 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 1 -- 172.21.15.5:0/1631507234 wait complete. 2023-09-23T04:58:25.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.022+0000 7f77d4b1db80 20 client.27526 trim_cache size 0 max 16384 2023-09-23T04:58:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:58:25.029 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-09-23T04:58:25.029 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-09-23T04:58:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 msgr2=0x7f77a80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 0x7f77a80a3bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009ec0 tx=0x7f77b4004b40 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 msgr2=0x7f77bc199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f77a400e970 tx=0x7f77a400ec80 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 shutdown_connections 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f77a80a1710 0x7f77a80a3bd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f77bc19daf0 0x7f77bc19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77bc1010d0 0x7f77bc199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 --2- 172.21.15.5:0/2508813326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77bc1061f0 0x7f77bc199690 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 >> 172.21.15.5:0/2508813326 conn(0x7f77bc0fc990 msgr2=0x7f77bc0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 shutdown_connections 2023-09-23T04:58:25.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.023+0000 7f77c11ff700 1 -- 172.21.15.5:0/2508813326 wait complete. 2023-09-23T04:58:25.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:24 smithi005 ceph-mon[97638]: pgmap v378: 129 pgs: 129 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-09-23T04:58:25.042 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T04:58:25.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T04:58:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:24 smithi118 ceph-mon[134297]: pgmap v378: 129 pgs: 129 active+clean; 497 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-09-23T04:58:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 -- 172.21.15.5:0/2797469309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068dc0 msgr2=0x7f4450105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 --2- 172.21.15.5:0/2797469309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068dc0 0x7f4450105aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4440009990 tx=0x7f4440009ca0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 -- 172.21.15.5:0/2797469309 shutdown_connections 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 --2- 172.21.15.5:0/2797469309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f445010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 --2- 172.21.15.5:0/2797469309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4450068410 0x7f44500687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 --2- 172.21.15.5:0/2797469309 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068dc0 0x7f4450105aa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 -- 172.21.15.5:0/2797469309 >> 172.21.15.5:0/2797469309 conn(0x7f4450076ea0 msgr2=0x7f44500772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 -- 172.21.15.5:0/2797469309 shutdown_connections 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.268+0000 7f4457bff700 1 -- 172.21.15.5:0/2797469309 wait complete. 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 Processor -- start 2023-09-23T04:58:25.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 -- start start 2023-09-23T04:58:25.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f44501974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445019b950 0x7f445019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019c3d0 con 0x7f445019b950 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019c540 con 0x7f44501061d0 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.270+0000 7f4457bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019c700 con 0x7f4450068410 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445519a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f44501974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42466/0 (socket says 172.21.15.5:42466) 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 -- 172.21.15.5:0/2102789061 learned_addr learned my addr 172.21.15.5:0/2102789061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445619c700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445019b950 0x7f445019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 -- 172.21.15.5:0/2102789061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 msgr2=0x7f44501974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f44501974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 -- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445019b950 msgr2=0x7f445019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445019b950 0x7f445019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 -- 172.21.15.5:0/2102789061 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400095f0 con 0x7f4450068410 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445519a700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f44501974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.271+0000 7f445599b700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9a0 tx=0x7f444c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.272+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c009790 con 0x7f4450068410 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.272+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444c010430 con 0x7f4450068410 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.272+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44501a3c60 con 0x7f4450068410 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.272+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c018680 con 0x7f4450068410 2023-09-23T04:58:25.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.272+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44501a4140 con 0x7f4450068410 2023-09-23T04:58:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.273+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450067ca0 con 0x7f4450068410 2023-09-23T04:58:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.275+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f444c0105a0 con 0x7f4450068410 2023-09-23T04:58:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.276+0000 7f4446ffd700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 0x7f443c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.276+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f444c014070 con 0x7f4450068410 2023-09-23T04:58:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.277+0000 7f445519a700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 0x7f443c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.277+0000 7f445519a700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 0x7f443c0a3bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44501985c0 tx=0x7f4440005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.280+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f444c051ec0 con 0x7f4450068410 2023-09-23T04:58:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.603+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4450198300 con 0x7f4450068410 2023-09-23T04:58:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.603+0000 7f4446ffd700 1 -- 172.21.15.5:0/2102789061 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f4450198300 con 0x7f4450068410 2023-09-23T04:58:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T04:58:25.605 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T04:58:06.889445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T04:58:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 msgr2=0x7f443c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 0x7f443c0a3bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44501985c0 tx=0x7f4440005fd0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 msgr2=0x7f4450196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9a0 tx=0x7f444c00ecb0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 shutdown_connections 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f443c0a1710 0x7f443c0a3bd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445019b950 0x7f445019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4450068410 0x7f4450196fb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 --2- 172.21.15.5:0/2102789061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44501061d0 0x7f44501974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.606+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 >> 172.21.15.5:0/2102789061 conn(0x7f4450076ea0 msgr2=0x7f44500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.607+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 shutdown_connections 2023-09-23T04:58:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.607+0000 7f4457bff700 1 -- 172.21.15.5:0/2102789061 wait complete. 2023-09-23T04:58:25.609 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2023-09-23T04:58:25.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2003128471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d341061f0 msgr2=0x7f9d3410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2003128471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d341061f0 0x7f9d3410dd70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00b0a0 tx=0x7f9d2c00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2003128471 shutdown_connections 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2003128471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d341061f0 0x7f9d3410dd70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2003128471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341010d0 0x7f9d341014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2003128471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d34101a80 0x7f9d34105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2003128471 >> 172.21.15.5:0/2003128471 conn(0x7f9d340fc990 msgr2=0x7f9d340fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2003128471 shutdown_connections 2023-09-23T04:58:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.861+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2003128471 wait complete. 2023-09-23T04:58:25.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.862+0000 7f9d3c1ff700 1 Processor -- start 2023-09-23T04:58:25.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.862+0000 7f9d3c1ff700 1 -- start start 2023-09-23T04:58:25.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 0x7f9d34073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d34073ba0 0x7f9d341a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341a58b0 con 0x7f9d341061f0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341a5a20 con 0x7f9d341010d0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3c1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341a5be0 con 0x7f9d34073ba0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60988/0 (socket says 172.21.15.5:60988) 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 -- 172.21.15.5:0/2613033690 learned_addr learned my addr 172.21.15.5:0/2613033690 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3979a700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 0x7f9d34073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d34073ba0 0x7f9d341a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 -- 172.21.15.5:0/2613033690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d34073ba0 msgr2=0x7f9d341a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d34073ba0 0x7f9d341a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 -- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 msgr2=0x7f9d34073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 0x7f9d34073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d39f9b700 1 -- 172.21.15.5:0/2613033690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009d00 con 0x7f9d341010d0 2023-09-23T04:58:25.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.863+0000 7f9d3979a700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 0x7f9d34073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:25.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e9a0 tx=0x7f9d3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d2affd700 1 -- 172.21.15.5:0/2613033690 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30009790 con 0x7f9d341010d0 2023-09-23T04:58:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d2affd700 1 -- 172.21.15.5:0/2613033690 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3000cbd0 con 0x7f9d341010d0 2023-09-23T04:58:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d341a5ec0 con 0x7f9d341010d0 2023-09-23T04:58:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d2affd700 1 -- 172.21.15.5:0/2613033690 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30018680 con 0x7f9d341010d0 2023-09-23T04:58:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.864+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d341a6390 con 0x7f9d341010d0 2023-09-23T04:58:25.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.868+0000 7f9d2affd700 1 -- 172.21.15.5:0/2613033690 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f9d3000cd40 con 0x7f9d341010d0 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 Processor -- start 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 -- start start 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d2affd700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 0x7f9d200a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564260401460 con 0x564260498400 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642603f2c20 con 0x564260498d20 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642603f0540 con 0x5642604a3bd0 2023-09-23T04:58:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d2affd700 1 -- 172.21.15.5:0/2613033690 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9d30014070 con 0x7f9d341010d0 2023-09-23T04:58:25.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 0x7f9d200a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:60998/0 (socket says 172.21.15.5:60998) 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.870+0000 7f9d3979a700 1 -- 172.21.15.5:0/861677828 learned_addr learned my addr 172.21.15.5:0/861677828 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 -- 172.21.15.5:0/861677828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 msgr2=0x5642604a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 -- 172.21.15.5:0/861677828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 msgr2=0x5642604987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 -- 172.21.15.5:0/861677828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56426044e8d0 con 0x564260498d20 2023-09-23T04:58:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3979a700 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d20 tx=0x7f9d240106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63afdb9fce6e197 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.871+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 0x7f9d200a3c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006130 tx=0x7f9d2c006040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.872+0000 7f9d1ffff700 1 -- 172.21.15.5:0/861677828 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24010e10 con 0x564260498d20 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.872+0000 7f9d1ffff700 1 -- 172.21.15.5:0/861677828 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d24015070 con 0x564260498d20 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.872+0000 7f9d1ffff700 1 -- 172.21.15.5:0/861677828 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24011440 con 0x564260498d20 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.872+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/861677828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 msgr2=0x5642604a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.872+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d20 tx=0x7f9d240106b0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/861677828 shutdown_connections 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/861677828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a3690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/861677828 >> 172.21.15.5:0/861677828 conn(0x56426040d260 msgr2=0x564260494790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/861677828 shutdown_connections 2023-09-23T04:58:25.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/861677828 wait complete. 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.873+0000 7f9d4fa60b80 1 -- start start 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 Processor -- start 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 0x5642604a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5642604a3bd0 0x5642604a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642603ebe30 con 0x564260498400 2023-09-23T04:58:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564260388710 con 0x5642604a3bd0 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642603883b0 con 0x564260498d20 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d4fa60b80 10 client.0 _ll_register_callbacks cb 0x5642601e5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40882/0 (socket says 172.21.15.5:40882) 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d39f9b700 1 -- 172.21.15.5:0/115552939 learned_addr learned my addr 172.21.15.5:0/115552939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 0x5642604a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.874+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5642604a3bd0 0x5642604a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 -- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 msgr2=0x5642604a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 0x5642604a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 -- 172.21.15.5:0/115552939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5642604a3bd0 msgr2=0x5642604a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5642604a3bd0 0x5642604a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 -- 172.21.15.5:0/115552939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d24009950 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d3979a700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 0x5642604a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9d300cacb0 tx=0x7f9d300cb1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc72092ad3b641c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3009a470 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9d3009e5c0 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.875+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56426048c250 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d300a2800 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 10 client.17496 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x564260216de0 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56426048c250 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9d300993d0 con 0x564260498400 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 10 client.17496 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:25.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 10 client.17496 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 20 client.17496 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5642604b3660 0x5642604b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 4 client.17496 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.876+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642603e4e10 con 0x5642604b3660 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.877+0000 7f9d3979a700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5642604b3660 0x5642604b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.877+0000 7f9d3979a700 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5642604b3660 0x5642604b5a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c0c684f74f733e7 server_cookie=7bb3b65d384656a9 in_seq=0 out_seq=0 2023-09-23T04:58:25.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.877+0000 7f9d1d7fa700 10 client.17496 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/115552939 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5642603e4e10 con 0x5642604b3660 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d1d7fa700 10 client.17496 handle_command_reply: tid=0 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 client.17496 shutdown 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 msgr2=0x5642604a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9d300cacb0 tx=0x7f9d300cb1b0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 shutdown_connections 2023-09-23T04:58:25.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5642604a3bd0 0x5642604a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x5642604b3660 0x5642604b5a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604a08f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/115552939 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564260498d20 0x5642604a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 >> 172.21.15.5:0/115552939 conn(0x56426040d260 msgr2=0x564260494380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d1d7fa700 0 client.17496 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 shutdown_connections 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/115552939 wait complete. 2023-09-23T04:58:25.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.878+0000 7f9d4fa60b80 20 client.17496 trim_cache size 0 max 16384 2023-09-23T04:58:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 Processor -- start 2023-09-23T04:58:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 -- start start 2023-09-23T04:58:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498d20 0x56426049f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x56426049faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426026c920 con 0x564260498d20 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642604a0020 con 0x564260498400 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.894+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642604af610 con 0x5642604a3bd0 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d3a79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x56426049faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32776/0 (socket says 172.21.15.5:32776) 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498d20 0x56426049f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 -- 172.21.15.5:0/3494268544 learned_addr learned my addr 172.21.15.5:0/3494268544 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 -- 172.21.15.5:0/3494268544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 msgr2=0x56426049faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x56426049faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 -- 172.21.15.5:0/3494268544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498d20 msgr2=0x56426049f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498d20 0x56426049f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 -- 172.21.15.5:0/3494268544 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642601ffc60 con 0x564260498400 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e970 tx=0x7f9d300998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85de3e3dec1a0cc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.895+0000 7f9d28ff9700 1 -- 172.21.15.5:0/3494268544 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3009e5c0 con 0x564260498400 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.896+0000 7f9d28ff9700 1 -- 172.21.15.5:0/3494268544 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3009a470 con 0x564260498400 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.896+0000 7f9d28ff9700 1 -- 172.21.15.5:0/3494268544 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3009e120 con 0x564260498400 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.896+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/3494268544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 msgr2=0x5642604a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.896+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e970 tx=0x7f9d300998a0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/3494268544 shutdown_connections 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x56426049faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498400 0x5642604a11a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/3494268544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498d20 0x56426049f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/3494268544 >> 172.21.15.5:0/3494268544 conn(0x56426040d260 msgr2=0x56426049ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/3494268544 shutdown_connections 2023-09-23T04:58:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/3494268544 wait complete. 2023-09-23T04:58:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 -- start start 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.897+0000 7f9d4fa60b80 1 Processor -- start 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642604a86d0 con 0x564260498400 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642604a8840 con 0x564260498d20 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642605064a0 con 0x5642604a3bd0 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d4fa60b80 10 client.0 _ll_register_callbacks cb 0x7f9d30099fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d39f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d3a79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d3979a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32784/0 (socket says 172.21.15.5:32784) 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d3979a700 1 -- 172.21.15.5:0/1532081620 learned_addr learned my addr 172.21.15.5:0/1532081620 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.898+0000 7f9d3a79c700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42526/0 (socket says 172.21.15.5:42526) 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 -- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 msgr2=0x5642604a7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 -- 172.21.15.5:0/1532081620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 msgr2=0x5642604a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 -- 172.21.15.5:0/1532081620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564260216de0 con 0x564260498400 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d3979a700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d39f9b700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604aa760 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c5b0 tx=0x7f9d3009a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37d32fe334c627e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3009e5c0 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d24015070 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9d3009e120 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d3009e2e0 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d4fa60b80 10 client.17502 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.899+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x56426023d250 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d24015070 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9d30083bf0 con 0x564260498400 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 10 client.17502 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 10 client.17502 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 20 client.17502 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564260507710 0x564260509b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 4 client.17502 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 --> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5642603e4b30 con 0x564260507710 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.900+0000 7f9d3979a700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564260507710 0x564260509b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d3979a700 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564260507710 0x564260509b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15639ceea819c008 server_cookie=d5f6621bed701c0e in_seq=0 out_seq=0 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d1d7fa700 10 client.17502 ms_handle_connect on v2:172.21.15.118:6834/3772048959 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d1d7fa700 1 -- 172.21.15.5:0/1532081620 <== mds.4 v2:172.21.15.118:6834/3772048959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5642603e4b30 con 0x564260507710 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d1d7fa700 10 client.17502 handle_command_reply: tid=0 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 client.17502 shutdown 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 msgr2=0x5642604aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604aa760 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c5b0 tx=0x7f9d3009a330 comp rx=0 tx=0).stop 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 shutdown_connections 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5642604a3bd0 0x5642604a7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.118:6834/3772048959,v1:172.21.15.118:6835/3772048959] conn(0x564260507710 0x564260509b00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564260498400 0x5642604aa760 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 --2- 172.21.15.5:0/1532081620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564260498d20 0x5642604a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 >> 172.21.15.5:0/1532081620 conn(0x56426040d260 msgr2=0x5642604a0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.901+0000 7f9d1d7fa700 0 client.17502 ms_handle_reset on v2:172.21.15.118:6834/3772048959 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 shutdown_connections 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d4fa60b80 1 -- 172.21.15.5:0/1532081620 wait complete. 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d4fa60b80 20 client.17502 trim_cache size 0 max 16384 2023-09-23T04:58:25.907 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:58:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 msgr2=0x7f9d200a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 0x7f9d200a3c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006130 tx=0x7f9d2c006040 comp rx=0 tx=0).stop 2023-09-23T04:58:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 msgr2=0x7f9d34073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e9a0 tx=0x7f9d3000ecb0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 shutdown_connections 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9d200a1790 0x7f9d200a3c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9d34073ba0 0x7f9d341a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d341010d0 0x7f9d34073120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.902+0000 7f9d3c1ff700 1 --2- 172.21.15.5:0/2613033690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d341061f0 0x7f9d34073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.903+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 >> 172.21.15.5:0/2613033690 conn(0x7f9d340fc990 msgr2=0x7f9d340fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.903+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 shutdown_connections 2023-09-23T04:58:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:25.903+0000 7f9d3c1ff700 1 -- 172.21.15.5:0/2613033690 wait complete. 2023-09-23T04:58:25.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:25 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2102789061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:58:25.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-23T04:58:25.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 614232 4.33 MB/sec execute 541 sec latency 889.815 ms 2023-09-23T04:58:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:25 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2102789061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.158+0000 7f3d45bff700 1 -- 172.21.15.5:0/2525475451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40106210 msgr2=0x7f3d4010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.158+0000 7f3d45bff700 1 --2- 172.21.15.5:0/2525475451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40106210 0x7f3d4010dd90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009f90 tx=0x7f3d3800b330 comp rx=0 tx=0).stop 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 -- 172.21.15.5:0/2525475451 shutdown_connections 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 --2- 172.21.15.5:0/2525475451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d40106210 0x7f3d4010dd90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 --2- 172.21.15.5:0/2525475451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d400690a0 0x7f3d40069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 --2- 172.21.15.5:0/2525475451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40069a50 0x7f3d40105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 -- 172.21.15.5:0/2525475451 >> 172.21.15.5:0/2525475451 conn(0x7f3d40077ca0 msgr2=0x7f3d400780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 -- 172.21.15.5:0/2525475451 shutdown_connections 2023-09-23T04:58:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.159+0000 7f3d45bff700 1 -- 172.21.15.5:0/2525475451 wait complete. 2023-09-23T04:58:26.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:25 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2102789061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.160+0000 7f3d45bff700 1 Processor -- start 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.160+0000 7f3d45bff700 1 -- start start 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40106210 0x7f3d40199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 0x7f3d4019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4019e4e0 con 0x7f3d40106210 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4019e650 con 0x7f3d400690a0 2023-09-23T04:58:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d45bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4019e810 con 0x7f3d4019da60 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40106210 0x7f3d40199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 0x7f3d4019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32798/0 (socket says 172.21.15.5:32798) 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 0x7f3d4019dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42536/0 (socket says 172.21.15.5:42536) 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 -- 172.21.15.5:0/1629546448 learned_addr learned my addr 172.21.15.5:0/1629546448 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 -- 172.21.15.5:0/1629546448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 msgr2=0x7f3d4019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 0x7f3d4019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 -- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40106210 msgr2=0x7f3d40199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40106210 0x7f3d40199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.161+0000 7f3d3f7fe700 1 -- 172.21.15.5:0/1629546448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300095f0 con 0x7f3d400690a0 2023-09-23T04:58:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000ec10 tx=0x7f3d3000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d3cff9700 1 -- 172.21.15.5:0/1629546448 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000cc90 con 0x7f3d400690a0 2023-09-23T04:58:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d38009c40 con 0x7f3d400690a0 2023-09-23T04:58:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d401a6110 con 0x7f3d400690a0 2023-09-23T04:58:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d3cff9700 1 -- 172.21.15.5:0/1629546448 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d300044e0 con 0x7f3d400690a0 2023-09-23T04:58:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.162+0000 7f3d3cff9700 1 -- 172.21.15.5:0/1629546448 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30010430 con 0x7f3d400690a0 2023-09-23T04:58:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.167+0000 7f3d3cff9700 1 -- 172.21.15.5:0/1629546448 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3d30003680 con 0x7f3d400690a0 2023-09-23T04:58:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.167+0000 7f3d59532b80 1 Processor -- start 2023-09-23T04:58:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.167+0000 7f3d59532b80 1 -- start start 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e862492d20 0x55e86249d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86249dbd0 0x55e86249ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8623fb460 con 0x55e86249dbd0 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8623ecc20 con 0x55e862492d20 2023-09-23T04:58:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d59532b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8623ea540 con 0x55e862492400 2023-09-23T04:58:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42552/0 (socket says 172.21.15.5:42552) 2023-09-23T04:58:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d3ffff700 1 -- 172.21.15.5:0/2121677837 learned_addr learned my addr 172.21.15.5:0/2121677837 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.168+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e862492d20 0x55e86249d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 -- 172.21.15.5:0/2121677837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e862492d20 msgr2=0x55e86249d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e862492d20 0x55e86249d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 -- 172.21.15.5:0/2121677837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86249dbd0 msgr2=0x55e86249ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86249dbd0 0x55e86249ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 -- 172.21.15.5:0/2121677837 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8624488d0 con 0x55e862492400 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007be0 tx=0x7f3d38007c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b67cce50a02868 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d25ffb700 1 -- 172.21.15.5:0/2121677837 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e040 con 0x55e862492400 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3cff9700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 0x7f3d280a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d25ffb700 1 -- 172.21.15.5:0/2121677837 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d38007e30 con 0x55e862492400 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d25ffb700 1 -- 172.21.15.5:0/2121677837 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3801c440 con 0x55e862492400 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.169+0000 7f3d3cff9700 1 -- 172.21.15.5:0/1629546448 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3d30014070 con 0x7f3d400690a0 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 0x7f3d280a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 -- 172.21.15.5:0/2121677837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 msgr2=0x55e8624927e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007be0 tx=0x7f3d38007c10 comp rx=0 tx=0).stop 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d3f7fe700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 0x7f3d280a3bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d30003ad0 tx=0x7f3d300d34b0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 -- 172.21.15.5:0/2121677837 shutdown_connections 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86249dbd0 0x55e86249ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624927e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 --2- 172.21.15.5:0/2121677837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e862492d20 0x55e86249d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.170+0000 7f3d59532b80 1 -- 172.21.15.5:0/2121677837 >> 172.21.15.5:0/2121677837 conn(0x55e862407260 msgr2=0x55e86248e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.171+0000 7f3d59532b80 1 -- 172.21.15.5:0/2121677837 shutdown_connections 2023-09-23T04:58:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.171+0000 7f3d59532b80 1 -- 172.21.15.5:0/2121677837 wait complete. 2023-09-23T04:58:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.171+0000 7f3d59532b80 1 -- start start 2023-09-23T04:58:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.171+0000 7f3d59532b80 1 Processor -- start 2023-09-23T04:58:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e86249d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e86249ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8623e5e30 con 0x55e862492d20 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e862382710 con 0x55e86249dbd0 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8623823b0 con 0x55e862492400 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d59532b80 10 client.0 _ll_register_callbacks cb 0x55e8621df800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e86249d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.172+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40932/0 (socket says 172.21.15.5:40932) 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 -- 172.21.15.5:0/759415134 learned_addr learned my addr 172.21.15.5:0/759415134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e86249ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 -- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 msgr2=0x55e86249d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e86249d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 -- 172.21.15.5:0/759415134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 msgr2=0x55e86249ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e86249ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 -- 172.21.15.5:0/759415134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4010e600 con 0x55e862492d20 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d3effd700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3d340116f0 tx=0x7f3d34011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639f967976235a0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.173+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34004520 con 0x55e862492d20 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3d34003680 con 0x55e862492d20 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e862486250 con 0x55e862492d20 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d34003840 con 0x55e862492d20 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 10 client.17511 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e862210de0 con 0x55e862492d20 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e862486250 con 0x55e862492d20 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3d3400fb90 con 0x55e862492d20 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 10 client.17511 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 10 client.17511 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 20 client.17511 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e8624ad6c0 0x55e8624afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 4 client.17511 mds_command: new command op to 24535 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.174+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8623dee10 con 0x55e8624ad6c0 2023-09-23T04:58:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.175+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e8624ad6c0 0x55e8624afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.175+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e8624ad6c0 0x55e8624afae0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55d05615ea4782fb server_cookie=f4c095cb4c563c84 in_seq=0 out_seq=0 2023-09-23T04:58:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.175+0000 7f3d1effd700 10 client.17511 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d1effd700 1 -- 172.21.15.5:0/759415134 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8623dee10 con 0x55e8624ad6c0 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d1effd700 10 client.17511 handle_command_reply: tid=0 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 client.17511 shutdown 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 msgr2=0x55e86249a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3d340116f0 tx=0x7f3d34011ab0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 shutdown_connections 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e86249ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e86249d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e8624ad6c0 0x55e8624afae0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 --2- 172.21.15.5:0/759415134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e86249a770 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 >> 172.21.15.5:0/759415134 conn(0x55e862407260 msgr2=0x55e86248e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.176+0000 7f3d1effd700 0 client.17511 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:58:26.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.177+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 shutdown_connections 2023-09-23T04:58:26.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.177+0000 7f3d59532b80 1 -- 172.21.15.5:0/759415134 wait complete. 2023-09-23T04:58:26.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.177+0000 7f3d59532b80 20 client.17511 trim_cache size 0 max 16384 2023-09-23T04:58:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 Processor -- start 2023-09-23T04:58:26.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 -- start start 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e862495a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e862495f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e862266920 con 0x55e862492d20 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e862493bf0 con 0x55e86249dbd0 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d59532b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e86249b2f0 con 0x55e862492400 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42574/0 (socket says 172.21.15.5:42574) 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.192+0000 7f3d3ffff700 1 -- 172.21.15.5:0/392957610 learned_addr learned my addr 172.21.15.5:0/392957610 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e862495f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3effd700 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e862495a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 -- 172.21.15.5:0/392957610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 msgr2=0x55e862495f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e862495f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 -- 172.21.15.5:0/392957610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 msgr2=0x55e862495a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e862495a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 -- 172.21.15.5:0/392957610 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8621f9c60 con 0x55e862492400 2023-09-23T04:58:26.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007500 tx=0x7f3d38007530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab68953c10d2f87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d267fc700 1 -- 172.21.15.5:0/392957610 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3802b020 con 0x55e862492400 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d267fc700 1 -- 172.21.15.5:0/392957610 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d38003d80 con 0x55e862492400 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.193+0000 7f3d267fc700 1 -- 172.21.15.5:0/392957610 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e040 con 0x55e862492400 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 -- 172.21.15.5:0/392957610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 msgr2=0x55e8624a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007500 tx=0x7f3d38007530 comp rx=0 tx=0).stop 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 -- 172.21.15.5:0/392957610 shutdown_connections 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e862495f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a8e70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 --2- 172.21.15.5:0/392957610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e862495a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 -- 172.21.15.5:0/392957610 >> 172.21.15.5:0/392957610 conn(0x55e862407260 msgr2=0x55e8624af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 -- 172.21.15.5:0/392957610 shutdown_connections 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.194+0000 7f3d59532b80 1 -- 172.21.15.5:0/392957610 wait complete. 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 -- start start 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 Processor -- start 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e8624a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e8624a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8624a2410 con 0x55e862492d20 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8624a2580 con 0x55e86249dbd0 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.195+0000 7f3d59532b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8624a2740 con 0x55e862492400 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d59532b80 10 client.0 _ll_register_callbacks cb 0x55e8621df800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42576/0 (socket says 172.21.15.5:42576) 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 -- 172.21.15.5:0/4250746919 learned_addr learned my addr 172.21.15.5:0/4250746919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e8624a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3effd700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e8624a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 -- 172.21.15.5:0/4250746919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 msgr2=0x55e8624a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e8624a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 -- 172.21.15.5:0/4250746919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 msgr2=0x55e8624a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e8624a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 -- 172.21.15.5:0/4250746919 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8624488d0 con 0x55e862492400 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3effd700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e8624a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d3ffff700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009980 tx=0x7f3d3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e8dabffd42e768 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.196+0000 7f3d1effd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:58:26.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3802b070 con 0x55e862492400 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d38007e30 con 0x55e862492400 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3d380303e0 con 0x55e862492400 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d3800e070 con 0x55e862492400 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 10 client.27568 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e862237250 con 0x55e862492400 2023-09-23T04:58:26.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d38040080 con 0x55e862492400 2023-09-23T04:58:26.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f3d38019030 con 0x55e862492400 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 10 client.27568 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 10 client.27568 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 20 client.27568 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e862501590 0x55e862503980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 4 client.27568 mds_command: new command op to 24535 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.197+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 --> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8623deb30 con 0x55e862501590 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d3effd700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e862501590 0x55e862503980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d3effd700 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e862501590 0x55e862503980 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94c84a9d80160af server_cookie=a84adb51f3e11585 in_seq=0 out_seq=0 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d1effd700 10 client.27568 ms_handle_connect on v2:172.21.15.118:6836/704534117 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d1effd700 1 -- 172.21.15.5:0/4250746919 <== mds.3 v2:172.21.15.118:6836/704534117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8623deb30 con 0x55e862501590 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d1effd700 10 client.27568 handle_command_reply: tid=0 2023-09-23T04:58:26.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.198+0000 7f3d59532b80 1 client.27568 shutdown 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 msgr2=0x55e8624a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009980 tx=0x7f3d3800b330 comp rx=0 tx=0).stop 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 shutdown_connections 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e86249dbd0 0x55e8624a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55e862492400 0x55e8624a4640 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.118:6836/704534117,v1:172.21.15.118:6837/704534117] conn(0x55e862501590 0x55e862503980 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 --2- 172.21.15.5:0/4250746919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e862492d20 0x55e8624a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d1effd700 0 client.27568 ms_handle_reset on v2:172.21.15.118:6836/704534117 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 >> 172.21.15.5:0/4250746919 conn(0x55e862407260 msgr2=0x55e8624a9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 shutdown_connections 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 1 -- 172.21.15.5:0/4250746919 wait complete. 2023-09-23T04:58:26.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.199+0000 7f3d59532b80 20 client.27568 trim_cache size 0 max 16384 2023-09-23T04:58:26.205 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:58:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 msgr2=0x7f3d280a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 0x7f3d280a3bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d30003ad0 tx=0x7f3d300d34b0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 msgr2=0x7f3d401990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000ec10 tx=0x7f3d3000ef20 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 shutdown_connections 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3d280a1710 0x7f3d280a3bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3d4019da60 0x7f3d4019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d400690a0 0x7f3d401990c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 --2- 172.21.15.5:0/1629546448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d40106210 0x7f3d40199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 >> 172.21.15.5:0/1629546448 conn(0x7f3d40077ca0 msgr2=0x7f3d4010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 shutdown_connections 2023-09-23T04:58:26.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.200+0000 7f3d45bff700 1 -- 172.21.15.5:0/1629546448 wait complete. 2023-09-23T04:58:26.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-23T04:58:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 -- 172.21.15.5:0/1061829506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878101a90 msgr2=0x7f2878105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 --2- 172.21.15.5:0/1061829506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878101a90 0x7f2878105ad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f286c009990 tx=0x7f286c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 -- 172.21.15.5:0/1061829506 shutdown_connections 2023-09-23T04:58:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 --2- 172.21.15.5:0/1061829506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f287810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 --2- 172.21.15.5:0/1061829506 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f28781014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 --2- 172.21.15.5:0/1061829506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878101a90 0x7f2878105ad0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 -- 172.21.15.5:0/1061829506 >> 172.21.15.5:0/1061829506 conn(0x7f28780fc980 msgr2=0x7f28780feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 -- 172.21.15.5:0/1061829506 shutdown_connections 2023-09-23T04:58:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.485+0000 7f287ddff700 1 -- 172.21.15.5:0/1061829506 wait complete. 2023-09-23T04:58:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.486+0000 7f287ddff700 1 Processor -- start 2023-09-23T04:58:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.486+0000 7f287ddff700 1 -- start start 2023-09-23T04:58:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f2878073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878073ba0 0x7f28781a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a5890 con 0x7f2878073ba0 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a5a00 con 0x7f2878106200 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f287ddff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a5bc0 con 0x7f28781010e0 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f2878073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42590/0 (socket says 172.21.15.5:42590) 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 -- 172.21.15.5:0/3143252575 learned_addr learned my addr 172.21.15.5:0/3143252575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f2877fff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878073ba0 0x7f28781a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 -- 172.21.15.5:0/3143252575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 msgr2=0x7f2878073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f2878073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 -- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878073ba0 msgr2=0x7f28781a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878073ba0 0x7f28781a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f28777fe700 1 -- 172.21.15.5:0/3143252575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2870009c40 con 0x7f28781010e0 2023-09-23T04:58:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.487+0000 7f2876ffd700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f2878073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f28777fe700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f286800e9a0 tx=0x7f286800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f2874ff9700 1 -- 172.21.15.5:0/3143252575 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868009790 con 0x7f28781010e0 2023-09-23T04:58:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f2874ff9700 1 -- 172.21.15.5:0/3143252575 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286800ce80 con 0x7f28781010e0 2023-09-23T04:58:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c0095f0 con 0x7f28781010e0 2023-09-23T04:58:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f2874ff9700 1 -- 172.21.15.5:0/3143252575 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868018680 con 0x7f28781010e0 2023-09-23T04:58:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.488+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28781a61e0 con 0x7f28781010e0 2023-09-23T04:58:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.492+0000 7f2874ff9700 1 -- 172.21.15.5:0/3143252575 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2868010430 con 0x7f28781010e0 2023-09-23T04:58:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.493+0000 7f2891645b80 1 Processor -- start 2023-09-23T04:58:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.493+0000 7f2874ff9700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 0x7f28600a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.493+0000 7f2891645b80 1 -- start start 2023-09-23T04:58:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2874ff9700 1 -- 172.21.15.5:0/3143252575 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f2868014070 con 0x7f28781010e0 2023-09-23T04:58:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2877fff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 0x7f28600a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee132547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee1325fbd0 0x55ee13261fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee131bd460 con 0x55ee13254400 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee131aec20 con 0x55ee1325fbd0 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2891645b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee131ac540 con 0x55ee13254d20 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f28777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee132547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42592/0 (socket says 172.21.15.5:42592) 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2876ffd700 1 -- 172.21.15.5:0/2566872078 learned_addr learned my addr 172.21.15.5:0/2566872078 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.494+0000 7f2877fff700 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee1325fbd0 0x55ee13261fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2877fff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 0x7f28600a3c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2870009fd0 tx=0x7f287000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 -- 172.21.15.5:0/2566872078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee1325fbd0 msgr2=0x55ee13261fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee1325fbd0 0x55ee13261fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 -- 172.21.15.5:0/2566872078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 msgr2=0x55ee132547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee132547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 -- 172.21.15.5:0/2566872078 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee1320a8d0 con 0x55ee13254d20 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f2876ffd700 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f286c009990 tx=0x7f286c005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647c64cf338cc027 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f285dffb700 1 -- 172.21.15.5:0/2566872078 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c01d050 con 0x55ee13254d20 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f285dffb700 1 -- 172.21.15.5:0/2566872078 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c01e040 con 0x55ee13254d20 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.495+0000 7f285dffb700 1 -- 172.21.15.5:0/2566872078 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c005430 con 0x55ee13254d20 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 -- 172.21.15.5:0/2566872078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 msgr2=0x55ee1325f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f286c009990 tx=0x7f286c005ea0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 -- 172.21.15.5:0/2566872078 shutdown_connections 2023-09-23T04:58:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee1325fbd0 0x55ee13261fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee132547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 --2- 172.21.15.5:0/2566872078 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325f690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 -- 172.21.15.5:0/2566872078 >> 172.21.15.5:0/2566872078 conn(0x55ee131c9260 msgr2=0x55ee13250790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 -- 172.21.15.5:0/2566872078 shutdown_connections 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.496+0000 7f2891645b80 1 -- 172.21.15.5:0/2566872078 wait complete. 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.497+0000 7f2891645b80 1 -- start start 2023-09-23T04:58:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.497+0000 7f2891645b80 1 Processor -- start 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 0x55ee1325cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee131a7e30 con 0x55ee1325fbd0 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee13144710 con 0x55ee13254400 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee131443b0 con 0x55ee13254d20 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2891645b80 10 client.0 _ll_register_callbacks cb 0x55ee12fa1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32870/0 (socket says 172.21.15.5:32870) 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 -- 172.21.15.5:0/2189174511 learned_addr learned my addr 172.21.15.5:0/2189174511 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2877fff700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 0x55ee1325cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 -- 172.21.15.5:0/2189174511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 msgr2=0x55ee1325c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 -- 172.21.15.5:0/2189174511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 msgr2=0x55ee1325cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 0x55ee1325cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 -- 172.21.15.5:0/2189174511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287810e5f0 con 0x55ee13254400 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f2877fff700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 0x55ee1325cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.498+0000 7f28777fe700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f286800e970 tx=0x7f286809a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928a0bef80056396 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286809e370 con 0x55ee13254400 2023-09-23T04:58:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f28680a2770 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee13248250 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28680998d0 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2891645b80 10 client.27518 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55ee12fd2de0 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee13248250 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.499+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f28680cd390 con 0x55ee13254400 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 10 client.27518 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 10 client.27518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 20 client.27518 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee1326f610 0x55ee13271a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 4 client.27518 mds_command: new command op to 24554 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee131a0e10 con 0x55ee1326f610 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2876ffd700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee1326f610 0x55ee13271a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2876ffd700 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee1326f610 0x55ee13271a00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d19a3f640509e665 server_cookie=6797e060a9bc807 in_seq=0 out_seq=0 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.500+0000 7f2856ffd700 10 client.27518 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2856ffd700 1 -- 172.21.15.5:0/2189174511 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee131a0e10 con 0x55ee1326f610 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2856ffd700 10 client.27518 handle_command_reply: tid=0 2023-09-23T04:58:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 client.27518 shutdown 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 msgr2=0x55ee1325f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f286800e970 tx=0x7f286809a3e0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 shutdown_connections 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee1326f610 0x55ee13271a00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1325fbd0 0x55ee1325cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee1325f680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 --2- 172.21.15.5:0/2189174511 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee13254d20 0x55ee1325c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 >> 172.21.15.5:0/2189174511 conn(0x55ee131c9260 msgr2=0x55ee13250320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.501+0000 7f2856ffd700 0 client.27518 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:58:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.502+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 shutdown_connections 2023-09-23T04:58:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.502+0000 7f2891645b80 1 -- 172.21.15.5:0/2189174511 wait complete. 2023-09-23T04:58:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.502+0000 7f2891645b80 20 client.27518 trim_cache size 0 max 16384 2023-09-23T04:58:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.517+0000 7f2891645b80 1 Processor -- start 2023-09-23T04:58:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.517+0000 7f2891645b80 1 -- start start 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee1325f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee1325a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee13028920 con 0x55ee13254400 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee132586e0 con 0x55ee13254d20 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2891645b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee13255e40 con 0x55ee1325fbd0 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2877fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee1325a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32886/0 (socket says 172.21.15.5:32886) 2023-09-23T04:58:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.518+0000 7f2876ffd700 1 -- 172.21.15.5:0/363474687 learned_addr learned my addr 172.21.15.5:0/363474687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 -- 172.21.15.5:0/363474687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 msgr2=0x55ee1325a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee1325a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 -- 172.21.15.5:0/363474687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 msgr2=0x55ee1325f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee1325f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 -- 172.21.15.5:0/363474687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee12fbbc60 con 0x55ee13254d20 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2877fff700 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee1325a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f2876ffd700 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f286c009350 tx=0x7f286c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0dbeb5e4906797c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f285e7fc700 1 -- 172.21.15.5:0/363474687 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c01d070 con 0x55ee13254d20 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f285e7fc700 1 -- 172.21.15.5:0/363474687 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c00bdc0 con 0x55ee13254d20 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.519+0000 7f285e7fc700 1 -- 172.21.15.5:0/363474687 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0056a0 con 0x55ee13254d20 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- 172.21.15.5:0/363474687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 msgr2=0x55ee13259b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f286c009350 tx=0x7f286c01a040 comp rx=0 tx=0).stop 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- 172.21.15.5:0/363474687 shutdown_connections 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee1325a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254400 0x55ee1325f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 --2- 172.21.15.5:0/363474687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254d20 0x55ee13259b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- 172.21.15.5:0/363474687 >> 172.21.15.5:0/363474687 conn(0x55ee131c9260 msgr2=0x55ee13257b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- 172.21.15.5:0/363474687 shutdown_connections 2023-09-23T04:58:26.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- 172.21.15.5:0/363474687 wait complete. 2023-09-23T04:58:26.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.520+0000 7f2891645b80 1 -- start start 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 Processor -- start 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 0x55ee132645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee132c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee13264db0 con 0x55ee13254d20 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee13264f20 con 0x55ee13254400 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee132c29d0 con 0x55ee1325fbd0 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f2891645b80 10 client.0 _ll_register_callbacks cb 0x7f2868099d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.521+0000 7f28777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32892/0 (socket says 172.21.15.5:32892) 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 -- 172.21.15.5:0/3155003973 learned_addr learned my addr 172.21.15.5:0/3155003973 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2877fff700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee132c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2876ffd700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 0x55ee132645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 -- 172.21.15.5:0/3155003973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 msgr2=0x55ee132c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee132c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 -- 172.21.15.5:0/3155003973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 msgr2=0x55ee132645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 0x55ee132645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 -- 172.21.15.5:0/3155003973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28700091d0 con 0x55ee13254400 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2877fff700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee132c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2876ffd700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 0x55ee132645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f28777fe700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28680a2240 tx=0x7f2868010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53d5c844a18597e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868099ad0 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f286807c020 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.522+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286c01e040 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28680a2770 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 10 client.27524 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55ee1320a8d0 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c01e040 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2868099340 con 0x55ee13254400 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 10 client.27524 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 10 client.27524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2023-09-23T04:58:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 20 client.27524 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee132c3d40 0x55ee132c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 4 client.27524 mds_command: new command op to 24554 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 --> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee131a0b30 con 0x55ee132c3d40 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.523+0000 7f2876ffd700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee132c3d40 0x55ee132c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2876ffd700 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee132c3d40 0x55ee132c6130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f20fba4804e27b3e server_cookie=a9321a728c06f757 in_seq=0 out_seq=0 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2856ffd700 10 client.27524 ms_handle_connect on v2:172.21.15.69:6838/1851157393 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2856ffd700 1 -- 172.21.15.5:0/3155003973 <== mds.2 v2:172.21.15.69:6838/1851157393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee131a0b30 con 0x55ee132c3d40 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2856ffd700 10 client.27524 handle_command_reply: tid=0 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 client.27524 shutdown 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 msgr2=0x55ee13264070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28680a2240 tx=0x7f2868010a10 comp rx=0 tx=0).stop 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 shutdown_connections 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:6838/1851157393,v1:172.21.15.69:6839/1851157393] conn(0x55ee132c3d40 0x55ee132c6130 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ee1325fbd0 0x55ee132c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee13254400 0x55ee13264070 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 --2- 172.21.15.5:0/3155003973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee13254d20 0x55ee132645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 >> 172.21.15.5:0/3155003973 conn(0x55ee131c9260 msgr2=0x55ee1324f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2856ffd700 0 client.27524 ms_handle_reset on v2:172.21.15.69:6838/1851157393 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 shutdown_connections 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.524+0000 7f2891645b80 1 -- 172.21.15.5:0/3155003973 wait complete. 2023-09-23T04:58:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f2891645b80 20 client.27524 trim_cache size 0 max 16384 2023-09-23T04:58:26.531 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:58:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 msgr2=0x7f28600a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 0x7f28600a3c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2870009fd0 tx=0x7f287000b040 comp rx=0 tx=0).stop 2023-09-23T04:58:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 msgr2=0x7f2878073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f286800e9a0 tx=0x7f286800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 shutdown_connections 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f28600a1790 0x7f28600a3c50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2878073ba0 0x7f28781a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f28781010e0 0x7f2878073120 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 --2- 172.21.15.5:0/3143252575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2878106200 0x7f2878073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 >> 172.21.15.5:0/3143252575 conn(0x7f28780fc980 msgr2=0x7f28780fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 shutdown_connections 2023-09-23T04:58:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.525+0000 7f287ddff700 1 -- 172.21.15.5:0/3143252575 wait complete. 2023-09-23T04:58:26.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-23T04:58:26.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 -- 172.21.15.5:0/1263513875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e01010f0 msgr2=0x7f16e01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1263513875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e01010f0 0x7f16e01014d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009990 tx=0x7f16d0009ca0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 -- 172.21.15.5:0/1263513875 shutdown_connections 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1263513875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e0106210 0x7f16e010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1263513875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e01010f0 0x7f16e01014d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1263513875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0101aa0 0x7f16e0105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 -- 172.21.15.5:0/1263513875 >> 172.21.15.5:0/1263513875 conn(0x7f16e00fc9d0 msgr2=0x7f16e00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 -- 172.21.15.5:0/1263513875 shutdown_connections 2023-09-23T04:58:26.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.773+0000 7f16e49ff700 1 -- 172.21.15.5:0/1263513875 wait complete. 2023-09-23T04:58:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 Processor -- start 2023-09-23T04:58:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 -- start start 2023-09-23T04:58:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 0x7f16e0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 0x7f16e01a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e01a5820 con 0x7f16e0106210 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e01a5990 con 0x7f16e0073ba0 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.775+0000 7f16e49ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e01a5b50 con 0x7f16e01010f0 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42636/0 (socket says 172.21.15.5:42636) 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16de59c700 1 -- 172.21.15.5:0/1802261324 learned_addr learned my addr 172.21.15.5:0/1802261324 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 0x7f16e0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 0x7f16e01a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 -- 172.21.15.5:0/1802261324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 msgr2=0x7f16e0073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 -- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 msgr2=0x7f16e0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 0x7f16e0073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 -- 172.21.15.5:0/1802261324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00095f0 con 0x7f16e0073ba0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16de59c700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 0x7f16e0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.776+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 0x7f16e01a5190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16d800efc0 tx=0x7f16d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.777+0000 7f16cf7fe700 1 -- 172.21.15.5:0/1802261324 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8018070 con 0x7f16e0073ba0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.777+0000 7f16cf7fe700 1 -- 172.21.15.5:0/1802261324 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d800f040 con 0x7f16e0073ba0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.777+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e01a5dd0 con 0x7f16e0073ba0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.777+0000 7f16cf7fe700 1 -- 172.21.15.5:0/1802261324 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800cd40 con 0x7f16e0073ba0 2023-09-23T04:58:26.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.777+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e01a62e0 con 0x7f16e0073ba0 2023-09-23T04:58:26.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.781+0000 7f16cf7fe700 1 -- 172.21.15.5:0/1802261324 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f16d8003a00 con 0x7f16e0073ba0 2023-09-23T04:58:26.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.782+0000 7f16f82cfb80 1 Processor -- start 2023-09-23T04:58:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.782+0000 7f16f82cfb80 1 -- start start 2023-09-23T04:58:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a443fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a39f460 con 0x55b38a441bd0 2023-09-23T04:58:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a390c20 con 0x55b38a436400 2023-09-23T04:58:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a38e540 con 0x55b38a436d20 2023-09-23T04:58:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16ddd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16ddd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32930/0 (socket says 172.21.15.5:32930) 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42642/0 (socket says 172.21.15.5:42642) 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16ddd9b700 1 -- 172.21.15.5:0/1485162266 learned_addr learned my addr 172.21.15.5:0/1485162266 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a443fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.783+0000 7f16cf7fe700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 0x7f16c80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 -- 172.21.15.5:0/1485162266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 msgr2=0x55b38a4367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 -- 172.21.15.5:0/1485162266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 msgr2=0x55b38a443fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a443fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16cf7fe700 1 -- 172.21.15.5:0/1802261324 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f16d80cd250 con 0x7f16e0073ba0 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 -- 172.21.15.5:0/1485162266 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b38a3ec8d0 con 0x55b38a436d20 2023-09-23T04:58:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 0x7f16c80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16de59c700 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16d000b6f0 tx=0x7f16d00058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c79a19518f55da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16ccff9700 1 -- 172.21.15.5:0/1485162266 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d070 con 0x55b38a436d20 2023-09-23T04:58:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16ccff9700 1 -- 172.21.15.5:0/1485162266 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d001e040 con 0x55b38a436d20 2023-09-23T04:58:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.784+0000 7f16ccff9700 1 -- 172.21.15.5:0/1485162266 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004cb0 con 0x55b38a436d20 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 -- 172.21.15.5:0/1485162266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 msgr2=0x55b38a441690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16d000b6f0 tx=0x7f16d00058e0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16ded9d700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 0x7f16c80a3c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ca570 tx=0x7f16d8004cc0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 -- 172.21.15.5:0/1485162266 shutdown_connections 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a443fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/1485162266 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a441690 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 -- 172.21.15.5:0/1485162266 >> 172.21.15.5:0/1485162266 conn(0x55b38a3ab260 msgr2=0x55b38a432790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.785+0000 7f16f82cfb80 1 -- 172.21.15.5:0/1485162266 shutdown_connections 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.786+0000 7f16f82cfb80 1 -- 172.21.15.5:0/1485162266 wait complete. 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.786+0000 7f16f82cfb80 1 -- start start 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.786+0000 7f16f82cfb80 1 Processor -- start 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 0x55b38a4415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a441bd0 0x55b38a43eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a389e30 con 0x55b38a436d20 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a326710 con 0x55b38a441bd0 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a3263b0 con 0x55b38a436400 2023-09-23T04:58:26.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16f82cfb80 10 client.0 _ll_register_callbacks cb 0x55b38a183800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16ddd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 0x55b38a4415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16ddd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 0x55b38a4415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42648/0 (socket says 172.21.15.5:42648) 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41054/0 (socket says 172.21.15.5:41054) 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16ddd9b700 1 -- 172.21.15.5:0/792451097 learned_addr learned my addr 172.21.15.5:0/792451097 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.787+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a441bd0 0x55b38a43eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 -- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 msgr2=0x55b38a4415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 0x55b38a4415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 -- 172.21.15.5:0/792451097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a441bd0 msgr2=0x55b38a43eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a441bd0 0x55b38a43eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 -- 172.21.15.5:0/792451097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d4019040 con 0x55b38a436d20 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16de59c700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009ce0 tx=0x7f16d0022ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf60d1bd3d79900 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d030 con 0x55b38a436d20 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b38a42a250 con 0x55b38a436d20 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f16d000f460 con 0x55b38a436d20 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.788+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d0005bc0 con 0x55b38a436d20 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 10 client.17541 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f16e010e600 con 0x55b38a436d20 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b38a42a250 con 0x55b38a436d20 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f16d002c400 con 0x55b38a436d20 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 10 client.17541 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 20 client.17541 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a451250 0x55b38a453670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 4 client.17541 mds_command: new command op to 24559 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b38a382e10 con 0x55b38a451250 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.789+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a451250 0x55b38a453670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.790+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a451250 0x55b38a453670 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8ee09ad63c5f33f server_cookie=a64d3f2999a93645 in_seq=0 out_seq=0 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.790+0000 7f16c5ffb700 10 client.17541 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.790+0000 7f16c5ffb700 1 -- 172.21.15.5:0/792451097 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b38a382e10 con 0x55b38a451250 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.790+0000 7f16c5ffb700 10 client.17541 handle_command_reply: tid=0 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 client.17541 shutdown 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 msgr2=0x55b38a43e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009ce0 tx=0x7f16d0022ae0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 shutdown_connections 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a451250 0x55b38a453670 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a441bd0 0x55b38a43eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16c5ffb700 0 client.17541 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436400 0x55b38a4415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/792451097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a43e820 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 >> 172.21.15.5:0/792451097 conn(0x55b38a3ab260 msgr2=0x55b38a432320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 shutdown_connections 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 1 -- 172.21.15.5:0/792451097 wait complete. 2023-09-23T04:58:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.791+0000 7f16f82cfb80 20 client.17541 trim_cache size 0 max 16384 2023-09-23T04:58:26.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 Processor -- start 2023-09-23T04:58:26.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 -- start start 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a439b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a43baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a20a920 con 0x55b38a441bd0 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a382e10 con 0x55b38a436400 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a433620 con 0x55b38a436d20 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.807+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42650/0 (socket says 172.21.15.5:42650) 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 -- 172.21.15.5:0/297239003 learned_addr learned my addr 172.21.15.5:0/297239003 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a439b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a43baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 -- 172.21.15.5:0/297239003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 msgr2=0x55b38a439b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a439b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 -- 172.21.15.5:0/297239003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 msgr2=0x55b38a43baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a43baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 -- 172.21.15.5:0/297239003 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b38a1b4de0 con 0x55b38a436d20 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a43baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16de59c700 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f16d0006010 tx=0x7f16d0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3db1e9e5e4120 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16cd7fa700 1 -- 172.21.15.5:0/297239003 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d040 con 0x55b38a436d20 2023-09-23T04:58:26.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16cd7fa700 1 -- 172.21.15.5:0/297239003 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d000b310 con 0x55b38a436d20 2023-09-23T04:58:26.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.808+0000 7f16cd7fa700 1 -- 172.21.15.5:0/297239003 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004500 con 0x55b38a436d20 2023-09-23T04:58:26.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 -- 172.21.15.5:0/297239003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 msgr2=0x55b38a43a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f16d0006010 tx=0x7f16d0009310 comp rx=0 tx=0).stop 2023-09-23T04:58:26.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 -- 172.21.15.5:0/297239003 shutdown_connections 2023-09-23T04:58:26.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a441bd0 0x55b38a43baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a439b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/297239003 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a436d20 0x55b38a43a0d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 -- 172.21.15.5:0/297239003 >> 172.21.15.5:0/297239003 conn(0x55b38a3ab260 msgr2=0x55b38a44d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 -- 172.21.15.5:0/297239003 shutdown_connections 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.809+0000 7f16f82cfb80 1 -- 172.21.15.5:0/297239003 wait complete. 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.810+0000 7f16f82cfb80 1 -- start start 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.810+0000 7f16f82cfb80 1 Processor -- start 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.810+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.810+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.810+0000 7f16f82cfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a441bd0 0x55b38a447120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a4a4040 con 0x55b38a436d20 2023-09-23T04:58:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a4a4180 con 0x55b38a436400 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16f82cfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38a4a42f0 con 0x55b38a441bd0 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16f82cfb80 10 client.0 _ll_register_callbacks cb 0x7f16d4011700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16ddd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41066/0 (socket says 172.21.15.5:41066) 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 -- 172.21.15.5:0/3036376540 learned_addr learned my addr 172.21.15.5:0/3036376540 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a441bd0 0x55b38a447120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 -- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a441bd0 msgr2=0x55b38a447120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a441bd0 0x55b38a447120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 -- 172.21.15.5:0/3036376540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 msgr2=0x55b38a4466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.811+0000 7f16de59c700 1 -- 172.21.15.5:0/3036376540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e010e600 con 0x55b38a436d20 2023-09-23T04:58:26.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16de59c700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f16d002c8b0 tx=0x7f16d00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a49dfa5270aa11 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:26.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:26.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d001d040 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b38a382b30 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f16d0005040 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d002cb80 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16f82cfb80 10 client.17547 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.812+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55b38a19dc60 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b38a382b30 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f16d0004ca0 con 0x55b38a436d20 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 10 client.17547 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 20 client.17547 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a4a5730 0x55b38a4a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 4 client.17547 mds_command: new command op to 24559 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 --> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b38a1d5f70 con 0x55b38a4a5730 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a4a5730 0x55b38a4a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.813+0000 7f16ddd9b700 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a4a5730 0x55b38a4a7b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9586f21c46e8f6a9 server_cookie=551c31f407b7aa6e in_seq=0 out_seq=0 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16c5ffb700 10 client.17547 ms_handle_connect on v2:172.21.15.118:6838/430957039 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16c5ffb700 1 -- 172.21.15.5:0/3036376540 <== mds.1 v2:172.21.15.118:6838/430957039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b38a1d5f70 con 0x55b38a4a5730 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16c5ffb700 10 client.17547 handle_command_reply: tid=0 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 client.17547 shutdown 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 msgr2=0x55b38a446be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f16d002c8b0 tx=0x7f16d00058e0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 shutdown_connections 2023-09-23T04:58:26.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55b38a441bd0 0x55b38a447120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.118:6838/430957039,v1:172.21.15.118:6839/430957039] conn(0x55b38a4a5730 0x55b38a4a7b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b38a436400 0x55b38a4466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 --2- 172.21.15.5:0/3036376540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b38a436d20 0x55b38a446be0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16c5ffb700 0 client.17547 ms_handle_reset on v2:172.21.15.118:6838/430957039 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 >> 172.21.15.5:0/3036376540 conn(0x55b38a3ab260 msgr2=0x55b38a439b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 shutdown_connections 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 1 -- 172.21.15.5:0/3036376540 wait complete. 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.814+0000 7f16f82cfb80 20 client.17547 trim_cache size 0 max 16384 2023-09-23T04:58:26.819 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 msgr2=0x7f16c80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 0x7f16c80a3c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ca570 tx=0x7f16d8004cc0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 msgr2=0x7f16e01a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 0x7f16e01a5190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f16d800efc0 tx=0x7f16d800c3a0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 shutdown_connections 2023-09-23T04:58:26.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f16c80a1790 0x7f16c80a3c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16e0073ba0 0x7f16e01a5190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f16e01010f0 0x7f16e0073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 --2- 172.21.15.5:0/1802261324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16e0106210 0x7f16e0073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 >> 172.21.15.5:0/1802261324 conn(0x7f16e00fc9d0 msgr2=0x7f16e0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 shutdown_connections 2023-09-23T04:58:26.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:26.815+0000 7f16e49ff700 1 -- 172.21.15.5:0/1802261324 wait complete. 2023-09-23T04:58:26.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:26 smithi005 ceph-mon[97638]: pgmap v379: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-09-23T04:58:26.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-23T04:58:26.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 615226 4.33 MB/sec execute 542 sec latency 17.317 ms 2023-09-23T04:58:26.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:26 smithi069 ceph-mon[133931]: pgmap v379: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-09-23T04:58:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/1319898125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54069080 msgr2=0x7f9e54069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/1319898125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54069080 0x7f9e54069460 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9e500098f0 tx=0x7f9e50009c00 comp rx=0 tx=0).stop 2023-09-23T04:58:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/1319898125 shutdown_connections 2023-09-23T04:58:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/1319898125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54106220 0x7f9e5410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/1319898125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54069080 0x7f9e54069460 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/1319898125 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e54069a30 0x7f9e54105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/1319898125 >> 172.21.15.5:0/1319898125 conn(0x7f9e54077ca0 msgr2=0x7f9e540780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/1319898125 shutdown_connections 2023-09-23T04:58:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.072+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/1319898125 wait complete. 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.073+0000 7f9e5b5ff700 1 Processor -- start 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 -- start start 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106220 0x7f9e54197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e5419b9c0 0x7f9e5419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419c440 con 0x7f9e54106220 2023-09-23T04:58:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419c5b0 con 0x7f9e54069080 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5b5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5419c770 con 0x7f9e5419b9c0 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5939b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5939b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:32996/0 (socket says 172.21.15.5:32996) 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e5939b700 1 -- 172.21.15.5:0/3112116178 learned_addr learned my addr 172.21.15.5:0/3112116178 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e5419b9c0 0x7f9e5419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.074+0000 7f9e58b9a700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106220 0x7f9e54197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 -- 172.21.15.5:0/3112116178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e5419b9c0 msgr2=0x7f9e5419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e5419b9c0 0x7f9e5419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 -- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106220 msgr2=0x7f9e54197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106220 0x7f9e54197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 -- 172.21.15.5:0/3112116178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c009c40 con 0x7f9e54069080 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009810 tx=0x7f9e5000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e4a7fc700 1 -- 172.21.15.5:0/3112116178 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001c070 con 0x7f9e54069080 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e50009520 con 0x7f9e54069080 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e4a7fc700 1 -- 172.21.15.5:0/3112116178 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5000bb40 con 0x7f9e54069080 2023-09-23T04:58:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.075+0000 7f9e4a7fc700 1 -- 172.21.15.5:0/3112116178 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004930 con 0x7f9e54069080 2023-09-23T04:58:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.076+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e541a3f00 con 0x7f9e54069080 2023-09-23T04:58:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.079+0000 7f9e4a7fc700 1 -- 172.21.15.5:0/3112116178 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f9e50020070 con 0x7f9e54069080 2023-09-23T04:58:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.080+0000 7f9e6ef97b80 1 Processor -- start 2023-09-23T04:58:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.080+0000 7f9e6ef97b80 1 -- start start 2023-09-23T04:58:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c86460 con 0x5599e6d2cfe0 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c77c20 con 0x5599e6d1d400 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c75540 con 0x5599e6d1dd20 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e4a7fc700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 0x7f9e400a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33010/0 (socket says 172.21.15.5:33010) 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e59b9c700 1 -- 172.21.15.5:0/3152727121 learned_addr learned my addr 172.21.15.5:0/3152727121 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.081+0000 7f9e58b9a700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e4a7fc700 1 -- 172.21.15.5:0/3112116178 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9e500ce430 con 0x7f9e54069080 2023-09-23T04:58:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 0x7f9e400a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 -- 172.21.15.5:0/3152727121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 msgr2=0x5599e6d1d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 -- 172.21.15.5:0/3152727121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d2cfe0 msgr2=0x5599e6d2f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 -- 172.21.15.5:0/3152727121 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e6cd38d0 con 0x5599e6d1dd20 2023-09-23T04:58:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2caa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e500cb6f0 tx=0x7f9e5006dfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1990087a183479f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/3152727121 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5009afa0 con 0x5599e6d1dd20 2023-09-23T04:58:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 0x7f9e400a3bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009fd0 tx=0x7f9e4c004b40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/3152727121 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5009f0f0 con 0x5599e6d1dd20 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.082+0000 7f9e3f7fe700 1 -- 172.21.15.5:0/3152727121 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a2e10 con 0x5599e6d1dd20 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3152727121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 msgr2=0x5599e6d2caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2caa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e500cb6f0 tx=0x7f9e5006dfc0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3152727121 shutdown_connections 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d1d400 0x5599e6d1d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3152727121 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2caa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3152727121 >> 172.21.15.5:0/3152727121 conn(0x5599e6c92260 msgr2=0x5599e6d19790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.083+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3152727121 shutdown_connections 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.084+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3152727121 wait complete. 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.084+0000 7f9e6ef97b80 1 -- start start 2023-09-23T04:58:27.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.084+0000 7f9e6ef97b80 1 Processor -- start 2023-09-23T04:58:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d2d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c70e30 con 0x5599e6d1d400 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c0d710 con 0x5599e6d2cfe0 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6c0d3b0 con 0x5599e6d1dd20 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e6ef97b80 10 client.0 _ll_register_callbacks cb 0x5599e6a6a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33020/0 (socket says 172.21.15.5:33020) 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.085+0000 7f9e59b9c700 1 -- 172.21.15.5:0/1695241941 learned_addr learned my addr 172.21.15.5:0/1695241941 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d2d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e58b9a700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 -- 172.21.15.5:0/1695241941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 msgr2=0x5599e6d2d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 -- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 msgr2=0x5599e6d2d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d2d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 -- 172.21.15.5:0/1695241941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c00e9b0 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e58b9a700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d2d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c019000 tx=0x7f9e4c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65bfeb6a9f2c2ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00e040 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9e4c01a670 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.086+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e6d11250 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e4c01a830 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 10 client.27554 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f9e541a3f00 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599e6d11250 con 0x5599e6d2cfe0 2023-09-23T04:58:27.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e4c025c80 con 0x5599e6d2cfe0 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 10 client.27554 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 10 client.27554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 20 client.27554 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d3c6c0 0x5599e6d3eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 4 client.27554 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.087+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e6c69e10 con 0x5599e6d3c6c0 2023-09-23T04:58:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.088+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d3c6c0 0x5599e6d3eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.088+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d3c6c0 0x5599e6d3eae0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1268a2ed5d9ef5b0 server_cookie=95fcf1e7caea0a75 in_seq=0 out_seq=0 2023-09-23T04:58:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.088+0000 7f9e3cff9700 10 client.27554 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e3cff9700 1 -- 172.21.15.5:0/1695241941 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599e6c69e10 con 0x5599e6d3c6c0 2023-09-23T04:58:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e3cff9700 10 client.27554 handle_command_reply: tid=0 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 client.27554 shutdown 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 msgr2=0x5599e6d2de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c019000 tx=0x7f9e4c004ad0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 shutdown_connections 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d3c6c0 0x5599e6d3eae0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d2de40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d2d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e3cff9700 0 client.27554 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1695241941 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d2d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.089+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 >> 172.21.15.5:0/1695241941 conn(0x5599e6c92260 msgr2=0x5599e6d19320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.090+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 shutdown_connections 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.090+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1695241941 wait complete. 2023-09-23T04:58:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.090+0000 7f9e6ef97b80 20 client.27554 trim_cache size 0 max 16384 2023-09-23T04:58:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 Processor -- start 2023-09-23T04:58:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 -- start start 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 0x5599e6d2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1dd20 0x5599e6d37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d29240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6af1920 con 0x5599e6d1dd20 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6d1b080 con 0x5599e6d2cfe0 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.105+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6d13860 con 0x5599e6d1d400 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d29240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d29240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33024/0 (socket says 172.21.15.5:33024) 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e59b9c700 1 -- 172.21.15.5:0/1013151199 learned_addr learned my addr 172.21.15.5:0/1013151199 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e58b9a700 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1dd20 0x5599e6d37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 0x5599e6d2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 -- 172.21.15.5:0/1013151199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 msgr2=0x5599e6d29240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d29240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 -- 172.21.15.5:0/1013151199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1dd20 msgr2=0x5599e6d37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1dd20 0x5599e6d37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.106+0000 7f9e5939b700 1 -- 172.21.15.5:0/1013151199 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e6a9bde0 con 0x5599e6d1d400 2023-09-23T04:58:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.107+0000 7f9e5939b700 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 0x5599e6d2dad0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e500a3460 tx=0x7f9e500c5610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a19d5efc3f30a65 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.107+0000 7f9e3ffff700 1 -- 172.21.15.5:0/1013151199 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500ce120 con 0x5599e6d1d400 2023-09-23T04:58:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.107+0000 7f9e3ffff700 1 -- 172.21.15.5:0/1013151199 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e500a10b0 con 0x5599e6d1d400 2023-09-23T04:58:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.107+0000 7f9e3ffff700 1 -- 172.21.15.5:0/1013151199 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5009b200 con 0x5599e6d1d400 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1013151199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 msgr2=0x5599e6d2dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 0x5599e6d2dad0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e500a3460 tx=0x7f9e500c5610 comp rx=0 tx=0).stop 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1013151199 shutdown_connections 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d29240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1d400 0x5599e6d2dad0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/1013151199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1dd20 0x5599e6d37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1013151199 >> 172.21.15.5:0/1013151199 conn(0x5599e6c92260 msgr2=0x5599e6d2d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1013151199 shutdown_connections 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.108+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/1013151199 wait complete. 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.109+0000 7f9e6ef97b80 1 -- start start 2023-09-23T04:58:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.109+0000 7f9e6ef97b80 1 Processor -- start 2023-09-23T04:58:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d30070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6d30d40 con 0x5599e6d1d400 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6d8fa60 con 0x5599e6d2cfe0 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e6d8fc20 con 0x5599e6d1dd20 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e6ef97b80 10 client.0 _ll_register_callbacks cb 0x7f9e4c0093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e58b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e59b9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33040/0 (socket says 172.21.15.5:33040) 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e58b9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d305b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42724/0 (socket says 172.21.15.5:42724) 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e59b9c700 1 -- 172.21.15.5:0/3953561819 learned_addr learned my addr 172.21.15.5:0/3953561819 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.110+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d30070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 -- 172.21.15.5:0/3953561819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 msgr2=0x5599e6d305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d305b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 -- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 msgr2=0x5599e6d30070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d30070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 -- 172.21.15.5:0/3953561819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e6cd38d0 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e59b9c700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009920 tx=0x7f9e4c01ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec7a607cfafedca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c006000 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e5009f0f0 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.111+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9e4c019950 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e4c00ed00 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 10 client.27560 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5599e6a84c60 con 0x5599e6d2cfe0 2023-09-23T04:58:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e5009f0f0 con 0x5599e6d2cfe0 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9e4c02d040 con 0x5599e6d2cfe0 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 10 client.27560 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 10 client.27560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 20 client.27560 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d91350 0x5599e6d93770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 4 client.27560 mds_command: new command op to 24577 tid=0[{"prefix": "damage ls"}] 2023-09-23T04:58:27.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599e6c69b30 con 0x5599e6d91350 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.112+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d91350 0x5599e6d93770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e5939b700 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d91350 0x5599e6d93770 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c5b30ea1c8f348 server_cookie=6a0ba3ca9c31370 in_seq=0 out_seq=0 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e3cff9700 10 client.27560 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e3cff9700 1 -- 172.21.15.5:0/3953561819 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599e6c69b30 con 0x5599e6d91350 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e3cff9700 10 client.27560 handle_command_reply: tid=0 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 client.27560 shutdown 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 msgr2=0x5599e6d8f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009920 tx=0x7f9e4c01ac60 comp rx=0 tx=0).stop 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 shutdown_connections 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5599e6d91350 0x5599e6d93770 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5599e6d2cfe0 0x5599e6d8f4f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599e6d1d400 0x5599e6d30070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e3cff9700 0 client.27560 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 --2- 172.21.15.5:0/3953561819 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5599e6d1dd20 0x5599e6d305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 >> 172.21.15.5:0/3953561819 conn(0x5599e6c92260 msgr2=0x5599e6d2dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.113+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 shutdown_connections 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e6ef97b80 1 -- 172.21.15.5:0/3953561819 wait complete. 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e6ef97b80 20 client.27560 trim_cache size 0 max 16384 2023-09-23T04:58:27.118 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 msgr2=0x7f9e400a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 0x7f9e400a3bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009fd0 tx=0x7f9e4c004b40 comp rx=0 tx=0).stop 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 msgr2=0x7f9e54197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009810 tx=0x7f9e5000fa00 comp rx=0 tx=0).stop 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 shutdown_connections 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f9e400a1710 0x7f9e400a3bd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9e5419b9c0 0x7f9e5419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54069080 0x7f9e54197020 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 --2- 172.21.15.5:0/3112116178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106220 0x7f9e54197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:27.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 >> 172.21.15.5:0/3112116178 conn(0x7f9e54077ca0 msgr2=0x7f9e540fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:27.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.114+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 shutdown_connections 2023-09-23T04:58:27.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:27.115+0000 7f9e5b5ff700 1 -- 172.21.15.5:0/3112116178 wait complete. 2023-09-23T04:58:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:26 smithi118 ceph-mon[134297]: pgmap v379: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2023-09-23T04:58:27.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 616129 4.34 MB/sec execute 543 sec latency 350.580 ms 2023-09-23T04:58:28.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-23T04:58:28.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-23T04:58:28.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 -- 172.21.15.5:0/2404159897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc10d700 msgr2=0x7ff9fc10fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 --2- 172.21.15.5:0/2404159897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc10d700 0x7ff9fc10fb30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009f90 tx=0x7ff9f400b330 comp rx=0 tx=0).stop 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 -- 172.21.15.5:0/2404159897 shutdown_connections 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 --2- 172.21.15.5:0/2404159897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc10d700 0x7ff9fc10fb30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 --2- 172.21.15.5:0/2404159897 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 0x7ff9fc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 --2- 172.21.15.5:0/2404159897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc073b10 0x7ff9fc10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 -- 172.21.15.5:0/2404159897 >> 172.21.15.5:0/2404159897 conn(0x7ff9fc0fc990 msgr2=0x7ff9fc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 -- 172.21.15.5:0/2404159897 shutdown_connections 2023-09-23T04:58:28.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.352+0000 7ffa013ff700 1 -- 172.21.15.5:0/2404159897 wait complete. 2023-09-23T04:58:28.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.353+0000 7ffa013ff700 1 Processor -- start 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.353+0000 7ffa013ff700 1 -- start start 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 0x7ff9fc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 0x7ff9fc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc19e280 con 0x7ff9fc10d700 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc19e3f0 con 0x7ff9fc073b10 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ffa013ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc19e5b0 con 0x7ff9fc0731f0 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33060/0 (socket says 172.21.15.5:33060) 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ff9fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 0x7ff9fc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ff9f3fff700 1 -- 172.21.15.5:0/101857221 learned_addr learned my addr 172.21.15.5:0/101857221 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.354+0000 7ff9faffd700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 0x7ff9fc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 -- 172.21.15.5:0/101857221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 msgr2=0x7ff9fc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 0x7ff9fc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 -- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 msgr2=0x7ff9fc19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 0x7ff9fc19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 -- 172.21.15.5:0/101857221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40095f0 con 0x7ff9fc073b10 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 0x7ff9fc19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400ec10 tx=0x7ff9e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f8ff9700 1 -- 172.21.15.5:0/101857221 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400cc90 con 0x7ff9fc073b10 2023-09-23T04:58:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f8ff9700 1 -- 172.21.15.5:0/101857221 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e40044e0 con 0x7ff9fc073b10 2023-09-23T04:58:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f4009c40 con 0x7ff9fc073b10 2023-09-23T04:58:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ff9f8ff9700 1 -- 172.21.15.5:0/101857221 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4010430 con 0x7ff9fc073b10 2023-09-23T04:58:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.355+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc1a5ef0 con 0x7ff9fc073b10 2023-09-23T04:58:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.359+0000 7ff9f8ff9700 1 -- 172.21.15.5:0/101857221 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7ff9e4003fb0 con 0x7ff9fc073b10 2023-09-23T04:58:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ffa14c8ab80 1 Processor -- start 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ff9f8ff9700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 0x7ff9e80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ffa14c8ab80 1 -- start start 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ff9f8ff9700 1 -- 172.21.15.5:0/101857221 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff9e4014070 con 0x7ff9fc073b10 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.360+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 0x7ff9e80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bac124e0 con 0x5573bacb4c50 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bac03ca0 con 0x5573baca9480 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bac015c0 con 0x5573baca9da0 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33068/0 (socket says 172.21.15.5:33068) 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9fb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 -- 172.21.15.5:0/3804398985 learned_addr learned my addr 172.21.15.5:0/3804398985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 -- 172.21.15.5:0/3804398985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 msgr2=0x5573bacb4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 -- 172.21.15.5:0/3804398985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 msgr2=0x5573bacb7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9faffd700 1 -- 172.21.15.5:0/3804398985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573bac5f950 con 0x5573baca9480 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.361+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 0x7ff9e80a3bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009790 tx=0x7ff9f4007c90 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009d80 tx=0x7ff9ec011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be19fae63efc860 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ff9f17fa700 1 -- 172.21.15.5:0/3804398985 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec004230 con 0x5573baca9480 2023-09-23T04:58:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ff9f17fa700 1 -- 172.21.15.5:0/3804398985 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ec004390 con 0x5573baca9480 2023-09-23T04:58:28.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ff9f17fa700 1 -- 172.21.15.5:0/3804398985 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0198c0 con 0x5573baca9480 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3804398985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 msgr2=0x5573baca9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.362+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009d80 tx=0x7ff9ec011720 comp rx=0 tx=0).stop 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3804398985 shutdown_connections 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573baca9860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3804398985 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3804398985 >> 172.21.15.5:0/3804398985 conn(0x5573bac1e2e0 msgr2=0x5573baca5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3804398985 shutdown_connections 2023-09-23T04:58:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3804398985 wait complete. 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 -- start start 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.363+0000 7ffa14c8ab80 1 Processor -- start 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573babfceb0 con 0x5573bacb4c50 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bab99790 con 0x5573baca9480 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bab99430 con 0x5573baca9da0 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ffa14c8ab80 10 client.0 _ll_register_callbacks cb 0x5573ba9f6880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33084/0 (socket says 172.21.15.5:33084) 2023-09-23T04:58:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ff9faffd700 1 -- 172.21.15.5:0/3714496512 learned_addr learned my addr 172.21.15.5:0/3714496512 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.364+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 -- 172.21.15.5:0/3714496512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 msgr2=0x5573bacb18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 -- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 msgr2=0x5573bacb2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 -- 172.21.15.5:0/3714496512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0099e0 con 0x5573baca9480 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7017a12c7e81d77 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9e27fc700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec005450 con 0x5573baca9480 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573bac9d2d0 con 0x5573baca9480 2023-09-23T04:58:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff9ec005ce0 con 0x5573baca9480 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ec009ed0 con 0x5573baca9480 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ffa14c8ab80 10 client.27572 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.365+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5573baa27e60 con 0x5573baca9480 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573bac9d2d0 con 0x5573baca9480 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9ec021040 con 0x5573baca9480 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 10 client.27572 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 10 client.27572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 20 client.27572 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bacc4540 0x5573bacc6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 4 client.27572 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573babf5e90 con 0x5573bacc4540 2023-09-23T04:58:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.366+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bacc4540 0x5573bacc6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bacc4540 0x5573bacc6960 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed3df27f470ace5 server_cookie=e1fbe8193a999c2a in_seq=0 out_seq=0 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ff9e27fc700 10 client.27572 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ff9e27fc700 1 -- 172.21.15.5:0/3714496512 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5573babf5e90 con 0x5573bacc4540 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ff9e27fc700 10 client.27572 handle_command_reply: tid=0 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ffa14c8ab80 1 client.27572 shutdown 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 msgr2=0x5573bacb46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.367+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec0038b0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 shutdown_connections 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bacc4540 0x5573bacc6960 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573bacb4c50 0x5573bacb2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573baca9480 0x5573bacb46e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3714496512 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ff9e27fc700 0 client.27572 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 >> 172.21.15.5:0/3714496512 conn(0x5573bac1e2e0 msgr2=0x5573baca53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 shutdown_connections 2023-09-23T04:58:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3714496512 wait complete. 2023-09-23T04:58:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.368+0000 7ffa14c8ab80 20 client.27572 trim_cache size 0 max 16384 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.385+0000 7ffa14c8ab80 1 Processor -- start 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.385+0000 7ffa14c8ab80 1 -- start start 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573baa7d9a0 con 0x5573baca9480 2023-09-23T04:58:28.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bac9f940 con 0x5573bacb4c50 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bac9d1d0 con 0x5573baca9da0 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41154/0 (socket says 172.21.15.5:41154) 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 -- 172.21.15.5:0/3692954228 learned_addr learned my addr 172.21.15.5:0/3692954228 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 -- 172.21.15.5:0/3692954228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 msgr2=0x5573bacb4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 -- 172.21.15.5:0/3692954228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 msgr2=0x5573bacb05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacb05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9faffd700 1 -- 172.21.15.5:0/3692954228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573baccb570 con 0x5573baca9480 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:28.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.386+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ff9faffd700 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec006010 tx=0x7ff9ec011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953f60441297044a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ff9f1ffb700 1 -- 172.21.15.5:0/3692954228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec009ed0 con 0x5573baca9480 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ff9f1ffb700 1 -- 172.21.15.5:0/3692954228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ec0039d0 con 0x5573baca9480 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ff9f1ffb700 1 -- 172.21.15.5:0/3692954228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec005720 con 0x5573baca9480 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3692954228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 msgr2=0x5573bacbb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.387+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec006010 tx=0x7ff9ec011ba0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3692954228 shutdown_connections 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacbb7a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/3692954228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacb4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3692954228 >> 172.21.15.5:0/3692954228 conn(0x5573bac1e2e0 msgr2=0x5573bacafeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3692954228 shutdown_connections 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/3692954228 wait complete. 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 -- start start 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.388+0000 7ffa14c8ab80 1 Processor -- start 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bad184e0 con 0x5573baca9480 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bad18650 con 0x5573bacb4c50 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573bad18810 con 0x5573baca9da0 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ffa14c8ab80 10 client.0 _ll_register_callbacks cb 0x7ff9e4009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.389+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41162/0 (socket says 172.21.15.5:41162) 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 -- 172.21.15.5:0/21932139 learned_addr learned my addr 172.21.15.5:0/21932139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9fb7fe700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 -- 172.21.15.5:0/21932139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 msgr2=0x5573bacba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 -- 172.21.15.5:0/21932139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 msgr2=0x5573bacba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 -- 172.21.15.5:0/21932139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573baa27e60 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9faffd700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec005e50 tx=0x7ff9ec0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45daf5fb3d2bd9ce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9e27fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec003e30 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573babf5bb0 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.390+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7ff9ec000e40 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ec023a80 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 10 client.17580 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5573baa10ce0 con 0x5573baca9480 2023-09-23T04:58:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573babf5bb0 con 0x5573baca9480 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9ec011790 con 0x5573baca9480 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 10 client.17580 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 20 client.17580 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bad19b90 0x5573bad1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 4 client.17580 mds_command: new command op to 24577 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5573baa48ff0 con 0x5573bad19b90 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.391+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bad19b90 0x5573bad1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.392+0000 7ff9f3fff700 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bad19b90 0x5573bad1bfb0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae87004d88307f1d server_cookie=970af53d44abc2c1 in_seq=0 out_seq=0 2023-09-23T04:58:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.392+0000 7ff9e27fc700 10 client.17580 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.393+0000 7ff9e27fc700 1 -- 172.21.15.5:0/21932139 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5573baa48ff0 con 0x5573bad19b90 2023-09-23T04:58:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.393+0000 7ff9e27fc700 10 client.17580 handle_command_reply: tid=0 2023-09-23T04:58:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.393+0000 7ffa14c8ab80 1 client.17580 shutdown 2023-09-23T04:58:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.393+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 msgr2=0x5573bacb9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.393+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec005e50 tx=0x7ff9ec0096d0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 shutdown_connections 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573bad19b90 0x5573bad1bfb0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573bacb4c50 0x5573bacba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573baca9480 0x5573bacb9e90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 --2- 172.21.15.5:0/21932139 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573baca9da0 0x5573bacba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ff9e27fc700 0 client.17580 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 >> 172.21.15.5:0/21932139 conn(0x5573bac1e2e0 msgr2=0x5573bacab410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 shutdown_connections 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 1 -- 172.21.15.5:0/21932139 wait complete. 2023-09-23T04:58:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.394+0000 7ffa14c8ab80 20 client.17580 trim_cache size 0 max 16384 2023-09-23T04:58:28.399 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:58:28.399 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-09-23T04:58:28.399 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T04:58:28.399 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-09-23T04:58:28.399 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 msgr2=0x7ff9e80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 0x7ff9e80a3bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009790 tx=0x7ff9f4007c90 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 msgr2=0x7ff9fc1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400ec10 tx=0x7ff9e400ef20 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 shutdown_connections 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7ff9e80a1710 0x7ff9e80a3bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9fc10d700 0x7ff9fc19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7ff9fc0731f0 0x7ff9fc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 --2- 172.21.15.5:0/101857221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9fc073b10 0x7ff9fc1997e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.395+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 >> 172.21.15.5:0/101857221 conn(0x7ff9fc0fc990 msgr2=0x7ff9fc105160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.396+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 shutdown_connections 2023-09-23T04:58:28.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.396+0000 7ffa013ff700 1 -- 172.21.15.5:0/101857221 wait complete. 2023-09-23T04:58:28.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:58:28.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2252613099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec1020c0 msgr2=0x7f6cec1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2252613099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec1020c0 0x7f6cec1024a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b0a0 tx=0x7f6cdc00b3b0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2252613099 shutdown_connections 2023-09-23T04:58:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2252613099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec10b490 0x7f6cec10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2252613099 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec1020c0 0x7f6cec1024a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2252613099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2252613099 >> 172.21.15.5:0/2252613099 conn(0x7f6cec076f30 msgr2=0x7f6cec077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2252613099 shutdown_connections 2023-09-23T04:58:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.677+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2252613099 wait complete. 2023-09-23T04:58:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.678+0000 7f6cf09ff700 1 Processor -- start 2023-09-23T04:58:28.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 -- start start 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec10b490 0x7f6cec19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19e310 con 0x7f6cec1020c0 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19e480 con 0x7f6cec1029e0 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cf09ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19e640 con 0x7f6cec10b490 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41174/0 (socket says 172.21.15.5:41174) 2023-09-23T04:58:28.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cea59c700 1 -- 172.21.15.5:0/2445959692 learned_addr learned my addr 172.21.15.5:0/2445959692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.679+0000 7f6cead9d700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec10b490 0x7f6cec19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 -- 172.21.15.5:0/2445959692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec10b490 msgr2=0x7f6cec19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec10b490 0x7f6cec19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 -- 172.21.15.5:0/2445959692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 msgr2=0x7f6cec199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 -- 172.21.15.5:0/2445959692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc009d00 con 0x7f6cec1020c0 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cea59c700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b070 tx=0x7f6cdc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cdb7fe700 1 -- 172.21.15.5:0/2445959692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00e040 con 0x7f6cec1020c0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.680+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec19e8c0 con 0x7f6cec1020c0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.681+0000 7f6cdb7fe700 1 -- 172.21.15.5:0/2445959692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cdc009480 con 0x7f6cec1020c0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.681+0000 7f6cdb7fe700 1 -- 172.21.15.5:0/2445959692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc01b6c0 con 0x7f6cec1020c0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.681+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1a6160 con 0x7f6cec1020c0 2023-09-23T04:58:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.684+0000 7f6cdb7fe700 1 -- 172.21.15.5:0/2445959692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f6cdc01d470 con 0x7f6cec1020c0 2023-09-23T04:58:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.687+0000 7f6cdb7fe700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 0x7f6cd40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.687+0000 7f6d042d8b80 1 Processor -- start 2023-09-23T04:58:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.687+0000 7f6cdb7fe700 1 -- 172.21.15.5:0/2445959692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6cdc0ce6c0 con 0x7f6cec1020c0 2023-09-23T04:58:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.687+0000 7f6d042d8b80 1 -- start start 2023-09-23T04:58:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.687+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 0x7f6cd40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5d20 0x5633da5f0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 0x5633da5f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da54e460 con 0x5633da5e5d20 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da53fc20 con 0x5633da5f0bd0 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da53d540 con 0x5633da5e5400 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42784/0 (socket says 172.21.15.5:42784) 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 -- 172.21.15.5:0/3144519558 learned_addr learned my addr 172.21.15.5:0/3144519558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cead9d700 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 0x5633da5f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 -- 172.21.15.5:0/3144519558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 msgr2=0x5633da5f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 0x5633da5f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 -- 172.21.15.5:0/3144519558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5d20 msgr2=0x5633da5f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5d20 0x5633da5f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cea59c700 1 -- 172.21.15.5:0/3144519558 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633da59b8d0 con 0x5633da5e5400 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 0x7f6cd40a3bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6cec19a940 tx=0x7f6ce0009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.688+0000 7f6cead9d700 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 0x5633da5f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.689+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0a3000 tx=0x7f6cdc0976a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883609aa8693620c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.689+0000 7f6cd8ff9700 1 -- 172.21.15.5:0/3144519558 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc09f170 con 0x5633da5e5400 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.689+0000 7f6cd8ff9700 1 -- 172.21.15.5:0/3144519558 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cdc09b230 con 0x5633da5e5400 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.689+0000 7f6cd8ff9700 1 -- 172.21.15.5:0/3144519558 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc01d980 con 0x5633da5e5400 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3144519558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 msgr2=0x5633da5e57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0a3000 tx=0x7f6cdc0976a0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3144519558 shutdown_connections 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5f0bd0 0x5633da5f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5400 0x5633da5e57e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3144519558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5d20 0x5633da5f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3144519558 >> 172.21.15.5:0/3144519558 conn(0x5633da55a260 msgr2=0x5633da5e1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3144519558 shutdown_connections 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3144519558 wait complete. 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.690+0000 7f6d042d8b80 1 -- start start 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.691+0000 7f6d042d8b80 1 Processor -- start 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.691+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.691+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da538e30 con 0x5633da5e5400 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da4d5710 con 0x5633da5e5d20 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da4d53b0 con 0x5633da5f0bd0 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6d042d8b80 10 client.0 _ll_register_callbacks cb 0x5633da332800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41198/0 (socket says 172.21.15.5:41198) 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 -- 172.21.15.5:0/3659118029 learned_addr learned my addr 172.21.15.5:0/3659118029 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cead9d700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 -- 172.21.15.5:0/3659118029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 msgr2=0x5633da5ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5ee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 -- 172.21.15.5:0/3659118029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 msgr2=0x5633da5ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.692+0000 7f6cea59c700 1 -- 172.21.15.5:0/3659118029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1a6160 con 0x5633da5e5400 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cea59c700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09b1c0 tx=0x7f6cdc01d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5cd9612a16dd34 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0a3530 con 0x5633da5e5400 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6cdc095e40 con 0x5633da5e5400 2023-09-23T04:58:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633da5d9250 con 0x5633da5e5400 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cdc09f170 con 0x5633da5e5400 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6d042d8b80 10 client.17589 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5633da363de0 con 0x5633da5e5400 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.693+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633da5d9250 con 0x5633da5e5400 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6cdc09a840 con 0x5633da5e5400 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 10 client.17589 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 20 client.17589 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da5fff20 0x5633da602310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 4 client.17589 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633da531e10 con 0x5633da5fff20 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.694+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da5fff20 0x5633da602310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.695+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da5fff20 0x5633da602310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee525919076a17d server_cookie=48a985a9c067b0a0 in_seq=0 out_seq=0 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.695+0000 7f6cd1ffb700 10 client.17589 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.695+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/3659118029 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633da531e10 con 0x5633da5fff20 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.695+0000 7f6cd1ffb700 10 client.17589 handle_command_reply: tid=0 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.695+0000 7f6d042d8b80 1 client.17589 shutdown 2023-09-23T04:58:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 msgr2=0x5633da5ed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09b1c0 tx=0x7f6cdc01d9d0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 shutdown_connections 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da5fff20 0x5633da602310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5ed950 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6cd1ffb700 0 client.17589 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/3659118029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 >> 172.21.15.5:0/3659118029 conn(0x5633da55a260 msgr2=0x5633da5e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 shutdown_connections 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 1 -- 172.21.15.5:0/3659118029 wait complete. 2023-09-23T04:58:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.696+0000 7f6d042d8b80 20 client.17589 trim_cache size 0 max 16384 2023-09-23T04:58:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 Processor -- start 2023-09-23T04:58:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 -- start start 2023-09-23T04:58:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 0x5633da5e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 0x5633da5eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da3b9920 con 0x5633da5f0bd0 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da5e6c60 con 0x5633da5e5400 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.717+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da5db860 con 0x5633da5e5d20 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42808/0 (socket says 172.21.15.5:42808) 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 -- 172.21.15.5:0/397147198 learned_addr learned my addr 172.21.15.5:0/397147198 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6cead9d700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 0x5633da5eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6cea59c700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 0x5633da5e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 -- 172.21.15.5:0/397147198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 msgr2=0x5633da5e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 0x5633da5e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 -- 172.21.15.5:0/397147198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 msgr2=0x5633da5eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 0x5633da5eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 -- 172.21.15.5:0/397147198 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633da34cc60 con 0x5633da5e5d20 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6cea59c700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 0x5633da5e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6cead9d700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 0x5633da5eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.718+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5633da602bf0 tx=0x7f6ce0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47640536d195b514 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.719+0000 7f6cd97fa700 1 -- 172.21.15.5:0/397147198 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0024040 con 0x5633da5e5d20 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.719+0000 7f6cd97fa700 1 -- 172.21.15.5:0/397147198 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce00262b0 con 0x5633da5e5d20 2023-09-23T04:58:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.719+0000 7f6cd97fa700 1 -- 172.21.15.5:0/397147198 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce001e650 con 0x5633da5e5d20 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.719+0000 7f6d042d8b80 1 -- 172.21.15.5:0/397147198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 msgr2=0x5633da5ee200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.719+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5633da602bf0 tx=0x7f6ce0014000 comp rx=0 tx=0).stop 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 -- 172.21.15.5:0/397147198 shutdown_connections 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5f0bd0 0x5633da5eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5400 0x5633da5e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/397147198 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5e5d20 0x5633da5ee200 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 -- 172.21.15.5:0/397147198 >> 172.21.15.5:0/397147198 conn(0x5633da55a260 msgr2=0x5633da5ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 -- 172.21.15.5:0/397147198 shutdown_connections 2023-09-23T04:58:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 -- 172.21.15.5:0/397147198 wait complete. 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.720+0000 7f6d042d8b80 1 -- start start 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.721+0000 7f6d042d8b80 1 Processor -- start 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.721+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.721+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6d042d8b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da5f6430 con 0x5633da5e5400 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da5f65a0 con 0x5633da5e5d20 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6d042d8b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633da65c510 con 0x5633da5f0bd0 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6d042d8b80 10 client.0 _ll_register_callbacks cb 0x7f6cdc09b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41220/0 (socket says 172.21.15.5:41220) 2023-09-23T04:58:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 -- 172.21.15.5:0/1461954197 learned_addr learned my addr 172.21.15.5:0/1461954197 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cead9d700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 -- 172.21.15.5:0/1461954197 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 msgr2=0x5633da5f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 -- 172.21.15.5:0/1461954197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 msgr2=0x5633da5f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cea59c700 1 -- 172.21.15.5:0/1461954197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce0017070 con 0x5633da5e5400 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.722+0000 7f6cead9d700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cea59c700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc097ca0 tx=0x7f6cdc01db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773e104dba4a631b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0ce470 con 0x5633da5e5400 2023-09-23T04:58:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f6cdc095e40 con 0x5633da5e5400 2023-09-23T04:58:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cdc09b230 con 0x5633da5e5400 2023-09-23T04:58:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cdc0a3530 con 0x5633da5e5400 2023-09-23T04:58:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6d042d8b80 10 client.17595 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5633da38a250 con 0x5633da5e5400 2023-09-23T04:58:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.723+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cdc09b230 con 0x5633da5e5400 2023-09-23T04:58:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6cdc09a720 con 0x5633da5e5400 2023-09-23T04:58:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 10 client.17595 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 20 client.17595 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da65d5d0 0x5633da65f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 4 client.17595 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633da531b30 con 0x5633da65d5d0 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.724+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da65d5d0 0x5633da65f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6ce9d9b700 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da65d5d0 0x5633da65f9c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8ae4f811547d10 server_cookie=52017119f14c5a3b in_seq=0 out_seq=0 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6cd1ffb700 10 client.17595 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6cd1ffb700 1 -- 172.21.15.5:0/1461954197 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5633da531b30 con 0x5633da65d5d0 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6cd1ffb700 10 client.17595 handle_command_reply: tid=0 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 client.17595 shutdown 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 msgr2=0x5633da5f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc097ca0 tx=0x7f6cdc01db10 comp rx=0 tx=0).stop 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 shutdown_connections 2023-09-23T04:58:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5633da65d5d0 0x5633da65f9c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5633da5f0bd0 0x5633da5f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633da5e5400 0x5633da5f7520 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6cd1ffb700 0 client.17595 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 --2- 172.21.15.5:0/1461954197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5633da5e5d20 0x5633da5f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 >> 172.21.15.5:0/1461954197 conn(0x5633da55a260 msgr2=0x5633da5e09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 shutdown_connections 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.725+0000 7f6d042d8b80 1 -- 172.21.15.5:0/1461954197 wait complete. 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6d042d8b80 20 client.17595 trim_cache size 0 max 16384 2023-09-23T04:58:28.730 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:58:28.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 msgr2=0x7f6cd40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 0x7f6cd40a3bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6cec19a940 tx=0x7f6ce0009260 comp rx=0 tx=0).stop 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 msgr2=0x7f6cec199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b070 tx=0x7f6cdc00bce0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 shutdown_connections 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6cd40a1710 0x7f6cd40a3bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6cec10b490 0x7f6cec19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cec1020c0 0x7f6cec199330 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 --2- 172.21.15.5:0/2445959692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec1029e0 0x7f6cec199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.726+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 >> 172.21.15.5:0/2445959692 conn(0x7f6cec076f30 msgr2=0x7f6cec107270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.727+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 shutdown_connections 2023-09-23T04:58:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:28.727+0000 7f6cf09ff700 1 -- 172.21.15.5:0/2445959692 wait complete. 2023-09-23T04:58:28.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:28 smithi005 ceph-mon[97638]: pgmap v380: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-23T04:58:28.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:28 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:28.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:28 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:28.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:28 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:28.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:28 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:28.749 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T04:58:28.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 616129 4.33 MB/sec execute 544 sec latency 1350.625 ms 2023-09-23T04:58:28.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:28 smithi069 ceph-mon[133931]: pgmap v380: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-23T04:58:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:28 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:28 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:28 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:28 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:28 smithi118 ceph-mon[134297]: pgmap v380: 129 pgs: 129 active+clean; 499 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2023-09-23T04:58:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:28 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:28 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:28 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:29.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:28 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 616129 4.32 MB/sec execute 545 sec latency 2350.694 ms 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub queued for path: / 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T04:58:29.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:29 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub queued for path: / 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T04:58:30.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:29 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub queued for path: / 2023-09-23T04:58:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T04:58:30.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:29 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:30.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 617039 4.32 MB/sec execute 546 sec latency 2870.033 ms 2023-09-23T04:58:30.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:30 smithi069 ceph-mon[133931]: pgmap v381: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:30 smithi005 ceph-mon[97638]: pgmap v381: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:31.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:30 smithi118 ceph-mon[134297]: pgmap v381: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:31.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 618507 4.33 MB/sec execute 547 sec latency 20.352 ms 2023-09-23T04:58:31.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:31 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:32.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:31 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:32.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:31 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T04:58:32.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619675 4.33 MB/sec execute 548 sec latency 124.135 ms 2023-09-23T04:58:32.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:32 smithi069 ceph-mon[133931]: pgmap v382: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-09-23T04:58:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:32 smithi005 ceph-mon[97638]: pgmap v382: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-09-23T04:58:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:32 smithi118 ceph-mon[134297]: pgmap v382: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2023-09-23T04:58:33.969 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619675 4.32 MB/sec execute 549 sec latency 1124.199 ms 2023-09-23T04:58:34.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619675 4.32 MB/sec execute 550 sec latency 2124.253 ms 2023-09-23T04:58:34.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:34 smithi069 ceph-mon[133931]: pgmap v383: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:58:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:34 smithi005 ceph-mon[97638]: pgmap v383: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:58:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:34 smithi118 ceph-mon[134297]: pgmap v383: 129 pgs: 129 active+clean; 511 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2023-09-23T04:58:35.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619675 4.31 MB/sec execute 551 sec latency 3124.310 ms 2023-09-23T04:58:36.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619809 4.30 MB/sec execute 552 sec latency 4055.938 ms 2023-09-23T04:58:36.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:36 smithi069 ceph-mon[133931]: pgmap v384: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-09-23T04:58:37.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:36 smithi005 ceph-mon[97638]: pgmap v384: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-09-23T04:58:37.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:36 smithi118 ceph-mon[134297]: pgmap v384: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2023-09-23T04:58:37.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 621744 4.32 MB/sec execute 553 sec latency 14.777 ms 2023-09-23T04:58:38.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 622735 4.31 MB/sec execute 554 sec latency 13.382 ms 2023-09-23T04:58:38.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:38 smithi069 ceph-mon[133931]: pgmap v385: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-09-23T04:58:39.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:38 smithi005 ceph-mon[97638]: pgmap v385: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-09-23T04:58:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:38 smithi118 ceph-mon[134297]: pgmap v385: 129 pgs: 129 active+clean; 512 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2023-09-23T04:58:39.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 623219 4.31 MB/sec execute 555 sec latency 606.196 ms 2023-09-23T04:58:40.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 623219 4.30 MB/sec execute 556 sec latency 1606.244 ms 2023-09-23T04:58:40.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:40 smithi069 ceph-mon[133931]: pgmap v386: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-09-23T04:58:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:40 smithi005 ceph-mon[97638]: pgmap v386: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-09-23T04:58:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:40 smithi118 ceph-mon[134297]: pgmap v386: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-09-23T04:58:41.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 623353 4.30 MB/sec execute 557 sec latency 2538.871 ms 2023-09-23T04:58:42.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 625283 4.31 MB/sec execute 558 sec latency 15.681 ms 2023-09-23T04:58:42.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:42 smithi069 ceph-mon[133931]: pgmap v387: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2023-09-23T04:58:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:42 smithi005 ceph-mon[97638]: pgmap v387: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2023-09-23T04:58:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:42 smithi118 ceph-mon[134297]: pgmap v387: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.2 MiB/s wr, 84 op/s 2023-09-23T04:58:43.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 626255 4.31 MB/sec execute 559 sec latency 14.987 ms 2023-09-23T04:58:44.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:58:44.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:58:44.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:58:44.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 626768 4.31 MB/sec execute 560 sec latency 461.506 ms 2023-09-23T04:58:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:44 smithi005 ceph-mon[97638]: pgmap v388: 129 pgs: 129 active+clean; 518 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2023-09-23T04:58:45.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:44 smithi118 ceph-mon[134297]: pgmap v388: 129 pgs: 129 active+clean; 518 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2023-09-23T04:58:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:44 smithi069 ceph-mon[133931]: pgmap v388: 129 pgs: 129 active+clean; 518 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2023-09-23T04:58:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:45.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 628564 4.32 MB/sec execute 561 sec latency 572.009 ms 2023-09-23T04:58:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:45 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:58:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:45 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:58:46.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:45 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:45 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:58:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:45 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:58:46.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:45 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:46.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:45 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:58:46.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:45 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:58:46.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:45 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:58:46.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 629550 4.32 MB/sec execute 562 sec latency 12.949 ms 2023-09-23T04:58:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:46 smithi005 ceph-mon[97638]: pgmap v389: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-09-23T04:58:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:46 smithi118 ceph-mon[134297]: pgmap v389: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-09-23T04:58:47.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:46 smithi069 ceph-mon[133931]: pgmap v389: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2023-09-23T04:58:47.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 630309 4.32 MB/sec execute 563 sec latency 476.404 ms 2023-09-23T04:58:48.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 630309 4.31 MB/sec execute 564 sec latency 1476.451 ms 2023-09-23T04:58:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:48 smithi005 ceph-mon[97638]: pgmap v390: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2023-09-23T04:58:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:48 smithi118 ceph-mon[134297]: pgmap v390: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2023-09-23T04:58:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:48 smithi069 ceph-mon[133931]: pgmap v390: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2023-09-23T04:58:49.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 630309 4.30 MB/sec execute 565 sec latency 2476.504 ms 2023-09-23T04:58:50.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 631378 4.30 MB/sec execute 566 sec latency 2992.290 ms 2023-09-23T04:58:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:50 smithi005 ceph-mon[97638]: pgmap v391: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:50 smithi118 ceph-mon[134297]: pgmap v391: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:51.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:50 smithi069 ceph-mon[133931]: pgmap v391: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2023-09-23T04:58:51.970 INFO:tasks.workunit.client.0.smithi005.stdout: 1 632701 4.31 MB/sec execute 567 sec latency 19.262 ms 2023-09-23T04:58:52.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633858 4.31 MB/sec execute 568 sec latency 165.914 ms 2023-09-23T04:58:53.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:52 smithi005 ceph-mon[97638]: pgmap v392: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2023-09-23T04:58:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:52 smithi118 ceph-mon[134297]: pgmap v392: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2023-09-23T04:58:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:52 smithi069 ceph-mon[133931]: pgmap v392: 129 pgs: 129 active+clean; 526 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 135 op/s 2023-09-23T04:58:53.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633858 4.31 MB/sec execute 569 sec latency 1165.975 ms 2023-09-23T04:58:54.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633858 4.30 MB/sec execute 570 sec latency 2166.019 ms 2023-09-23T04:58:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:54 smithi005 ceph-mon[97638]: pgmap v393: 129 pgs: 129 active+clean; 528 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-09-23T04:58:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:54 smithi118 ceph-mon[134297]: pgmap v393: 129 pgs: 129 active+clean; 528 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-09-23T04:58:55.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:54 smithi069 ceph-mon[133931]: pgmap v393: 129 pgs: 129 active+clean; 528 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-09-23T04:58:55.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633858 4.29 MB/sec execute 571 sec latency 3166.064 ms 2023-09-23T04:58:56.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633945 4.28 MB/sec execute 572 sec latency 4110.102 ms 2023-09-23T04:58:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:56 smithi005 ceph-mon[97638]: pgmap v394: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2023-09-23T04:58:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:56 smithi118 ceph-mon[134297]: pgmap v394: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2023-09-23T04:58:57.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:56 smithi069 ceph-mon[133931]: pgmap v394: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2023-09-23T04:58:57.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 635854 4.30 MB/sec execute 573 sec latency 20.914 ms 2023-09-23T04:58:58.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:58:58.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636813 4.30 MB/sec execute 574 sec latency 14.240 ms 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.025+0000 7f9a053ff700 1 -- 172.21.15.5:0/1672873292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00108430 msgr2=0x7f9a0010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.025+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1672873292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00108430 0x7f9a0010ffb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f99f800b0a0 tx=0x7f99f800b3b0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 -- 172.21.15.5:0/1672873292 shutdown_connections 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1672873292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00108430 0x7f9a0010ffb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1672873292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00103310 0x7f9a001036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1672873292 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a00103cc0 0x7f9a00107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 -- 172.21.15.5:0/1672873292 >> 172.21.15.5:0/1672873292 conn(0x7f9a000feb50 msgr2=0x7f9a00100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 -- 172.21.15.5:0/1672873292 shutdown_connections 2023-09-23T04:58:59.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.026+0000 7f9a053ff700 1 -- 172.21.15.5:0/1672873292 wait complete. 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 Processor -- start 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 -- start start 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 0x7f9a001995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a0019da50 0x7f9a0019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0019e4d0 con 0x7f9a00108430 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0019e640 con 0x7f9a00103310 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.027+0000 7f9a053ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0019e800 con 0x7f9a0019da50 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34078/0 (socket says 172.21.15.5:34078) 2023-09-23T04:58:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 -- 172.21.15.5:0/1548635025 learned_addr learned my addr 172.21.15.5:0/1548635025 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a0019da50 0x7f9a0019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 0x7f9a001995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 -- 172.21.15.5:0/1548635025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a0019da50 msgr2=0x7f9a0019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a0019da50 0x7f9a0019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 -- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 msgr2=0x7f9a001995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 0x7f9a001995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 -- 172.21.15.5:0/1548635025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0009520 con 0x7f9a00103310 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 0x7f9a001995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.028+0000 7f99feffd700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99f000eb30 tx=0x7f99f000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.029+0000 7f99e7fff700 1 -- 172.21.15.5:0/1548635025 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0009880 con 0x7f9a00103310 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.029+0000 7f99e7fff700 1 -- 172.21.15.5:0/1548635025 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f00044e0 con 0x7f9a00103310 2023-09-23T04:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.029+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f8009d00 con 0x7f9a00103310 2023-09-23T04:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.029+0000 7f99e7fff700 1 -- 172.21.15.5:0/1548635025 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0003e90 con 0x7f9a00103310 2023-09-23T04:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.029+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a001a6100 con 0x7f9a00103310 2023-09-23T04:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.033+0000 7f99e7fff700 1 -- 172.21.15.5:0/1548635025 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f99f0004000 con 0x7f9a00103310 2023-09-23T04:58:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.035+0000 7f9a18ddfb80 1 Processor -- start 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.035+0000 7f9a18ddfb80 1 -- start start 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.035+0000 7f99e7fff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 0x7f99e80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99e7fff700 1 -- 172.21.15.5:0/1548635025 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f99f0014070 con 0x7f9a00103310 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718555d20 0x564718560690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718562fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647184be460 con 0x564718560bd0 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647184afc20 con 0x564718555400 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647184ad540 con 0x564718555d20 2023-09-23T04:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 0x7f99e80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34090/0 (socket says 172.21.15.5:34090) 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99feffd700 1 -- 172.21.15.5:0/344362888 learned_addr learned my addr 172.21.15.5:0/344362888 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718555d20 0x564718560690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.036+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718562fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 -- 172.21.15.5:0/344362888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718555d20 msgr2=0x564718560690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718555d20 0x564718560690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 -- 172.21.15.5:0/344362888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 msgr2=0x564718562fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718562fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 -- 172.21.15.5:0/344362888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56471850b8d0 con 0x564718555400 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99feffd700 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99f0097100 tx=0x7f99f009a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839afc2359d51da7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 0x7f99e80a3c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99f800b780 tx=0x7f99f800bf00 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99e57fa700 1 -- 172.21.15.5:0/344362888 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f000cc90 con 0x564718555400 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99e57fa700 1 -- 172.21.15.5:0/344362888 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f009a690 con 0x564718555400 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.037+0000 7f99e57fa700 1 -- 172.21.15.5:0/344362888 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0098a30 con 0x564718555400 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/344362888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 msgr2=0x5647185557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99f0097100 tx=0x7f99f009a2c0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/344362888 shutdown_connections 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718562fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185557e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/344362888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718555d20 0x564718560690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/344362888 >> 172.21.15.5:0/344362888 conn(0x5647184ca260 msgr2=0x564718551790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/344362888 shutdown_connections 2023-09-23T04:58:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.038+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/344362888 wait complete. 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.039+0000 7f9a18ddfb80 1 -- start start 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.039+0000 7f9a18ddfb80 1 Processor -- start 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x56471855d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x56471855e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647184a8e30 con 0x564718555400 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564718445710 con 0x564718555d20 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647184453b0 con 0x564718560bd0 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f9a18ddfb80 10 client.0 _ll_register_callbacks cb 0x5647182a2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34104/0 (socket says 172.21.15.5:34104) 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 -- 172.21.15.5:0/774764076 learned_addr learned my addr 172.21.15.5:0/774764076 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99feffd700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x56471855d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x56471855e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 -- 172.21.15.5:0/774764076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 msgr2=0x56471855e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x56471855e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 -- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 msgr2=0x56471855d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x56471855d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99fe7fc700 1 -- 172.21.15.5:0/774764076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a001a6100 con 0x564718555d20 2023-09-23T04:58:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.040+0000 7f99ff7fe700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x56471855e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009be0 tx=0x7f99f4009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0773d1e86c20f1c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f99e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4016070 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564718549250 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f99f4011440 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f4003e70 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f9a18ddfb80 10 client.27605 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.041+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5647182d3de0 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564718549250 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f99f401b750 con 0x564718555d20 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 10 client.27605 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 10 client.27605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 20 client.27605 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185706f0 0x564718572b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 4 client.27605 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647184a1e10 con 0x5647185706f0 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.042+0000 7f99feffd700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185706f0 0x564718572b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.043+0000 7f99e2ff5700 10 client.27605 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.043+0000 7f99feffd700 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185706f0 0x564718572b10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8785ba6d9b1042b5 server_cookie=94327259a71b3ded in_seq=0 out_seq=0 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.044+0000 7f99e2ff5700 1 -- 172.21.15.5:0/774764076 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647184a1e10 con 0x5647185706f0 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.044+0000 7f99e2ff5700 10 client.27605 handle_command_reply: tid=0 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.044+0000 7f9a18ddfb80 1 client.27605 shutdown 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.044+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 msgr2=0x56471855dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.044+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009be0 tx=0x7f99f4009ef0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 shutdown_connections 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185706f0 0x564718572b10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x56471855e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x56471855d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/774764076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555d20 0x56471855dea0 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009be0 tx=0x7f99f4009ef0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 >> 172.21.15.5:0/774764076 conn(0x5647184ca260 msgr2=0x564718551320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f99e2ff5700 0 client.27605 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 shutdown_connections 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.045+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/774764076 wait complete. 2023-09-23T04:58:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.046+0000 7f9a18ddfb80 20 client.27605 trim_cache size 0 max 16384 2023-09-23T04:58:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 Processor -- start 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 -- start start 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718567710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 0x56471855ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.066+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564718329920 con 0x564718560bd0 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647185525e0 con 0x564718555400 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471855ecb0 con 0x5647185593e0 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 0x56471855ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99ff7fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 0x56471855ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34610/0 (socket says 172.21.15.5:34610) 2023-09-23T04:58:59.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34106/0 (socket says 172.21.15.5:34106) 2023-09-23T04:58:59.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99ff7fe700 1 -- 172.21.15.5:0/1487438884 learned_addr learned my addr 172.21.15.5:0/1487438884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:59.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718567710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 -- 172.21.15.5:0/1487438884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 msgr2=0x56471855ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 0x56471855ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 -- 172.21.15.5:0/1487438884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 msgr2=0x564718567710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718567710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99feffd700 1 -- 172.21.15.5:0/1487438884 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56471854d480 con 0x564718555400 2023-09-23T04:58:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.067+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718567710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f99feffd700 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5647185561f0 tx=0x7f99f00cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999064f44768b53d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f99e5ffb700 1 -- 172.21.15.5:0/1487438884 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00987d0 con 0x564718555400 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f99e5ffb700 1 -- 172.21.15.5:0/1487438884 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f00a2a20 con 0x564718555400 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f99e5ffb700 1 -- 172.21.15.5:0/1487438884 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0098b10 con 0x564718555400 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1487438884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 msgr2=0x5647185582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.068+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5647185561f0 tx=0x7f99f00cd040 comp rx=0 tx=0).stop 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1487438884 shutdown_connections 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5647185593e0 0x56471855ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718555400 0x5647185582a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1487438884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718560bd0 0x564718567710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1487438884 >> 172.21.15.5:0/1487438884 conn(0x5647184ca260 msgr2=0x564718558cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1487438884 shutdown_connections 2023-09-23T04:58:59.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1487438884 wait complete. 2023-09-23T04:58:59.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 -- start start 2023-09-23T04:58:59.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.069+0000 7f9a18ddfb80 1 Processor -- start 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x564718567550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718565dc0 0x564718566240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564718567c80 con 0x564718555400 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647185c4550 con 0x564718565dc0 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647185ccea0 con 0x564718560bd0 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f9a18ddfb80 10 client.0 _ll_register_callbacks cb 0x7f99f4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f99fe7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f99ff7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718565dc0 0x564718566240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f99fe7fc700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34624/0 (socket says 172.21.15.5:34624) 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.070+0000 7f99fe7fc700 1 -- 172.21.15.5:0/1313757802 learned_addr learned my addr 172.21.15.5:0/1313757802 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99feffd700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x564718567550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 -- 172.21.15.5:0/1313757802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718565dc0 msgr2=0x564718566240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718565dc0 0x564718566240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 -- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 msgr2=0x564718567550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x564718567550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 -- 172.21.15.5:0/1313757802 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56471850b8d0 con 0x564718560bd0 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99feffd700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x564718567550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99fe7fc700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99f4000c00 tx=0x7f99f401b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab762145035a424 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4016040 con 0x564718560bd0 2023-09-23T04:58:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99f009a690 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.071+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f99f4011640 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f4011800 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 10 client.27673 fetch_fsmap learned FSMap version 57 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5647182bcc60 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f009a690 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f99f402f020 con 0x564718560bd0 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 10 client.27673 fetch_fsmap finished waiting for FSMap version 57 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 10 client.27673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 20 client.27673 populate_metadata read hostname 'smithi005' 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185ce660 0x5647185d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 4 client.27673 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:58:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647184a1b30 con 0x5647185ce660 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.072+0000 7f99feffd700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185ce660 0x5647185d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f99feffd700 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185ce660 0x5647185d0a80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108cddc0de322f6b server_cookie=d40eead36e7d85a5 in_seq=0 out_seq=0 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f99e2ff5700 10 client.27673 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f99e2ff5700 1 -- 172.21.15.5:0/1313757802 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5647184a1b30 con 0x5647185ce660 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f99e2ff5700 10 client.27673 handle_command_reply: tid=0 2023-09-23T04:58:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 client.27673 shutdown 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 msgr2=0x564718565880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99f4000c00 tx=0x7f99f401b9a0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 shutdown_connections 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5647185ce660 0x5647185d0a80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564718565dc0 0x564718566240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564718555400 0x564718567550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 --2- 172.21.15.5:0/1313757802 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564718560bd0 0x564718565880 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 >> 172.21.15.5:0/1313757802 conn(0x5647184ca260 msgr2=0x56471855d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.073+0000 7f99e2ff5700 0 client.27673 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 shutdown_connections 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a18ddfb80 1 -- 172.21.15.5:0/1313757802 wait complete. 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a18ddfb80 20 client.27673 trim_cache size 0 max 16384 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T04:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 msgr2=0x7f99e80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 0x7f99e80a3c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99f800b780 tx=0x7f99f800bf00 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 msgr2=0x7f9a001990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.074+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f99f000eb30 tx=0x7f99f000ee40 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 shutdown_connections 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e80a1790 0x7f99e80a3c50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9a0019da50 0x7f9a0019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a00103310 0x7f9a001990b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 --2- 172.21.15.5:0/1548635025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a00108430 0x7f9a001995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 >> 172.21.15.5:0/1548635025 conn(0x7f9a000feb50 msgr2=0x7f9a000ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 shutdown_connections 2023-09-23T04:58:59.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:58:59.075+0000 7f9a053ff700 1 -- 172.21.15.5:0/1548635025 wait complete. 2023-09-23T04:58:59.092 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:58:58 smithi005 ceph-mon[97638]: pgmap v395: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:58:59.094 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T04:58:59.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:58:58 smithi118 ceph-mon[134297]: pgmap v395: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:58:59.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:58:58 smithi069 ceph-mon[133931]: pgmap v395: 129 pgs: 129 active+clean; 531 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2023-09-23T04:58:59.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637402 4.29 MB/sec execute 575 sec latency 565.925 ms 2023-09-23T04:59:00.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637402 4.29 MB/sec execute 576 sec latency 1565.971 ms 2023-09-23T04:59:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:00 smithi005 ceph-mon[97638]: pgmap v396: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-09-23T04:59:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:00 smithi118 ceph-mon[134297]: pgmap v396: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-09-23T04:59:01.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:00 smithi069 ceph-mon[133931]: pgmap v396: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2023-09-23T04:59:01.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637495 4.28 MB/sec execute 577 sec latency 2497.288 ms 2023-09-23T04:59:02.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 638001 4.27 MB/sec execute 578 sec latency 106.056 ms 2023-09-23T04:59:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:02 smithi005 ceph-mon[97638]: pgmap v397: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2023-09-23T04:59:03.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:02 smithi118 ceph-mon[134297]: pgmap v397: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2023-09-23T04:59:03.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:02 smithi069 ceph-mon[133931]: pgmap v397: 129 pgs: 129 active+clean; 534 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 92 op/s 2023-09-23T04:59:03.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 639518 4.29 MB/sec execute 579 sec latency 55.272 ms 2023-09-23T04:59:04.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 640516 4.28 MB/sec execute 580 sec latency 23.125 ms 2023-09-23T04:59:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:04 smithi005 ceph-mon[97638]: pgmap v398: 129 pgs: 129 active+clean; 536 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.7 MiB/s wr, 130 op/s 2023-09-23T04:59:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:04 smithi118 ceph-mon[134297]: pgmap v398: 129 pgs: 129 active+clean; 536 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.7 MiB/s wr, 130 op/s 2023-09-23T04:59:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:04 smithi069 ceph-mon[133931]: pgmap v398: 129 pgs: 129 active+clean; 536 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.7 MiB/s wr, 130 op/s 2023-09-23T04:59:05.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 640948 4.28 MB/sec execute 581 sec latency 727.146 ms 2023-09-23T04:59:06.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 641082 4.28 MB/sec execute 582 sec latency 1655.873 ms 2023-09-23T04:59:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:06 smithi005 ceph-mon[97638]: pgmap v399: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2023-09-23T04:59:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:06 smithi005 ceph-mon[97638]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T04:59:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:06 smithi118 ceph-mon[134297]: pgmap v399: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2023-09-23T04:59:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:06 smithi118 ceph-mon[134297]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T04:59:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:06 smithi069 ceph-mon[133931]: pgmap v399: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.7 MiB/s wr, 137 op/s 2023-09-23T04:59:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:06 smithi069 ceph-mon[133931]: osdmap e105: 12 total, 12 up, 12 in 2023-09-23T04:59:07.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643011 4.29 MB/sec execute 583 sec latency 16.731 ms 2023-09-23T04:59:08.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643924 4.29 MB/sec execute 584 sec latency 94.822 ms 2023-09-23T04:59:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:08 smithi005 ceph-mon[97638]: pgmap v401: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2023-09-23T04:59:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:08 smithi118 ceph-mon[134297]: pgmap v401: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2023-09-23T04:59:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:08 smithi069 ceph-mon[133931]: pgmap v401: 129 pgs: 129 active+clean; 537 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 138 op/s 2023-09-23T04:59:09.971 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644492 4.29 MB/sec execute 585 sec latency 384.518 ms 2023-09-23T04:59:10.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 645408 4.28 MB/sec execute 586 sec latency 901.408 ms 2023-09-23T04:59:11.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:10 smithi005 ceph-mon[97638]: pgmap v402: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-09-23T04:59:11.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:10 smithi005 ceph-mon[97638]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T04:59:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:10 smithi118 ceph-mon[134297]: pgmap v402: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-09-23T04:59:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:10 smithi118 ceph-mon[134297]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T04:59:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:10 smithi069 ceph-mon[133931]: pgmap v402: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2023-09-23T04:59:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:10 smithi069 ceph-mon[133931]: osdmap e106: 12 total, 12 up, 12 in 2023-09-23T04:59:11.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 646825 4.29 MB/sec execute 587 sec latency 16.951 ms 2023-09-23T04:59:12.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648042 4.30 MB/sec execute 588 sec latency 148.299 ms 2023-09-23T04:59:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:12 smithi005 ceph-mon[97638]: pgmap v404: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2023-09-23T04:59:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:12 smithi118 ceph-mon[134297]: pgmap v404: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2023-09-23T04:59:13.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:12 smithi069 ceph-mon[133931]: pgmap v404: 129 pgs: 129 active+clean; 542 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.8 MiB/s wr, 152 op/s 2023-09-23T04:59:13.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648042 4.29 MB/sec execute 589 sec latency 1148.366 ms 2023-09-23T04:59:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:13 smithi005 ceph-mon[97638]: pgmap v405: 129 pgs: 129 active+clean; 549 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2023-09-23T04:59:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:13 smithi118 ceph-mon[134297]: pgmap v405: 129 pgs: 129 active+clean; 549 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2023-09-23T04:59:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:13 smithi069 ceph-mon[133931]: pgmap v405: 129 pgs: 129 active+clean; 549 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2023-09-23T04:59:14.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648042 4.28 MB/sec execute 590 sec latency 2148.416 ms 2023-09-23T04:59:15.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648042 4.27 MB/sec execute 591 sec latency 3148.466 ms 2023-09-23T04:59:16.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648176 4.27 MB/sec execute 592 sec latency 4081.412 ms 2023-09-23T04:59:16.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:16 smithi069 ceph-mon[133931]: pgmap v406: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2023-09-23T04:59:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:16 smithi005 ceph-mon[97638]: pgmap v406: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2023-09-23T04:59:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:16 smithi118 ceph-mon[134297]: pgmap v406: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2023-09-23T04:59:17.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650052 4.28 MB/sec execute 593 sec latency 15.525 ms 2023-09-23T04:59:18.972 INFO:tasks.workunit.client.0.smithi005.stdout: 1 651074 4.28 MB/sec execute 594 sec latency 13.208 ms 2023-09-23T04:59:18.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:18 smithi069 ceph-mon[133931]: pgmap v407: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-09-23T04:59:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:18 smithi005 ceph-mon[97638]: pgmap v407: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-09-23T04:59:19.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:18 smithi118 ceph-mon[134297]: pgmap v407: 129 pgs: 129 active+clean; 550 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2023-09-23T04:59:19.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 651583 4.28 MB/sec execute 595 sec latency 687.526 ms 2023-09-23T04:59:20.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 651583 4.27 MB/sec execute 596 sec latency 1688.110 ms 2023-09-23T04:59:20.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:20 smithi069 ceph-mon[133931]: pgmap v408: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-09-23T04:59:21.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:20 smithi005 ceph-mon[97638]: pgmap v408: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-09-23T04:59:21.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:20 smithi118 ceph-mon[134297]: pgmap v408: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2023-09-23T04:59:21.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 651667 4.26 MB/sec execute 597 sec latency 2627.315 ms 2023-09-23T04:59:22.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 653583 4.28 MB/sec execute 598 sec latency 29.878 ms 2023-09-23T04:59:22.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:22 smithi069 ceph-mon[133931]: pgmap v409: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T04:59:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:22 smithi005 ceph-mon[97638]: pgmap v409: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T04:59:23.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:22 smithi118 ceph-mon[134297]: pgmap v409: 129 pgs: 129 active+clean; 553 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T04:59:23.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 654658 4.28 MB/sec execute 599 sec latency 12.820 ms 2023-09-23T04:59:24.973 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 600 sec 2023-09-23T04:59:24.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:24 smithi069 ceph-mon[133931]: pgmap v410: 129 pgs: 129 active+clean; 557 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-09-23T04:59:25.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:24 smithi005 ceph-mon[97638]: pgmap v410: 129 pgs: 129 active+clean; 557 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-09-23T04:59:25.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:24 smithi118 ceph-mon[134297]: pgmap v410: 129 pgs: 129 active+clean; 557 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-09-23T04:59:25.320 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 600 sec 2023-09-23T04:59:25.320 INFO:tasks.workunit.client.0.smithi005.stdout: 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 81733 1.699 4734.680 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Close 60041 0.004 0.177 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 3460 69.910 4469.497 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 16499 2.306 8340.974 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Deltree 2 101.997 129.025 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Mkdir 1 0.025 0.025 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 74080 1.637 4964.656 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 12982 0.002 0.028 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 13576 0.222 32.620 2023-09-23T04:59:25.321 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 6654 0.032 5.999 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: Find 28636 0.693 12505.708 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 40737 0.218 106.047 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 128072 0.018 9.643 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 266 0.618 1.691 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 266 0.625 2.931 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 5728 4.073 310.496 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout: 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 4.27611 MB/sec 1 clients 1 procs max_latency=12505.722 ms 2023-09-23T04:59:25.322 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 10 2023-09-23T04:59:25.323 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-09-23T04:59:25.323 INFO:tasks.workunit.client.0.smithi005.stdout: 2023-09-23T04:59:25.323 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-09-23T04:59:25.328 INFO:tasks.workunit.client.0.smithi005.stdout:9 of 10 processes prepared for launch 0 sec 2023-09-23T04:59:25.429 INFO:tasks.workunit.client.0.smithi005.stdout:10 of 10 processes prepared for launch 0 sec 2023-09-23T04:59:25.429 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2023-09-23T04:59:26.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 341 123.17 MB/sec warmup 1 sec latency 85.550 ms 2023-09-23T04:59:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:26 smithi069 ceph-mon[133931]: pgmap v411: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 107 op/s 2023-09-23T04:59:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:26 smithi005 ceph-mon[97638]: pgmap v411: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 107 op/s 2023-09-23T04:59:27.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:26 smithi118 ceph-mon[134297]: pgmap v411: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 107 op/s 2023-09-23T04:59:27.429 INFO:tasks.workunit.client.0.smithi005.stdout: 10 712 103.33 MB/sec warmup 2 sec latency 114.118 ms 2023-09-23T04:59:28.429 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1101 75.38 MB/sec warmup 3 sec latency 134.504 ms 2023-09-23T04:59:28.990 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:28 smithi069 ceph-mon[133931]: pgmap v412: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2023-09-23T04:59:28.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:28 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:29.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:59:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:28 smithi005 ceph-mon[97638]: pgmap v412: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2023-09-23T04:59:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:28 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:28 smithi118 ceph-mon[134297]: pgmap v412: 129 pgs: 129 active+clean; 558 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2023-09-23T04:59:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:28 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 -- 172.21.15.5:0/1820872451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa558101a80 msgr2=0x7fa558105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/1820872451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa558101a80 0x7fa558105ac0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009990 tx=0x7fa54c009ca0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 -- 172.21.15.5:0/1820872451 shutdown_connections 2023-09-23T04:59:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/1820872451 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa5581061f0 0x7fa55810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/1820872451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5581010d0 0x7fa5581014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/1820872451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa558101a80 0x7fa558105ac0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 -- 172.21.15.5:0/1820872451 >> 172.21.15.5:0/1820872451 conn(0x7fa5580fc990 msgr2=0x7fa5580fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 -- 172.21.15.5:0/1820872451 shutdown_connections 2023-09-23T04:59:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.380+0000 7fa55d5ff700 1 -- 172.21.15.5:0/1820872451 wait complete. 2023-09-23T04:59:29.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.381+0000 7fa55d5ff700 1 Processor -- start 2023-09-23T04:59:29.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.382+0000 7fa55d5ff700 1 -- start start 2023-09-23T04:59:29.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.382+0000 7fa55d5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49852/0 (socket says 172.21.15.5:49852) 2023-09-23T04:59:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa55d5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5581061f0 0x7fa558199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa55d5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa55819da60 0x7fa55819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa55d5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55819e4e0 con 0x7fa5581061f0 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa55d5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55819e650 con 0x7fa5581010d0 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa55d5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55819e810 con 0x7fa55819da60 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa556ffd700 1 -- 172.21.15.5:0/3227032729 learned_addr learned my addr 172.21.15.5:0/3227032729 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa5577fe700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa55819da60 0x7fa55819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa556ffd700 1 -- 172.21.15.5:0/3227032729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa55819da60 msgr2=0x7fa55819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.383+0000 7fa556ffd700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa55819da60 0x7fa55819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa556ffd700 1 -- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5581061f0 msgr2=0x7fa558199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa556ffd700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5581061f0 0x7fa558199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa556ffd700 1 -- 172.21.15.5:0/3227032729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0095f0 con 0x7fa5581010d0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa556ffd700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e9a0 tx=0x7fa54800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa53ffff700 1 -- 172.21.15.5:0/3227032729 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548009790 con 0x7fa5581010d0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa53ffff700 1 -- 172.21.15.5:0/3227032729 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54800cbd0 con 0x7fa5581010d0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa53ffff700 1 -- 172.21.15.5:0/3227032729 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548018680 con 0x7fa5581010d0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5581a5d70 con 0x7fa5581010d0 2023-09-23T04:59:29.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.384+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581a61a0 con 0x7fa5581010d0 2023-09-23T04:59:29.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.390+0000 7fa53ffff700 1 -- 172.21.15.5:0/3227032729 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa54800cd40 con 0x7fa5581010d0 2023-09-23T04:59:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.392+0000 7fa570ee6b80 1 Processor -- start 2023-09-23T04:59:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.392+0000 7fa53ffff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 0x7fa5400a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.392+0000 7fa570ee6b80 1 -- start start 2023-09-23T04:59:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.392+0000 7fa53ffff700 1 -- 172.21.15.5:0/3227032729 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa548014070 con 0x7fa5581010d0 2023-09-23T04:59:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.392+0000 7fa5577fe700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 0x7fa5400a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a754460 con 0x56385a7eb400 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a745c20 con 0x56385a7f6bd0 2023-09-23T04:59:29.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a743540 con 0x56385a7ebd20 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.393+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41352/0 (socket says 172.21.15.5:41352) 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 -- 172.21.15.5:0/2479682345 learned_addr learned my addr 172.21.15.5:0/2479682345 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa5577fe700 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa5577fe700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 0x7fa5400a3d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55000f540 tx=0x7fa55000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 -- 172.21.15.5:0/2479682345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 msgr2=0x56385a7f6690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7f6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 -- 172.21.15.5:0/2479682345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 msgr2=0x56385a7f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 -- 172.21.15.5:0/2479682345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385a7a18d0 con 0x56385a7eb400 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa5577fe700 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:59:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.394+0000 7fa556ffd700 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5480a2280 tx=0x7fa548099d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4872586f7453452 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.395+0000 7fa53d7fa700 1 -- 172.21.15.5:0/2479682345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54809e570 con 0x56385a7eb400 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.395+0000 7fa53d7fa700 1 -- 172.21.15.5:0/2479682345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5480a27b0 con 0x56385a7eb400 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.395+0000 7fa53d7fa700 1 -- 172.21.15.5:0/2479682345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548099910 con 0x56385a7eb400 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 -- 172.21.15.5:0/2479682345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 msgr2=0x56385a7eb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5480a2280 tx=0x7fa548099d80 comp rx=0 tx=0).stop 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 -- 172.21.15.5:0/2479682345 shutdown_connections 2023-09-23T04:59:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7eb7e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/2479682345 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 -- 172.21.15.5:0/2479682345 >> 172.21.15.5:0/2479682345 conn(0x56385a760260 msgr2=0x56385a7e7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 -- 172.21.15.5:0/2479682345 shutdown_connections 2023-09-23T04:59:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.396+0000 7fa570ee6b80 1 -- 172.21.15.5:0/2479682345 wait complete. 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.397+0000 7fa570ee6b80 1 -- start start 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.397+0000 7fa570ee6b80 1 Processor -- start 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 0x56385a7f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 0x56385a7f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a73ee30 con 0x56385a7ebd20 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a6db710 con 0x56385a7eb400 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a6db3b0 con 0x56385a7f6bd0 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa570ee6b80 10 client.0 _ll_register_callbacks cb 0x56385a538800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 0x56385a7f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 0x56385a7f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 0x56385a7f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41362/0 (socket says 172.21.15.5:41362) 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa5567fc700 1 -- 172.21.15.5:0/314903104 learned_addr learned my addr 172.21.15.5:0/314903104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.398+0000 7fa5577fe700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 -- 172.21.15.5:0/314903104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 msgr2=0x56385a7f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 -- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 msgr2=0x56385a7f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 0x56385a7f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 -- 172.21.15.5:0/314903104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa550017440 con 0x56385a7eb400 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa556ffd700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 0x56385a7f3920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa54809acd0 tx=0x7fa548099ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b16744eb2397b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa53aff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.399+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54809e570 con 0x56385a7eb400 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.400+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa54809a890 con 0x56385a7eb400 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.400+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56385a7df250 con 0x56385a7eb400 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.400+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5480a2320 con 0x56385a7eb400 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.400+0000 7fa570ee6b80 10 client.27623 fetch_fsmap learned FSMap version 58 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.400+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fa5581a61a0 con 0x56385a7eb400 2023-09-23T04:59:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56385a7df250 con 0x56385a7eb400 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 58) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa548099380 con 0x56385a7eb400 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 10 client.27623 fetch_fsmap finished waiting for FSMap version 58 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 10 client.27623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 20 client.27623 populate_metadata read hostname 'smithi005' 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a806450 0x56385a808870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 4 client.27623 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:59:29.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.401+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56385a737e10 con 0x56385a806450 2023-09-23T04:59:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.402+0000 7fa5567fc700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a806450 0x56385a808870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.403+0000 7fa5567fc700 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a806450 0x56385a808870 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cacfd5bcfa7afa9 server_cookie=33939e90f2d0c11e in_seq=0 out_seq=0 2023-09-23T04:59:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.403+0000 7fa53aff5700 10 client.27623 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:59:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.405+0000 7fa53aff5700 1 -- 172.21.15.5:0/314903104 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56385a737e10 con 0x56385a806450 2023-09-23T04:59:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.405+0000 7fa53aff5700 10 client.27623 handle_command_reply: tid=0 2023-09-23T04:59:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.405+0000 7fa570ee6b80 1 client.27623 shutdown 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 msgr2=0x56385a7f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 0x56385a7f3920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa54809acd0 tx=0x7fa548099ba0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 shutdown_connections 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a806450 0x56385a808870 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7eb400 0x56385a7f3920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/314903104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7ebd20 0x56385a7f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 >> 172.21.15.5:0/314903104 conn(0x56385a760260 msgr2=0x56385a7e7320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa53aff5700 0 client.27623 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:59:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 shutdown_connections 2023-09-23T04:59:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 1 -- 172.21.15.5:0/314903104 wait complete. 2023-09-23T04:59:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.406+0000 7fa570ee6b80 20 client.27623 trim_cache size 0 max 16384 2023-09-23T04:59:29.429 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1756 64.98 MB/sec warmup 4 sec latency 54.897 ms 2023-09-23T04:59:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 Processor -- start 2023-09-23T04:59:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 -- start start 2023-09-23T04:59:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a5bf920 con 0x56385a7eb400 2023-09-23T04:59:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a7e85b0 con 0x56385a7f6bd0 2023-09-23T04:59:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.438+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a7df100 con 0x56385a7ebd20 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa5577fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41368/0 (socket says 172.21.15.5:41368) 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 -- 172.21.15.5:0/1197626207 learned_addr learned my addr 172.21.15.5:0/1197626207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa5567fc700 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 -- 172.21.15.5:0/1197626207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 msgr2=0x56385a7ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7ee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 -- 172.21.15.5:0/1197626207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 msgr2=0x56385a7f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa556ffd700 1 -- 172.21.15.5:0/1197626207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385a569de0 con 0x56385a7eb400 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.439+0000 7fa5577fe700 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.440+0000 7fa556ffd700 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa548096e90 tx=0x7fa548099d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f3051f41deeca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.440+0000 7fa53dffb700 1 -- 172.21.15.5:0/1197626207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548096760 con 0x56385a7eb400 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.440+0000 7fa53dffb700 1 -- 172.21.15.5:0/1197626207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54809e570 con 0x56385a7eb400 2023-09-23T04:59:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.440+0000 7fa53dffb700 1 -- 172.21.15.5:0/1197626207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548096210 con 0x56385a7eb400 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 -- 172.21.15.5:0/1197626207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 msgr2=0x56385a7f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa548096e90 tx=0x7fa548099d80 comp rx=0 tx=0).stop 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 -- 172.21.15.5:0/1197626207 shutdown_connections 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7f6bd0 0x56385a7f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7f61d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/1197626207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7ebd20 0x56385a7ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 -- 172.21.15.5:0/1197626207 >> 172.21.15.5:0/1197626207 conn(0x56385a760260 msgr2=0x56385a7f5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 -- 172.21.15.5:0/1197626207 shutdown_connections 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.441+0000 7fa570ee6b80 1 -- 172.21.15.5:0/1197626207 wait complete. 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 -- start start 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 Processor -- start 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a7fbb20 con 0x56385a7eb400 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a7fbc90 con 0x56385a7ebd20 2023-09-23T04:59:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385a7fbe50 con 0x56385a7f6bd0 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.442+0000 7fa570ee6b80 10 client.0 _ll_register_callbacks cb 0x7fa55000f2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:49874/0 (socket says 172.21.15.5:49874) 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa556ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 -- 172.21.15.5:0/973663371 learned_addr learned my addr 172.21.15.5:0/973663371 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5577fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 -- 172.21.15.5:0/973663371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 msgr2=0x56385a7fb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7fb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 -- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 msgr2=0x56385a7fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.443+0000 7fa5567fc700 1 -- 172.21.15.5:0/973663371 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa550009040 con 0x56385a7ebd20 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa5567fc700 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56385a7f4ed0 tx=0x7fa54c025450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5915ccdde98825 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa53aff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c01d040 con 0x56385a7ebd20 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa54c0259f0 con 0x56385a7ebd20 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5480a27b0 con 0x56385a7ebd20 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.444+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa54c025bb0 con 0x56385a7ebd20 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 10 client.27629 fetch_fsmap learned FSMap version 58 2023-09-23T04:59:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fa5581a61a0 con 0x56385a7ebd20 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5480a27b0 con 0x56385a7ebd20 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 58) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa54c004670 con 0x56385a7ebd20 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 10 client.27629 fetch_fsmap finished waiting for FSMap version 58 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 10 client.27629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 20 client.27629 populate_metadata read hostname 'smithi005' 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a85b410 0x56385a85d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 4 client.27629 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T04:59:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56385a737b30 con 0x56385a85b410 2023-09-23T04:59:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.445+0000 7fa556ffd700 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a85b410 0x56385a85d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.446+0000 7fa556ffd700 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a85b410 0x56385a85d830 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52af52791de31d63 server_cookie=f4bd58cea331245e in_seq=0 out_seq=0 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.446+0000 7fa53aff5700 10 client.27629 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa53aff5700 1 -- 172.21.15.5:0/973663371 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56385a737b30 con 0x56385a85b410 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa53aff5700 10 client.27629 handle_command_reply: tid=0 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 client.27629 shutdown 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 msgr2=0x56385a7fd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56385a7f4ed0 tx=0x7fa54c025450 comp rx=0 tx=0).stop 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 shutdown_connections 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56385a85b410 0x56385a85d830 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56385a7f6bd0 0x56385a7fb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56385a7eb400 0x56385a7fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 --2- 172.21.15.5:0/973663371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56385a7ebd20 0x56385a7fd5f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 >> 172.21.15.5:0/973663371 conn(0x56385a760260 msgr2=0x56385a7f3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa53aff5700 0 client.27629 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 shutdown_connections 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.448+0000 7fa570ee6b80 1 -- 172.21.15.5:0/973663371 wait complete. 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.449+0000 7fa570ee6b80 20 client.27629 trim_cache size 0 max 16384 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T04:59:29.451 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.449+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 msgr2=0x7fa5400a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.449+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 0x7fa5400a3d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55000f540 tx=0x7fa55000b040 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.449+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 msgr2=0x7fa558199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.449+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e9a0 tx=0x7fa54800ecb0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 shutdown_connections 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa5400a1860 0x7fa5400a3d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa55819da60 0x7fa55819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5581010d0 0x7fa558199110 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 --2- 172.21.15.5:0/3227032729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5581061f0 0x7fa558199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 >> 172.21.15.5:0/3227032729 conn(0x7fa5580fc990 msgr2=0x7fa5580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 shutdown_connections 2023-09-23T04:59:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:29.450+0000 7fa55d5ff700 1 -- 172.21.15.5:0/3227032729 wait complete. 2023-09-23T04:59:29.477 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T04:59:30.430 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2356 56.13 MB/sec warmup 5 sec latency 208.791 ms 2023-09-23T04:59:30.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:30 smithi069 ceph-mon[133931]: pgmap v413: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2023-09-23T04:59:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:30 smithi005 ceph-mon[97638]: pgmap v413: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2023-09-23T04:59:31.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:30 smithi118 ceph-mon[134297]: pgmap v413: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2023-09-23T04:59:31.430 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2836 54.61 MB/sec warmup 6 sec latency 242.299 ms 2023-09-23T04:59:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:31 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:32.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:31 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:32.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:31 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:32.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3409 53.17 MB/sec warmup 7 sec latency 112.139 ms 2023-09-23T04:59:32.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:32 smithi069 ceph-mon[133931]: pgmap v414: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2023-09-23T04:59:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:32 smithi005 ceph-mon[97638]: pgmap v414: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2023-09-23T04:59:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:32 smithi118 ceph-mon[134297]: pgmap v414: 129 pgs: 129 active+clean; 570 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2023-09-23T04:59:33.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3713 48.35 MB/sec warmup 8 sec latency 140.705 ms 2023-09-23T04:59:34.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3916 43.78 MB/sec warmup 9 sec latency 55.713 ms 2023-09-23T04:59:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:34 smithi005 ceph-mon[97638]: pgmap v415: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 277 op/s 2023-09-23T04:59:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:34 smithi118 ceph-mon[134297]: pgmap v415: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 277 op/s 2023-09-23T04:59:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:34 smithi069 ceph-mon[133931]: pgmap v415: 129 pgs: 129 active+clean; 598 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 277 op/s 2023-09-23T04:59:35.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4155 39.76 MB/sec warmup 10 sec latency 97.284 ms 2023-09-23T04:59:36.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4350 36.82 MB/sec warmup 11 sec latency 390.582 ms 2023-09-23T04:59:37.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:36 smithi005 ceph-mon[97638]: pgmap v416: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 280 op/s 2023-09-23T04:59:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:36 smithi118 ceph-mon[134297]: pgmap v416: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 280 op/s 2023-09-23T04:59:37.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:36 smithi069 ceph-mon[133931]: pgmap v416: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 280 op/s 2023-09-23T04:59:37.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4410 33.86 MB/sec warmup 12 sec latency 505.877 ms 2023-09-23T04:59:38.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4739 32.72 MB/sec warmup 13 sec latency 446.250 ms 2023-09-23T04:59:39.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:38 smithi005 ceph-mon[97638]: pgmap v417: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 269 op/s 2023-09-23T04:59:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:38 smithi118 ceph-mon[134297]: pgmap v417: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 269 op/s 2023-09-23T04:59:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:38 smithi069 ceph-mon[133931]: pgmap v417: 129 pgs: 129 active+clean; 612 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 269 op/s 2023-09-23T04:59:39.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5112 32.00 MB/sec warmup 14 sec latency 93.472 ms 2023-09-23T04:59:40.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5771 30.96 MB/sec warmup 15 sec latency 59.188 ms 2023-09-23T04:59:41.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:40 smithi005 ceph-mon[97638]: pgmap v418: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 422 op/s 2023-09-23T04:59:41.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:40 smithi118 ceph-mon[134297]: pgmap v418: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 422 op/s 2023-09-23T04:59:41.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:40 smithi069 ceph-mon[133931]: pgmap v418: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 422 op/s 2023-09-23T04:59:41.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6285 31.99 MB/sec warmup 16 sec latency 61.551 ms 2023-09-23T04:59:42.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6902 32.84 MB/sec warmup 17 sec latency 58.523 ms 2023-09-23T04:59:43.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:42 smithi005 ceph-mon[97638]: pgmap v419: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 21 MiB/s wr, 301 op/s 2023-09-23T04:59:43.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:42 smithi118 ceph-mon[134297]: pgmap v419: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 21 MiB/s wr, 301 op/s 2023-09-23T04:59:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:42 smithi069 ceph-mon[133931]: pgmap v419: 129 pgs: 129 active+clean; 799 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 21 MiB/s wr, 301 op/s 2023-09-23T04:59:43.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7133 31.95 MB/sec warmup 18 sec latency 96.446 ms 2023-09-23T04:59:44.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7258 30.40 MB/sec warmup 19 sec latency 266.821 ms 2023-09-23T04:59:45.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:44 smithi005 ceph-mon[97638]: pgmap v420: 129 pgs: 129 active+clean; 802 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 376 op/s 2023-09-23T04:59:45.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:44 smithi118 ceph-mon[134297]: pgmap v420: 129 pgs: 129 active+clean; 802 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 376 op/s 2023-09-23T04:59:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:44 smithi069 ceph-mon[133931]: pgmap v420: 129 pgs: 129 active+clean; 802 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 22 MiB/s wr, 376 op/s 2023-09-23T04:59:45.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7395 28.92 MB/sec warmup 20 sec latency 439.500 ms 2023-09-23T04:59:46.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:45 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:59:46.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:45 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:59:46.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:45 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T04:59:46.431 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7660 27.78 MB/sec warmup 21 sec latency 68.091 ms 2023-09-23T04:59:47.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:46 smithi005 ceph-mon[97638]: pgmap v421: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 337 op/s 2023-09-23T04:59:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:46 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:59:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:46 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:59:47.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:46 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:59:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:46 smithi118 ceph-mon[134297]: pgmap v421: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 337 op/s 2023-09-23T04:59:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:46 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:59:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:46 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:59:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:46 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:59:47.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:46 smithi069 ceph-mon[133931]: pgmap v421: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 19 MiB/s wr, 337 op/s 2023-09-23T04:59:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:46 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T04:59:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:46 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T04:59:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:46 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T04:59:47.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8005 27.22 MB/sec warmup 22 sec latency 87.916 ms 2023-09-23T04:59:48.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8484 27.43 MB/sec warmup 23 sec latency 207.690 ms 2023-09-23T04:59:49.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:48 smithi005 ceph-mon[97638]: pgmap v422: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 293 op/s 2023-09-23T04:59:49.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:48 smithi118 ceph-mon[134297]: pgmap v422: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 293 op/s 2023-09-23T04:59:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:48 smithi069 ceph-mon[133931]: pgmap v422: 129 pgs: 129 active+clean; 807 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 293 op/s 2023-09-23T04:59:49.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9083 27.01 MB/sec warmup 24 sec latency 235.932 ms 2023-09-23T04:59:50.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9472 26.77 MB/sec warmup 25 sec latency 405.906 ms 2023-09-23T04:59:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:50 smithi005 ceph-mon[97638]: pgmap v423: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 380 op/s 2023-09-23T04:59:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:50 smithi118 ceph-mon[134297]: pgmap v423: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 380 op/s 2023-09-23T04:59:51.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:50 smithi069 ceph-mon[133931]: pgmap v423: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 380 op/s 2023-09-23T04:59:51.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10151 27.95 MB/sec warmup 26 sec latency 70.841 ms 2023-09-23T04:59:52.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:51 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:51 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:52.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:51 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:52.432 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10713 28.37 MB/sec warmup 27 sec latency 57.526 ms 2023-09-23T04:59:53.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:52 smithi005 ceph-mon[97638]: pgmap v424: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:59:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:52 smithi118 ceph-mon[134297]: pgmap v424: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:59:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:52 smithi069 ceph-mon[133931]: pgmap v424: 129 pgs: 129 active+clean; 819 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2023-09-23T04:59:53.434 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10877 27.49 MB/sec warmup 28 sec latency 612.274 ms 2023-09-23T04:59:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:53 smithi005 ceph-mon[97638]: pgmap v425: 129 pgs: 129 active+clean; 833 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-09-23T04:59:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:53 smithi118 ceph-mon[134297]: pgmap v425: 129 pgs: 129 active+clean; 833 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-09-23T04:59:54.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:53 smithi069 ceph-mon[133931]: pgmap v425: 129 pgs: 129 active+clean; 833 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2023-09-23T04:59:54.435 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11080 26.97 MB/sec warmup 29 sec latency 1148.364 ms 2023-09-23T04:59:55.436 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11545 26.63 MB/sec warmup 30 sec latency 80.936 ms 2023-09-23T04:59:56.438 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11907 26.46 MB/sec warmup 31 sec latency 306.302 ms 2023-09-23T04:59:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:56 smithi069 ceph-mon[133931]: pgmap v426: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2023-09-23T04:59:56.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:56 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:57.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:56 smithi005 ceph-mon[97638]: pgmap v426: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2023-09-23T04:59:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:56 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:56 smithi118 ceph-mon[134297]: pgmap v426: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 292 op/s 2023-09-23T04:59:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:56 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T04:59:57.438 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12582 26.48 MB/sec warmup 32 sec latency 56.941 ms 2023-09-23T04:59:58.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13799 27.37 MB/sec warmup 33 sec latency 96.117 ms 2023-09-23T04:59:58.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 04:59:58 smithi069 ceph-mon[133931]: pgmap v427: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2023-09-23T04:59:59.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 04:59:58 smithi005 ceph-mon[97638]: pgmap v427: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2023-09-23T04:59:59.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 04:59:58 smithi118 ceph-mon[134297]: pgmap v427: 129 pgs: 129 active+clean; 851 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2023-09-23T04:59:59.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14698 28.58 MB/sec warmup 34 sec latency 152.405 ms 2023-09-23T04:59:59.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T04:59:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 -- 172.21.15.5:0/1616159201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 msgr2=0x7f31141014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1616159201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f31141014b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3110009990 tx=0x7f3110009ca0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 -- 172.21.15.5:0/1616159201 shutdown_connections 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1616159201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 0x7f311410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1616159201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f31141014b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1616159201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114101a80 0x7f3114105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 -- 172.21.15.5:0/1616159201 >> 172.21.15.5:0/1616159201 conn(0x7f31140fc990 msgr2=0x7f31140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 -- 172.21.15.5:0/1616159201 shutdown_connections 2023-09-23T04:59:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.751+0000 7f311b9ff700 1 -- 172.21.15.5:0/1616159201 wait complete. 2023-09-23T04:59:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.753+0000 7f311b9ff700 1 Processor -- start 2023-09-23T04:59:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.753+0000 7f311b9ff700 1 -- start start 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 0x7f3114073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 0x7f31141a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a5710 con 0x7f3114073ba0 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a5880 con 0x7f31141010d0 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311b9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a5a40 con 0x7f31141061f0 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34918/0 (socket says 172.21.15.5:34918) 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 -- 172.21.15.5:0/1878165804 learned_addr learned my addr 172.21.15.5:0/1878165804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:59.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f3119f9c700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 0x7f31141a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f3118f9a700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 0x7f3114073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 -- 172.21.15.5:0/1878165804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 msgr2=0x7f3114073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 0x7f3114073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 -- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 msgr2=0x7f31141a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 0x7f31141a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f311979b700 1 -- 172.21.15.5:0/1878165804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009520 con 0x7f31141010d0 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.754+0000 7f3119f9c700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 0x7f31141a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T04:59:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.755+0000 7f311979b700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3110005720 tx=0x7f3110005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.755+0000 7f310a7fc700 1 -- 172.21.15.5:0/1878165804 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311001d070 con 0x7f31141010d0 2023-09-23T04:59:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.755+0000 7f310a7fc700 1 -- 172.21.15.5:0/1878165804 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3110022470 con 0x7f31141010d0 2023-09-23T04:59:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.755+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31100095f0 con 0x7f31141010d0 2023-09-23T04:59:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.755+0000 7f310a7fc700 1 -- 172.21.15.5:0/1878165804 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110017650 con 0x7f31141010d0 2023-09-23T04:59:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.756+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141a6000 con 0x7f31141010d0 2023-09-23T04:59:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.759+0000 7f310a7fc700 1 -- 172.21.15.5:0/1878165804 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f31100225e0 con 0x7f31141010d0 2023-09-23T04:59:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 Processor -- start 2023-09-23T04:59:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 -- start start 2023-09-23T04:59:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f310a7fc700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 0x7f31000a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55850280d460 con 0x5585028a4d20 2023-09-23T04:59:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585027fec20 con 0x5585028a4400 2023-09-23T04:59:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f312f261b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585027fc540 con 0x5585028afbd0 2023-09-23T04:59:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f310a7fc700 1 -- 172.21.15.5:0/1878165804 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f31100ce9d0 con 0x7f31141010d0 2023-09-23T04:59:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f311979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.761+0000 7f3119f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55156/0 (socket says 172.21.15.5:55156) 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 -- 172.21.15.5:0/2472236540 learned_addr learned my addr 172.21.15.5:0/2472236540 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f3119f9c700 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f3118f9a700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 0x7f31000a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 -- 172.21.15.5:0/2472236540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 msgr2=0x5585028b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 -- 172.21.15.5:0/2472236540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 msgr2=0x5585028a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 -- 172.21.15.5:0/2472236540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850285a8d0 con 0x5585028a4d20 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f3118f9a700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 0x7f31000a3c50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3114074910 tx=0x7f3104009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.762+0000 7f311979b700 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f31100980d0 tx=0x7f3110097ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b3dca987ead615 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f30ff7fe700 1 -- 172.21.15.5:0/2472236540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009afc0 con 0x5585028a4d20 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f30ff7fe700 1 -- 172.21.15.5:0/2472236540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f311009bd60 con 0x5585028a4d20 2023-09-23T04:59:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f30ff7fe700 1 -- 172.21.15.5:0/2472236540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009b8c0 con 0x5585028a4d20 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 -- 172.21.15.5:0/2472236540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 msgr2=0x5585028af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f31100980d0 tx=0x7f3110097ac0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 -- 172.21.15.5:0/2472236540 shutdown_connections 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 --2- 172.21.15.5:0/2472236540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028af690 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.763+0000 7f312f261b80 1 -- 172.21.15.5:0/2472236540 >> 172.21.15.5:0/2472236540 conn(0x558502819260 msgr2=0x5585028a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.764+0000 7f312f261b80 1 -- 172.21.15.5:0/2472236540 shutdown_connections 2023-09-23T04:59:59.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.764+0000 7f312f261b80 1 -- 172.21.15.5:0/2472236540 wait complete. 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.764+0000 7f312f261b80 1 -- start start 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.764+0000 7f312f261b80 1 Processor -- start 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585027f7e30 con 0x5585028a4d20 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558502794710 con 0x5585028a4400 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585027943b0 con 0x5585028afbd0 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f312f261b80 10 client.0 _ll_register_callbacks cb 0x5585025f1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T04:59:59.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f3118f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f311979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f311979b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34934/0 (socket says 172.21.15.5:34934) 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f311979b700 1 -- 172.21.15.5:0/176562207 learned_addr learned my addr 172.21.15.5:0/176562207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.765+0000 7f3119f9c700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 -- 172.21.15.5:0/176562207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 msgr2=0x5585028b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 -- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 msgr2=0x5585028b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 -- 172.21.15.5:0/176562207 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141a6000 con 0x5585028a4400 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f311979b700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f311000f740 tx=0x7f3110022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6004f6d0cbbdb481 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T04:59:59.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f30fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009afc0 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f311009b660 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558502898250 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311009f7b0 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f312f261b80 10 client.27641 fetch_fsmap learned FSMap version 61 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.766+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x558502622de0 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558502898250 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 61) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f31100b1020 con 0x5585028a4400 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 10 client.27641 fetch_fsmap finished waiting for FSMap version 61 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 10 client.27641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 20 client.27641 populate_metadata read hostname 'smithi005' 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5585028beea0 0x5585028c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T04:59:59.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 4 client.27641 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T04:59:59.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.767+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585027f0e10 con 0x5585028beea0 2023-09-23T04:59:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.984+0000 7f3118f9a700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5585028beea0 0x5585028c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T04:59:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.986+0000 7f3118f9a700 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5585028beea0 0x5585028c12c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546acd008dc7e0d5 server_cookie=e4a7ef7f881d3609 in_seq=0 out_seq=0 2023-09-23T04:59:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.986+0000 7f30fcff9700 10 client.27641 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T04:59:59.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.997+0000 7f30fcff9700 1 -- 172.21.15.5:0/176562207 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5585027f0e10 con 0x5585028beea0 2023-09-23T04:59:59.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.997+0000 7f30fcff9700 10 client.27641 handle_command_reply: tid=0 2023-09-23T04:59:59.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.997+0000 7f312f261b80 1 client.27641 shutdown 2023-09-23T04:59:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.998+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 msgr2=0x5585028affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T04:59:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T04:59:59.998+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f311000f740 tx=0x7f3110022af0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 shutdown_connections 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5585028beea0 0x5585028c12c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028afbd0 0x5585028b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028a4400 0x5585028affb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 --2- 172.21.15.5:0/176562207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x5585028b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 >> 172.21.15.5:0/176562207 conn(0x558502819260 msgr2=0x5585028a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:00.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.002+0000 7f30fcff9700 0 client.27641 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:00.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.004+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 shutdown_connections 2023-09-23T05:00:00.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.007+0000 7f312f261b80 1 -- 172.21.15.5:0/176562207 wait complete. 2023-09-23T05:00:00.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.009+0000 7f312f261b80 20 client.27641 trim_cache size 0 max 16384 2023-09-23T05:00:00.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.039+0000 7f312f261b80 1 Processor -- start 2023-09-23T05:00:00.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.039+0000 7f312f261b80 1 -- start start 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4400 0x5585028b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x5585028aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558502678920 con 0x5585028a4400 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55850289cd60 con 0x5585028afbd0 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f312f261b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585028a5140 con 0x5585028a4d20 2023-09-23T05:00:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f3118f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f3118f9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:36200/0 (socket says 172.21.15.5:36200) 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.040+0000 7f3118f9a700 1 -- 172.21.15.5:0/3333100423 learned_addr learned my addr 172.21.15.5:0/3333100423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 -- 172.21.15.5:0/3333100423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 msgr2=0x5585028aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x5585028aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 -- 172.21.15.5:0/3333100423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4400 msgr2=0x5585028b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4400 0x5585028b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 -- 172.21.15.5:0/3333100423 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850289c480 con 0x5585028a4d20 2023-09-23T05:00:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.041+0000 7f3118f9a700 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5585028af320 tx=0x7f3104014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b7ce615fbe481c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.042+0000 7f30fffff700 1 -- 172.21.15.5:0/3333100423 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104025070 con 0x5585028a4d20 2023-09-23T05:00:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.042+0000 7f30fffff700 1 -- 172.21.15.5:0/3333100423 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3104026b00 con 0x5585028a4d20 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.042+0000 7f30fffff700 1 -- 172.21.15.5:0/3333100423 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104003b50 con 0x5585028a4d20 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 -- 172.21.15.5:0/3333100423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 msgr2=0x5585028a9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5585028af320 tx=0x7f3104014000 comp rx=0 tx=0).stop 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 -- 172.21.15.5:0/3333100423 shutdown_connections 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x5585028aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4400 0x5585028b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 --2- 172.21.15.5:0/3333100423 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4d20 0x5585028a9fb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.043+0000 7f312f261b80 1 -- 172.21.15.5:0/3333100423 >> 172.21.15.5:0/3333100423 conn(0x558502819260 msgr2=0x5585028bb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.044+0000 7f312f261b80 1 -- 172.21.15.5:0/3333100423 shutdown_connections 2023-09-23T05:00:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.044+0000 7f312f261b80 1 -- 172.21.15.5:0/3333100423 wait complete. 2023-09-23T05:00:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.045+0000 7f312f261b80 1 -- start start 2023-09-23T05:00:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.045+0000 7f312f261b80 1 Processor -- start 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x55850290ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x558502906c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558502907300 con 0x5585028a4d20 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558502907470 con 0x5585028afbd0 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558502907630 con 0x5585028a4400 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f312f261b80 10 client.0 _ll_register_callbacks cb 0x5585025f1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f311979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f311979b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:36214/0 (socket says 172.21.15.5:36214) 2023-09-23T05:00:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.046+0000 7f311979b700 1 -- 172.21.15.5:0/4213476031 learned_addr learned my addr 172.21.15.5:0/4213476031 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f3119f9c700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x558502906c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f3118f9a700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x55850290ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f311979b700 1 -- 172.21.15.5:0/4213476031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 msgr2=0x558502906c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f311979b700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x558502906c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f311979b700 1 -- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 msgr2=0x55850290ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f311979b700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x55850290ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f311979b700 1 -- 172.21.15.5:0/4213476031 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3104017070 con 0x5585028a4400 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.047+0000 7f3118f9a700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x55850290ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.048+0000 7f311979b700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31100a3430 tx=0x7f3110005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3008bd2edb567128 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.048+0000 7f30fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.048+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311009b660 con 0x5585028a4400 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.048+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f311009f7b0 con 0x5585028a4400 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.049+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f311009bd60 con 0x5585028a4400 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.049+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31100a3960 con 0x5585028a4400 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 10 client.27706 fetch_fsmap learned FSMap version 61 2023-09-23T05:00:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55850260bc60 con 0x5585028a4400 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f311009bd60 con 0x5585028a4400 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 61) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f31100b1070 con 0x5585028a4400 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 10 client.27706 fetch_fsmap finished waiting for FSMap version 61 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 10 client.27706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 20 client.27706 populate_metadata read hostname 'smithi005' 2023-09-23T05:00:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558502913e60 0x558502916250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 4 client.27706 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:00:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.051+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585027f0b30 con 0x558502913e60 2023-09-23T05:00:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.052+0000 7f3118f9a700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558502913e60 0x558502916250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.053+0000 7f30fcff9700 10 client.27706 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.053+0000 7f3118f9a700 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558502913e60 0x558502916250 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8239fef1523b55c9 server_cookie=fc6bbe2c197ca74 in_seq=0 out_seq=0 2023-09-23T05:00:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.064+0000 7f30fcff9700 1 -- 172.21.15.5:0/4213476031 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5585027f0b30 con 0x558502913e60 2023-09-23T05:00:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.065+0000 7f30fcff9700 10 client.27706 handle_command_reply: tid=0 2023-09-23T05:00:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.066+0000 7f312f261b80 1 client.27706 shutdown 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 msgr2=0x55850290c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31100a3430 tx=0x7f3110005c70 comp rx=0 tx=0).stop 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 shutdown_connections 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558502913e60 0x558502916250 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5585028afbd0 0x558502906c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5585028a4400 0x55850290c520 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 --2- 172.21.15.5:0/4213476031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585028a4d20 0x55850290ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.067+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 >> 172.21.15.5:0/4213476031 conn(0x558502819260 msgr2=0x5585028a8410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.069+0000 7f30fcff9700 0 client.27706 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.069+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 shutdown_connections 2023-09-23T05:00:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.070+0000 7f312f261b80 1 -- 172.21.15.5:0/4213476031 wait complete. 2023-09-23T05:00:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.076+0000 7f312f261b80 20 client.27706 trim_cache size 0 max 16384 2023-09-23T05:00:00.076 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:00:00.077 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.077+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 msgr2=0x7f31000a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.077+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 0x7f31000a3c50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3114074910 tx=0x7f3104009260 comp rx=0 tx=0).stop 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.077+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 msgr2=0x7f3114073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.077+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3110005720 tx=0x7f3110005750 comp rx=0 tx=0).stop 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 shutdown_connections 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f31000a1790 0x7f31000a3c50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3114073ba0 0x7f31141a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31141010d0 0x7f3114073120 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3110005720 tx=0x7f3110005750 comp rx=0 tx=0).stop 2023-09-23T05:00:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 --2- 172.21.15.5:0/1878165804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f31141061f0 0x7f3114073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:00.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 >> 172.21.15.5:0/1878165804 conn(0x7f31140fc990 msgr2=0x7f3114104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:00.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 shutdown_connections 2023-09-23T05:00:00.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:00.078+0000 7f311b9ff700 1 -- 172.21.15.5:0/1878165804 wait complete. 2023-09-23T05:00:00.106 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:00:00.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15127 28.54 MB/sec warmup 35 sec latency 433.812 ms 2023-09-23T05:00:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:00 smithi069 ceph-mon[133931]: pgmap v428: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-09-23T05:00:00.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:00 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T05:00:01.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:00 smithi005 ceph-mon[97638]: pgmap v428: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-09-23T05:00:01.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:00 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T05:00:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:00 smithi118 ceph-mon[134297]: pgmap v428: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-09-23T05:00:01.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:00 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T05:00:01.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15420 28.19 MB/sec warmup 36 sec latency 457.975 ms 2023-09-23T05:00:02.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:01 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:01 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:01 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:02.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15420 27.43 MB/sec warmup 37 sec latency 1458.023 ms 2023-09-23T05:00:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:02 smithi005 ceph-mon[97638]: pgmap v429: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 427 op/s 2023-09-23T05:00:03.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:02 smithi118 ceph-mon[134297]: pgmap v429: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 427 op/s 2023-09-23T05:00:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:02 smithi069 ceph-mon[133931]: pgmap v429: 129 pgs: 129 active+clean; 858 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.8 MiB/s wr, 427 op/s 2023-09-23T05:00:03.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15420 26.71 MB/sec warmup 38 sec latency 2458.075 ms 2023-09-23T05:00:04.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15420 26.02 MB/sec warmup 39 sec latency 3458.126 ms 2023-09-23T05:00:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:04 smithi005 ceph-mon[97638]: pgmap v430: 129 pgs: 129 active+clean; 871 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 563 op/s 2023-09-23T05:00:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:04 smithi118 ceph-mon[134297]: pgmap v430: 129 pgs: 129 active+clean; 871 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 563 op/s 2023-09-23T05:00:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:04 smithi069 ceph-mon[133931]: pgmap v430: 129 pgs: 129 active+clean; 871 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 563 op/s 2023-09-23T05:00:05.440 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15587 25.43 MB/sec warmup 40 sec latency 4108.822 ms 2023-09-23T05:00:06.441 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15859 24.97 MB/sec warmup 41 sec latency 309.046 ms 2023-09-23T05:00:07.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:06 smithi005 ceph-mon[97638]: pgmap v431: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 494 op/s 2023-09-23T05:00:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:06 smithi005 ceph-mon[97638]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T05:00:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:06 smithi118 ceph-mon[134297]: pgmap v431: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 494 op/s 2023-09-23T05:00:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:06 smithi118 ceph-mon[134297]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T05:00:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:06 smithi069 ceph-mon[133931]: pgmap v431: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 11 MiB/s wr, 494 op/s 2023-09-23T05:00:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:06 smithi069 ceph-mon[133931]: osdmap e107: 12 total, 12 up, 12 in 2023-09-23T05:00:07.441 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15907 24.49 MB/sec warmup 42 sec latency 217.318 ms 2023-09-23T05:00:08.441 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16239 24.45 MB/sec warmup 43 sec latency 186.600 ms 2023-09-23T05:00:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:08 smithi005 ceph-mon[97638]: pgmap v433: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 527 op/s 2023-09-23T05:00:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:08 smithi118 ceph-mon[134297]: pgmap v433: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 527 op/s 2023-09-23T05:00:09.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:08 smithi069 ceph-mon[133931]: pgmap v433: 129 pgs: 129 active+clean; 863 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 527 op/s 2023-09-23T05:00:09.442 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17328 24.95 MB/sec warmup 44 sec latency 179.557 ms 2023-09-23T05:00:10.442 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18285 25.91 MB/sec warmup 45 sec latency 77.722 ms 2023-09-23T05:00:11.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:10 smithi005 ceph-mon[97638]: pgmap v434: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2023-09-23T05:00:11.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:10 smithi005 ceph-mon[97638]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T05:00:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:10 smithi118 ceph-mon[134297]: pgmap v434: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2023-09-23T05:00:11.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:10 smithi118 ceph-mon[134297]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T05:00:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:10 smithi069 ceph-mon[133931]: pgmap v434: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2023-09-23T05:00:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:10 smithi069 ceph-mon[133931]: osdmap e108: 12 total, 12 up, 12 in 2023-09-23T05:00:11.442 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18764 25.83 MB/sec warmup 46 sec latency 1077.775 ms 2023-09-23T05:00:12.442 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19598 26.02 MB/sec warmup 47 sec latency 2077.840 ms 2023-09-23T05:00:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:12 smithi005 ceph-mon[97638]: pgmap v436: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-23T05:00:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:12 smithi118 ceph-mon[134297]: pgmap v436: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-23T05:00:13.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:12 smithi069 ceph-mon[133931]: pgmap v436: 129 pgs: 129 active+clean; 851 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2023-09-23T05:00:13.442 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20777 26.91 MB/sec warmup 48 sec latency 3077.902 ms 2023-09-23T05:00:14.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:13 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:14.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:13 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:13 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:14.443 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21622 27.35 MB/sec warmup 49 sec latency 4078.491 ms 2023-09-23T05:00:15.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:14 smithi005 ceph-mon[97638]: pgmap v437: 129 pgs: 129 active+clean; 873 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 657 op/s 2023-09-23T05:00:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:14 smithi118 ceph-mon[134297]: pgmap v437: 129 pgs: 129 active+clean; 873 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 657 op/s 2023-09-23T05:00:15.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:14 smithi069 ceph-mon[133931]: pgmap v437: 129 pgs: 129 active+clean; 873 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 657 op/s 2023-09-23T05:00:15.443 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22218 27.42 MB/sec warmup 50 sec latency 5079.658 ms 2023-09-23T05:00:16.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23190 27.81 MB/sec warmup 51 sec latency 6079.710 ms 2023-09-23T05:00:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:16 smithi005 ceph-mon[97638]: pgmap v438: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 649 op/s 2023-09-23T05:00:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:16 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:16 smithi118 ceph-mon[134297]: pgmap v438: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 649 op/s 2023-09-23T05:00:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:16 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:17.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:16 smithi069 ceph-mon[133931]: pgmap v438: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 649 op/s 2023-09-23T05:00:17.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:16 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:17.443 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23949 28.14 MB/sec warmup 52 sec latency 7079.763 ms 2023-09-23T05:00:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:17 smithi005 ceph-mon[97638]: pgmap v439: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 588 op/s 2023-09-23T05:00:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:17 smithi118 ceph-mon[134297]: pgmap v439: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 588 op/s 2023-09-23T05:00:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:17 smithi069 ceph-mon[133931]: pgmap v439: 129 pgs: 129 active+clean; 880 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 588 op/s 2023-09-23T05:00:18.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24473 28.04 MB/sec warmup 53 sec latency 8079.819 ms 2023-09-23T05:00:19.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24985 28.11 MB/sec warmup 54 sec latency 9079.879 ms 2023-09-23T05:00:20.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:19 smithi069 ceph-mon[133931]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2023-09-23T05:00:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:19 smithi005 ceph-mon[97638]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2023-09-23T05:00:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:19 smithi118 ceph-mon[134297]: pgmap v440: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.4 MiB/s wr, 636 op/s 2023-09-23T05:00:20.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25675 28.15 MB/sec warmup 55 sec latency 10079.941 ms 2023-09-23T05:00:21.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26446 28.62 MB/sec warmup 56 sec latency 11079.999 ms 2023-09-23T05:00:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:21 smithi069 ceph-mon[133931]: scrub summary: active paths [/] 2023-09-23T05:00:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:21 smithi069 ceph-mon[133931]: scrub summary: idle+waiting paths [/] 2023-09-23T05:00:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:21 smithi069 ceph-mon[133931]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2023-09-23T05:00:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:21 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:22.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:21 smithi005 ceph-mon[97638]: scrub summary: active paths [/] 2023-09-23T05:00:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:21 smithi005 ceph-mon[97638]: scrub summary: idle+waiting paths [/] 2023-09-23T05:00:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:21 smithi005 ceph-mon[97638]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2023-09-23T05:00:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:21 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:21 smithi118 ceph-mon[134297]: scrub summary: active paths [/] 2023-09-23T05:00:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:21 smithi118 ceph-mon[134297]: scrub summary: idle+waiting paths [/] 2023-09-23T05:00:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:21 smithi118 ceph-mon[134297]: pgmap v441: 129 pgs: 129 active+clean; 916 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2023-09-23T05:00:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:21 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:22.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27094 28.65 MB/sec warmup 57 sec latency 11654.858 ms 2023-09-23T05:00:23.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28047 29.03 MB/sec warmup 58 sec latency 108.172 ms 2023-09-23T05:00:24.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:23 smithi069 ceph-mon[133931]: pgmap v442: 129 pgs: 129 active+clean; 949 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 730 op/s 2023-09-23T05:00:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:23 smithi005 ceph-mon[97638]: pgmap v442: 129 pgs: 129 active+clean; 949 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 730 op/s 2023-09-23T05:00:24.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:23 smithi118 ceph-mon[134297]: pgmap v442: 129 pgs: 129 active+clean; 949 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 730 op/s 2023-09-23T05:00:24.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28594 29.01 MB/sec warmup 59 sec latency 1108.233 ms 2023-09-23T05:00:25.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28921 28.87 MB/sec warmup 60 sec latency 2108.332 ms 2023-09-23T05:00:26.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29304 28.79 MB/sec warmup 61 sec latency 3108.394 ms 2023-09-23T05:00:27.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:26 smithi005 ceph-mon[97638]: pgmap v443: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2023-09-23T05:00:27.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:26 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:27.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:26 smithi118 ceph-mon[134297]: pgmap v443: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2023-09-23T05:00:27.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:26 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:26 smithi069 ceph-mon[133931]: pgmap v443: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.5 MiB/s wr, 602 op/s 2023-09-23T05:00:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:26 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:27.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29859 28.74 MB/sec warmup 62 sec latency 4108.450 ms 2023-09-23T05:00:28.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30646 28.95 MB/sec warmup 63 sec latency 5108.517 ms 2023-09-23T05:00:29.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:28 smithi005 ceph-mon[97638]: pgmap v444: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.9 MiB/s wr, 550 op/s 2023-09-23T05:00:29.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:28 smithi118 ceph-mon[134297]: pgmap v444: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.9 MiB/s wr, 550 op/s 2023-09-23T05:00:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:28 smithi069 ceph-mon[133931]: pgmap v444: 129 pgs: 129 active+clean; 961 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.9 MiB/s wr, 550 op/s 2023-09-23T05:00:29.444 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31638 29.42 MB/sec warmup 64 sec latency 6108.577 ms 2023-09-23T05:00:30.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:00:30.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32347 29.54 MB/sec warmup 65 sec latency 7108.631 ms 2023-09-23T05:00:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.471+0000 7fabbe3ff700 1 -- 172.21.15.5:0/4076209147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f9d0 msgr2=0x7fabb810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.471+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/4076209147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f9d0 0x7fabb810d7e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.474+0000 7fabbe3ff700 1 -- 172.21.15.5:0/4076209147 shutdown_connections 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.474+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/4076209147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb8107e20 0x7fabb81082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.474+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/4076209147 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fabb810f5f0 0x7fabb810d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.474+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/4076209147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f9d0 0x7fabb810d7e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.474+0000 7fabbe3ff700 1 -- 172.21.15.5:0/4076209147 >> 172.21.15.5:0/4076209147 conn(0x7fabb80725f0 msgr2=0x7fabb806d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.475+0000 7fabbe3ff700 1 -- 172.21.15.5:0/4076209147 shutdown_connections 2023-09-23T05:00:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.476+0000 7fabbe3ff700 1 -- 172.21.15.5:0/4076209147 wait complete. 2023-09-23T05:00:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.478+0000 7fabbe3ff700 1 Processor -- start 2023-09-23T05:00:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.480+0000 7fabbe3ff700 1 -- start start 2023-09-23T05:00:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb8107e20 0x7fabb811b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fabb810f9d0 0x7fabb8116800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8116e90 con 0x7fabb8107e20 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8117000 con 0x7fabb810f5f0 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.481+0000 7fabbe3ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81171c0 con 0x7fabb810f9d0 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59904/0 (socket says 172.21.15.5:59904) 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 -- 172.21.15.5:0/3358748835 learned_addr learned my addr 172.21.15.5:0/3358748835 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 -- 172.21.15.5:0/3358748835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fabb810f9d0 msgr2=0x7fabb8116800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T05:00:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fabb810f9d0 0x7fabb8116800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 -- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb8107e20 msgr2=0x7fabb811b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb8107e20 0x7fabb811b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.482+0000 7fabb77fe700 1 -- 172.21.15.5:0/3358748835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80095f0 con 0x7fabb810f5f0 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.483+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.483+0000 7fabb57fa700 1 -- 172.21.15.5:0/3358748835 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba801d070 con 0x7fabb810f5f0 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.483+0000 7fabb57fa700 1 -- 172.21.15.5:0/3358748835 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba800fb50 con 0x7fabb810f5f0 2023-09-23T05:00:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.483+0000 7fabb57fa700 1 -- 172.21.15.5:0/3358748835 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80173f0 con 0x7fabb810f5f0 2023-09-23T05:00:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.484+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb8117440 con 0x7fabb810f5f0 2023-09-23T05:00:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.484+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb811a5c0 con 0x7fabb810f5f0 2023-09-23T05:00:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.493+0000 7fabd1ddfb80 1 Processor -- start 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.510+0000 7fabb57fa700 1 -- 172.21.15.5:0/3358748835 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7faba800fcc0 con 0x7fabb810f5f0 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 -- start start 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801ae6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801a42460 con 0x560801ae4bd0 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801a33c20 con 0x560801ad9400 2023-09-23T05:00:30.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.511+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801a31540 con 0x560801ad9d20 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.512+0000 7fabbc99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.512+0000 7fabbc99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59908/0 (socket says 172.21.15.5:59908) 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.512+0000 7fabbc99c700 1 -- 172.21.15.5:0/415523344 learned_addr learned my addr 172.21.15.5:0/415523344 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.512+0000 7fabb57fa700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 0x7faba00a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.513+0000 7fabb57fa700 1 -- 172.21.15.5:0/3358748835 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9593+0+0 (secure 0 0 0) 0x7faba80ce830 con 0x7fabb810f5f0 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.513+0000 7fabbc99c700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 0x7faba00a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.514+0000 7fabb77fe700 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801ae6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.515+0000 7fabbc99c700 1 -- 172.21.15.5:0/415523344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 msgr2=0x560801ae4690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:00:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.515+0000 7fabbc99c700 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae4690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.515+0000 7fabbc99c700 1 -- 172.21.15.5:0/415523344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 msgr2=0x560801ae6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.515+0000 7fabbc99c700 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801ae6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.515+0000 7fabbc99c700 1 -- 172.21.15.5:0/415523344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560801a8f8d0 con 0x560801ad9400 2023-09-23T05:00:30.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.517+0000 7fabbc99c700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 0x7faba00a3d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabac0144c0 tx=0x7fabac009770 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.519+0000 7fabbc99c700 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fabac014c10 tx=0x7fabac017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0fadcba877a0023 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.520+0000 7fab9e7fc700 1 -- 172.21.15.5:0/415523344 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac016e20 con 0x560801ad9400 2023-09-23T05:00:30.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.520+0000 7fab9e7fc700 1 -- 172.21.15.5:0/415523344 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabac009200 con 0x560801ad9400 2023-09-23T05:00:30.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.520+0000 7fab9e7fc700 1 -- 172.21.15.5:0/415523344 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac01f660 con 0x560801ad9400 2023-09-23T05:00:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/415523344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 msgr2=0x560801ad97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fabac014c10 tx=0x7fabac017000 comp rx=0 tx=0).stop 2023-09-23T05:00:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/415523344 shutdown_connections 2023-09-23T05:00:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801ae6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801ad97e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/415523344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.521+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/415523344 >> 172.21.15.5:0/415523344 conn(0x560801a4e260 msgr2=0x560801ad5790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.538+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/415523344 shutdown_connections 2023-09-23T05:00:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.541+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/415523344 wait complete. 2023-09-23T05:00:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.542+0000 7fabd1ddfb80 1 -- start start 2023-09-23T05:00:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.543+0000 7fabd1ddfb80 1 Processor -- start 2023-09-23T05:00:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801ae1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 0x560801ae2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801a2ce30 con 0x560801ad9400 2023-09-23T05:00:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608019c9710 con 0x560801ae4bd0 2023-09-23T05:00:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608019c93b0 con 0x560801ad9d20 2023-09-23T05:00:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.545+0000 7fabd1ddfb80 10 client.0 _ll_register_callbacks cb 0x560801826800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:00:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.548+0000 7fabbc99c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 0x560801ae2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.548+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.548+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42772/0 (socket says 172.21.15.5:42772) 2023-09-23T05:00:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.548+0000 7fabb77fe700 1 -- 172.21.15.5:0/2442059089 learned_addr learned my addr 172.21.15.5:0/2442059089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 -- 172.21.15.5:0/2442059089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 msgr2=0x560801ae1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 -- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 msgr2=0x560801ae1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801ae1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 -- 172.21.15.5:0/2442059089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac00f070 con 0x560801ae4bd0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fabbc99c700 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 0x560801ae2380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fabac0048a0 tx=0x7fabac006c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18377272df9848bc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fab977fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac005e60 con 0x560801ae4bd0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.549+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fabac017410 con 0x560801ae4bd0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.550+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560801acd250 con 0x560801ae4bd0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.550+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabac0175f0 con 0x560801ae4bd0 2023-09-23T05:00:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.550+0000 7fabd1ddfb80 10 client.27656 fetch_fsmap learned FSMap version 66 2023-09-23T05:00:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.550+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x560801857de0 con 0x560801ae4bd0 2023-09-23T05:00:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.552+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560801acd250 con 0x560801ae4bd0 2023-09-23T05:00:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.552+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 66) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fabac02e020 con 0x560801ae4bd0 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 10 client.27656 fetch_fsmap finished waiting for FSMap version 66 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 10 client.27656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 20 client.27656 populate_metadata read hostname 'smithi005' 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801af4400 0x560801af6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 4 client.27656 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:00:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.554+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560801a25e10 con 0x560801af4400 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.556+0000 7fabb7fff700 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801af4400 0x560801af6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.557+0000 7fabb7fff700 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801af4400 0x560801af6820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8225f4d70cbd4f server_cookie=d13e4e29de057d43 in_seq=0 out_seq=0 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.557+0000 7fab977fe700 10 client.27656 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.559+0000 7fab977fe700 1 -- 172.21.15.5:0/2442059089 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560801a25e10 con 0x560801af4400 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.559+0000 7fab977fe700 10 client.27656 handle_command_reply: tid=0 2023-09-23T05:00:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.559+0000 7fabd1ddfb80 1 client.27656 shutdown 2023-09-23T05:00:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.563+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 msgr2=0x560801ae2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.563+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 0x560801ae2380 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fabac0048a0 tx=0x7fabac006c80 comp rx=0 tx=0).stop 2023-09-23T05:00:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 shutdown_connections 2023-09-23T05:00:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801af4400 0x560801af6820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ae4bd0 0x560801ae2380 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801ae1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/2442059089 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801ae1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.564+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 >> 172.21.15.5:0/2442059089 conn(0x560801a4e260 msgr2=0x560801ad5320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.565+0000 7fab977fe700 0 client.27656 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.566+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 shutdown_connections 2023-09-23T05:00:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.566+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/2442059089 wait complete. 2023-09-23T05:00:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.567+0000 7fabd1ddfb80 20 client.27656 trim_cache size 0 max 16384 2023-09-23T05:00:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.603+0000 7fabd1ddfb80 1 Processor -- start 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 -- start start 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801adc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ae4bd0 0x560801adcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608018ad920 con 0x560801ad9400 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801adaec0 con 0x560801ad9d20 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.607+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801aebb90 con 0x560801ae4bd0 2023-09-23T05:00:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.609+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.609+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59918/0 (socket says 172.21.15.5:59918) 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.609+0000 7fabb77fe700 1 -- 172.21.15.5:0/3117063465 learned_addr learned my addr 172.21.15.5:0/3117063465 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.611+0000 7fabb77fe700 1 -- 172.21.15.5:0/3117063465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ae4bd0 msgr2=0x560801adcc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.611+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ae4bd0 0x560801adcc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.611+0000 7fabb77fe700 1 -- 172.21.15.5:0/3117063465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 msgr2=0x560801adc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.611+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801adc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.611+0000 7fabb77fe700 1 -- 172.21.15.5:0/3117063465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560801ad1480 con 0x560801ad9d20 2023-09-23T05:00:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.612+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faba80978a0 tx=0x7faba809b010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74d32149cf45459 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.612+0000 7fab9effd700 1 -- 172.21.15.5:0/3117063465 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba809b3a0 con 0x560801ad9d20 2023-09-23T05:00:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.612+0000 7fab9effd700 1 -- 172.21.15.5:0/3117063465 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba809f4f0 con 0x560801ad9d20 2023-09-23T05:00:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.612+0000 7fab9effd700 1 -- 172.21.15.5:0/3117063465 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba809aaf0 con 0x560801ad9d20 2023-09-23T05:00:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.613+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3117063465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 msgr2=0x560801aeb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.613+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faba80978a0 tx=0x7faba809b010 comp rx=0 tx=0).stop 2023-09-23T05:00:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.614+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3117063465 shutdown_connections 2023-09-23T05:00:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.614+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ae4bd0 0x560801adcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.614+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ad9400 0x560801adc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.614+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3117063465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9d20 0x560801aeb650 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.614+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3117063465 >> 172.21.15.5:0/3117063465 conn(0x560801a4e260 msgr2=0x560801af6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.620+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3117063465 shutdown_connections 2023-09-23T05:00:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.621+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3117063465 wait complete. 2023-09-23T05:00:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.621+0000 7fabd1ddfb80 1 -- start start 2023-09-23T05:00:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.621+0000 7fabd1ddfb80 1 Processor -- start 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801aeaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801aeafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801b47230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801b47770 con 0x560801ae4bd0 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801b478b0 con 0x560801ad9400 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560801b479f0 con 0x560801ad9d20 2023-09-23T05:00:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.622+0000 7fabd1ddfb80 10 client.0 _ll_register_callbacks cb 0x560801826800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:00:30.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.624+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801aeafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.624+0000 7fabb77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801aeafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42778/0 (socket says 172.21.15.5:42778) 2023-09-23T05:00:30.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.624+0000 7fabb77fe700 1 -- 172.21.15.5:0/3616376481 learned_addr learned my addr 172.21.15.5:0/3616376481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:00:30.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.626+0000 7fabb7fff700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801aeaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.627+0000 7fabb7fff700 1 -- 172.21.15.5:0/3616376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 msgr2=0x560801aeafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.627+0000 7fabb7fff700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801aeafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.627+0000 7fabb7fff700 1 -- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 msgr2=0x560801b47230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:00:30.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.627+0000 7fabb7fff700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801b47230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.627+0000 7fabb7fff700 1 -- 172.21.15.5:0/3616376481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb811a5c0 con 0x560801ad9400 2023-09-23T05:00:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.631+0000 7fabb7fff700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801aeaa70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x560801ad9cb0 tx=0x7fabb0018c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=560d6b7b3de2a311 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:00:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.631+0000 7fab977fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:00:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.640+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0010040 con 0x560801ad9400 2023-09-23T05:00:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.640+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fabb0018e40 con 0x560801ad9400 2023-09-23T05:00:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.640+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabac009200 con 0x560801ad9400 2023-09-23T05:00:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.641+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb0004db0 con 0x560801ad9400 2023-09-23T05:00:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.646+0000 7fabd1ddfb80 10 client.27662 fetch_fsmap learned FSMap version 66 2023-09-23T05:00:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.646+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x560801840c60 con 0x560801ad9400 2023-09-23T05:00:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabac009200 con 0x560801ad9400 2023-09-23T05:00:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 66) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fabb0004050 con 0x560801ad9400 2023-09-23T05:00:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 10 client.27662 fetch_fsmap finished waiting for FSMap version 66 2023-09-23T05:00:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 10 client.27662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:00:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 20 client.27662 populate_metadata read hostname 'smithi005' 2023-09-23T05:00:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801b6b2d0 0x560801b6d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:00:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 4 client.27662 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:00:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.652+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560801a25b30 con 0x560801b6b2d0 2023-09-23T05:00:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.658+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801b6b2d0 0x560801b6d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:00:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.663+0000 7fabb77fe700 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801b6b2d0 0x560801b6d6f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3356a2062c2b808 server_cookie=1c0bd61ca9fd19ce in_seq=0 out_seq=0 2023-09-23T05:00:30.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.668+0000 7fab977fe700 10 client.27662 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:30.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.668+0000 7fab977fe700 1 -- 172.21.15.5:0/3616376481 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560801a25b30 con 0x560801b6b2d0 2023-09-23T05:00:30.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.668+0000 7fab977fe700 10 client.27662 handle_command_reply: tid=0 2023-09-23T05:00:30.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.668+0000 7fabd1ddfb80 1 client.27662 shutdown 2023-09-23T05:00:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.672+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 msgr2=0x560801aeaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.672+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801aeaa70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x560801ad9cb0 tx=0x7fabb0018c60 comp rx=0 tx=0).stop 2023-09-23T05:00:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 shutdown_connections 2023-09-23T05:00:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560801b6b2d0 0x560801b6d6f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560801ae4bd0 0x560801b47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560801ad9400 0x560801aeaa70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 --2- 172.21.15.5:0/3616376481 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560801ad9d20 0x560801aeafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 >> 172.21.15.5:0/3616376481 conn(0x560801a4e260 msgr2=0x560801adcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.675+0000 7fab977fe700 0 client.27662 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:00:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.676+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 shutdown_connections 2023-09-23T05:00:30.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.676+0000 7fabd1ddfb80 1 -- 172.21.15.5:0/3616376481 wait complete. 2023-09-23T05:00:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.679+0000 7fabd1ddfb80 20 client.27662 trim_cache size 0 max 16384 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:00:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:00:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:00:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 msgr2=0x7faba00a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 0x7faba00a3d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabac0144c0 tx=0x7fabac009770 comp rx=0 tx=0).stop 2023-09-23T05:00:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 msgr2=0x7fabb811bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:00:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba8000c00 tx=0x7faba80057f0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 shutdown_connections 2023-09-23T05:00:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7faba00a1850 0x7faba00a3d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fabb810f9d0 0x7fabb8116800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb8107e20 0x7fabb811b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 --2- 172.21.15.5:0/3358748835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabb810f5f0 0x7fabb811bbc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:00:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 >> 172.21.15.5:0/3358748835 conn(0x7fabb80725f0 msgr2=0x7fabb8109c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:00:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 shutdown_connections 2023-09-23T05:00:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:00:30.687+0000 7fabbe3ff700 1 -- 172.21.15.5:0/3358748835 wait complete. 2023-09-23T05:00:30.722 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:00:31.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:30 smithi005 ceph-mon[97638]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 961 op/s 2023-09-23T05:00:31.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:30 smithi118 ceph-mon[134297]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 961 op/s 2023-09-23T05:00:31.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:30 smithi069 ceph-mon[133931]: pgmap v445: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 961 op/s 2023-09-23T05:00:31.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32994 29.55 MB/sec warmup 66 sec latency 7356.918 ms 2023-09-23T05:00:32.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33768 29.76 MB/sec warmup 67 sec latency 6480.277 ms 2023-09-23T05:00:33.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:32 smithi005 ceph-mon[97638]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-09-23T05:00:33.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:32 smithi118 ceph-mon[134297]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-09-23T05:00:33.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:32 smithi069 ceph-mon[133931]: pgmap v446: 129 pgs: 129 active+clean; 972 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 698 op/s 2023-09-23T05:00:33.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34456 29.88 MB/sec warmup 68 sec latency 7480.328 ms 2023-09-23T05:00:34.445 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35301 30.08 MB/sec warmup 69 sec latency 8481.118 ms 2023-09-23T05:00:35.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:34 smithi005 ceph-mon[97638]: pgmap v447: 129 pgs: 129 active+clean; 995 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 906 op/s 2023-09-23T05:00:35.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:34 smithi118 ceph-mon[134297]: pgmap v447: 129 pgs: 129 active+clean; 995 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 906 op/s 2023-09-23T05:00:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:34 smithi069 ceph-mon[133931]: pgmap v447: 129 pgs: 129 active+clean; 995 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 906 op/s 2023-09-23T05:00:35.447 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36085 30.30 MB/sec warmup 70 sec latency 9483.097 ms 2023-09-23T05:00:36.451 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36707 30.29 MB/sec warmup 71 sec latency 9675.280 ms 2023-09-23T05:00:37.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:36 smithi005 ceph-mon[97638]: pgmap v448: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 759 op/s 2023-09-23T05:00:37.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:36 smithi118 ceph-mon[134297]: pgmap v448: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 759 op/s 2023-09-23T05:00:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:36 smithi069 ceph-mon[133931]: pgmap v448: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 759 op/s 2023-09-23T05:00:37.450 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37462 30.41 MB/sec warmup 72 sec latency 48.042 ms 2023-09-23T05:00:38.450 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38308 30.63 MB/sec warmup 73 sec latency 63.214 ms 2023-09-23T05:00:39.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:38 smithi005 ceph-mon[97638]: pgmap v449: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 671 op/s 2023-09-23T05:00:39.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:38 smithi118 ceph-mon[134297]: pgmap v449: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 671 op/s 2023-09-23T05:00:39.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:38 smithi069 ceph-mon[133931]: pgmap v449: 129 pgs: 129 active+clean; 1000 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 12 MiB/s wr, 671 op/s 2023-09-23T05:00:39.451 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39224 30.92 MB/sec warmup 74 sec latency 56.441 ms 2023-09-23T05:00:40.451 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39556 30.74 MB/sec warmup 75 sec latency 391.248 ms 2023-09-23T05:00:41.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:40 smithi005 ceph-mon[97638]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 918 op/s 2023-09-23T05:00:41.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:40 smithi118 ceph-mon[134297]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 918 op/s 2023-09-23T05:00:41.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:40 smithi069 ceph-mon[133931]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 918 op/s 2023-09-23T05:00:41.451 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39881 30.57 MB/sec warmup 76 sec latency 490.394 ms 2023-09-23T05:00:42.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:41 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:42.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:41 smithi005 ceph-mon[97638]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-09-23T05:00:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:41 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:41 smithi118 ceph-mon[134297]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-09-23T05:00:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:41 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:00:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:41 smithi069 ceph-mon[133931]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-09-23T05:00:42.453 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40086 30.33 MB/sec warmup 77 sec latency 508.566 ms 2023-09-23T05:00:43.453 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40337 30.06 MB/sec warmup 78 sec latency 749.437 ms 2023-09-23T05:00:44.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:43 smithi005 ceph-mon[97638]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 815 op/s 2023-09-23T05:00:44.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:43 smithi118 ceph-mon[134297]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 815 op/s 2023-09-23T05:00:44.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:43 smithi069 ceph-mon[133931]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 815 op/s 2023-09-23T05:00:44.454 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40735 29.91 MB/sec warmup 79 sec latency 347.562 ms 2023-09-23T05:00:45.455 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41331 29.95 MB/sec warmup 80 sec latency 113.780 ms 2023-09-23T05:00:46.456 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42289 30.25 MB/sec warmup 81 sec latency 47.107 ms 2023-09-23T05:00:47.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:46 smithi118 ceph-mon[134297]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 734 op/s 2023-09-23T05:00:47.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:46 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:00:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:46 smithi069 ceph-mon[133931]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 734 op/s 2023-09-23T05:00:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:46 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:00:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:46 smithi005 ceph-mon[97638]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 734 op/s 2023-09-23T05:00:47.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:46 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:00:47.461 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43017 30.47 MB/sec warmup 82 sec latency 104.964 ms 2023-09-23T05:00:48.461 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43447 30.32 MB/sec warmup 83 sec latency 442.466 ms 2023-09-23T05:00:49.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:48 smithi118 ceph-mon[134297]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:00:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:48 smithi069 ceph-mon[133931]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:00:49.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:48 smithi005 ceph-mon[97638]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:00:49.461 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43806 30.11 MB/sec warmup 84 sec latency 257.005 ms 2023-09-23T05:00:50.461 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44409 30.16 MB/sec warmup 85 sec latency 101.278 ms 2023-09-23T05:00:51.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:50 smithi005 ceph-mon[97638]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:51.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:50 smithi118 ceph-mon[134297]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:51.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:50 smithi069 ceph-mon[133931]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:51.461 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45325 30.37 MB/sec warmup 86 sec latency 87.843 ms 2023-09-23T05:00:52.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46270 30.65 MB/sec warmup 87 sec latency 101.681 ms 2023-09-23T05:00:53.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:52 smithi005 ceph-mon[97638]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-09-23T05:00:53.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:52 smithi118 ceph-mon[134297]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-09-23T05:00:53.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:52 smithi069 ceph-mon[133931]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-09-23T05:00:53.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46711 30.54 MB/sec warmup 88 sec latency 140.764 ms 2023-09-23T05:00:54.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:53 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:00:54.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:53 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:00:54.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:53 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:00:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:53 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:00:54.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:53 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:00:54.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:53 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:00:54.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:53 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:00:54.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:53 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:00:54.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:53 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:00:54.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47206 30.51 MB/sec warmup 89 sec latency 262.795 ms 2023-09-23T05:00:55.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:54 smithi005 ceph-mon[97638]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:55.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:54 smithi118 ceph-mon[134297]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:55.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:54 smithi069 ceph-mon[133931]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2023-09-23T05:00:55.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47821 30.61 MB/sec warmup 90 sec latency 78.493 ms 2023-09-23T05:00:56.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48714 30.74 MB/sec warmup 91 sec latency 55.728 ms 2023-09-23T05:00:57.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:56 smithi005 ceph-mon[97638]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 779 op/s 2023-09-23T05:00:57.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:56 smithi118 ceph-mon[134297]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 779 op/s 2023-09-23T05:00:57.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:56 smithi069 ceph-mon[133931]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 779 op/s 2023-09-23T05:00:57.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49529 30.96 MB/sec warmup 92 sec latency 48.685 ms 2023-09-23T05:00:58.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:57 smithi005 ceph-mon[97638]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 652 op/s 2023-09-23T05:00:58.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:57 smithi118 ceph-mon[134297]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 652 op/s 2023-09-23T05:00:58.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:57 smithi069 ceph-mon[133931]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.8 MiB/s wr, 652 op/s 2023-09-23T05:00:58.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50164 30.90 MB/sec warmup 93 sec latency 73.549 ms 2023-09-23T05:00:59.462 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50836 30.94 MB/sec warmup 94 sec latency 124.465 ms 2023-09-23T05:01:00.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:00:59 smithi118 ceph-mon[134297]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 959 op/s 2023-09-23T05:01:00.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:00:59 smithi069 ceph-mon[133931]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 959 op/s 2023-09-23T05:01:00.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:00:59 smithi005 ceph-mon[97638]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 959 op/s 2023-09-23T05:01:00.464 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51612 31.06 MB/sec warmup 95 sec latency 107.430 ms 2023-09-23T05:01:00.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:01:01.465 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51711 30.83 MB/sec warmup 96 sec latency 825.207 ms 2023-09-23T05:01:02.465 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52272 30.80 MB/sec warmup 97 sec latency 900.679 ms 2023-09-23T05:01:03.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:02 smithi005 ceph-mon[97638]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.5 MiB/s wr, 617 op/s 2023-09-23T05:01:03.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:02 smithi118 ceph-mon[134297]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.5 MiB/s wr, 617 op/s 2023-09-23T05:01:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:02 smithi069 ceph-mon[133931]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.5 MiB/s wr, 617 op/s 2023-09-23T05:01:03.467 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52876 30.89 MB/sec warmup 98 sec latency 94.151 ms 2023-09-23T05:01:04.467 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53680 30.97 MB/sec warmup 99 sec latency 60.366 ms 2023-09-23T05:01:05.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:04 smithi005 ceph-mon[97638]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-09-23T05:01:05.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:04 smithi118 ceph-mon[134297]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-09-23T05:01:05.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:04 smithi069 ceph-mon[133931]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-09-23T05:01:05.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54103 30.91 MB/sec warmup 100 sec latency 273.016 ms 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 -- 172.21.15.5:0/2137782586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 msgr2=0x7f0a4010d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 --2- 172.21.15.5:0/2137782586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 0x7f0a4010d7e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a3401a9a0 tx=0x7f0a3401acb0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 -- 172.21.15.5:0/2137782586 shutdown_connections 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 --2- 172.21.15.5:0/2137782586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a40107e20 0x7f0a401082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 --2- 172.21.15.5:0/2137782586 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a4010f5f0 0x7f0a4010d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 --2- 172.21.15.5:0/2137782586 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 0x7f0a4010d7e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 -- 172.21.15.5:0/2137782586 >> 172.21.15.5:0/2137782586 conn(0x7f0a400725f0 msgr2=0x7f0a4006d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 -- 172.21.15.5:0/2137782586 shutdown_connections 2023-09-23T05:01:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.249+0000 7f0a453ff700 1 -- 172.21.15.5:0/2137782586 wait complete. 2023-09-23T05:01:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.250+0000 7f0a453ff700 1 Processor -- start 2023-09-23T05:01:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 -- start start 2023-09-23T05:01:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 0x7f0a4019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 0x7f0a401a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a401a25e0 con 0x7f0a4010f5f0 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a401a2750 con 0x7f0a4010f9d0 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a453ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a401a2910 con 0x7f0a40107e20 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.253+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37242/0 (socket says 172.21.15.5:37242) 2023-09-23T05:01:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 -- 172.21.15.5:0/260721937 learned_addr learned my addr 172.21.15.5:0/260721937 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 0x7f0a4019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 -- 172.21.15.5:0/260721937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 msgr2=0x7f0a401a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 0x7f0a401a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 -- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 msgr2=0x7f0a4019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 0x7f0a4019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 -- 172.21.15.5:0/260721937 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3401a600 con 0x7f0a40107e20 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 0x7f0a4019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:01:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.254+0000 7f0a3effd700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000e9a0 tx=0x7f0a3000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.256+0000 7f0a27fff700 1 -- 172.21.15.5:0/260721937 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30009790 con 0x7f0a40107e20 2023-09-23T05:01:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.256+0000 7f0a27fff700 1 -- 172.21.15.5:0/260721937 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3000cb50 con 0x7f0a40107e20 2023-09-23T05:01:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.256+0000 7f0a27fff700 1 -- 172.21.15.5:0/260721937 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30018680 con 0x7f0a40107e20 2023-09-23T05:01:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.257+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a401a2bf0 con 0x7f0a40107e20 2023-09-23T05:01:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.257+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a401aa430 con 0x7f0a40107e20 2023-09-23T05:01:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.267+0000 7f0a27fff700 1 -- 172.21.15.5:0/260721937 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f0a3000ccc0 con 0x7f0a40107e20 2023-09-23T05:01:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.269+0000 7f0a27fff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 0x7f0a280a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.269+0000 7f0a27fff700 1 -- 172.21.15.5:0/260721937 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f0a30014070 con 0x7f0a40107e20 2023-09-23T05:01:06.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.270+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 0x7f0a280a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.271+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 0x7f0a280a3da0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a34000c00 tx=0x7f0a34005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.281+0000 7f0a58c49b80 1 Processor -- start 2023-09-23T05:01:06.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 -- start start 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8bb8460 con 0x5558c8c5abd0 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8ba9c20 con 0x5558c8c4f400 2023-09-23T05:01:06.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.282+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8ba7540 con 0x5558c8c4fd20 2023-09-23T05:01:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55566/0 (socket says 172.21.15.5:55566) 2023-09-23T05:01:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 -- 172.21.15.5:0/1391751630 learned_addr learned my addr 172.21.15.5:0/1391751630 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 -- 172.21.15.5:0/1391751630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 msgr2=0x5558c8c5a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 -- 172.21.15.5:0/1391751630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 msgr2=0x5558c8c5cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.283+0000 7f0a3effd700 1 -- 172.21.15.5:0/1391751630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c8c058d0 con 0x5558c8c4f400 2023-09-23T05:01:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.284+0000 7f0a3effd700 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000f010 tx=0x7f0a30099d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e753815a9958732 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.284+0000 7f0a257fa700 1 -- 172.21.15.5:0/1391751630 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3009e500 con 0x5558c8c4f400 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.284+0000 7f0a257fa700 1 -- 172.21.15.5:0/1391751630 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a300a2740 con 0x5558c8c4f400 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.284+0000 7f0a257fa700 1 -- 172.21.15.5:0/1391751630 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30099b00 con 0x5558c8c4f400 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 -- 172.21.15.5:0/1391751630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 msgr2=0x5558c8c4f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000f010 tx=0x7f0a30099d90 comp rx=0 tx=0).stop 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 -- 172.21.15.5:0/1391751630 shutdown_connections 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c4f7e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/1391751630 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.285+0000 7f0a58c49b80 1 -- 172.21.15.5:0/1391751630 >> 172.21.15.5:0/1391751630 conn(0x5558c8bc4260 msgr2=0x5558c8c4b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.286+0000 7f0a58c49b80 1 -- 172.21.15.5:0/1391751630 shutdown_connections 2023-09-23T05:01:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.286+0000 7f0a58c49b80 1 -- 172.21.15.5:0/1391751630 wait complete. 2023-09-23T05:01:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.287+0000 7f0a58c49b80 1 -- start start 2023-09-23T05:01:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.287+0000 7f0a58c49b80 1 Processor -- start 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4f400 0x5558c8c57960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c5abd0 0x5558c8c583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8ba2e30 con 0x5558c8c4f400 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8b3f710 con 0x5558c8c4fd20 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8b3f3b0 con 0x5558c8c5abd0 2023-09-23T05:01:06.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.289+0000 7f0a58c49b80 10 client.0 _ll_register_callbacks cb 0x5558c899c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:01:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.290+0000 7f0a3e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.290+0000 7f0a3e7fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55570/0 (socket says 172.21.15.5:55570) 2023-09-23T05:01:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.290+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/2898271491 learned_addr learned my addr 172.21.15.5:0/2898271491 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c5abd0 0x5558c8c583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3effd700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4f400 0x5558c8c57960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/2898271491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c5abd0 msgr2=0x5558c8c583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c5abd0 0x5558c8c583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4f400 msgr2=0x5558c8c57960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4f400 0x5558c8c57960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 -- 172.21.15.5:0/2898271491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a401aa430 con 0x5558c8c4fd20 2023-09-23T05:01:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.291+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0a3401a9a0 tx=0x7f0a3401a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ccaf08a6bc22fc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.292+0000 7f0a22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:01:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.292+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34035da0 con 0x5558c8c4fd20 2023-09-23T05:01:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.292+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f0a34003960 con 0x5558c8c4fd20 2023-09-23T05:01:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.292+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558c8c43250 con 0x5558c8c4fd20 2023-09-23T05:01:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.293+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a34003b60 con 0x5558c8c4fd20 2023-09-23T05:01:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.293+0000 7f0a58c49b80 10 client.27671 fetch_fsmap learned FSMap version 68 2023-09-23T05:01:06.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.293+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5558c89cdde0 con 0x5558c8c4fd20 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.293+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a34049030 con 0x5558c8c4fd20 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 68) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f0a34040070 con 0x5558c8c4fd20 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 10 client.27671 fetch_fsmap finished waiting for FSMap version 68 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 10 client.27671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 20 client.27671 populate_metadata read hostname 'smithi005' 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8c6a620 0x5558c8c6ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 4 client.27671 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:01:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558c8b9be10 con 0x5558c8c6a620 2023-09-23T05:01:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.294+0000 7f0a3effd700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8c6a620 0x5558c8c6ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.296+0000 7f0a22ff5700 10 client.27671 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.296+0000 7f0a3effd700 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8c6a620 0x5558c8c6ca40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c50c57ae47b13d server_cookie=ee2bf05bd4b65509 in_seq=0 out_seq=0 2023-09-23T05:01:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.299+0000 7f0a22ff5700 1 -- 172.21.15.5:0/2898271491 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5558c8b9be10 con 0x5558c8c6a620 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.299+0000 7f0a22ff5700 10 client.27671 handle_command_reply: tid=0 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.299+0000 7f0a58c49b80 1 client.27671 shutdown 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 msgr2=0x5558c8c57ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0a3401a9a0 tx=0x7f0a3401a320 comp rx=0 tx=0).stop 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 shutdown_connections 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8c6a620 0x5558c8c6ca40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c5abd0 0x5558c8c583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4f400 0x5558c8c57960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2898271491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4fd20 0x5558c8c57ea0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 >> 172.21.15.5:0/2898271491 conn(0x5558c8bc4260 msgr2=0x5558c8c4b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a22ff5700 0 client.27671 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 shutdown_connections 2023-09-23T05:01:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.300+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2898271491 wait complete. 2023-09-23T05:01:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.301+0000 7f0a58c49b80 20 client.27671 trim_cache size 0 max 16384 2023-09-23T05:01:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.329+0000 7f0a58c49b80 1 Processor -- start 2023-09-23T05:01:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 -- start start 2023-09-23T05:01:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4f400 0x5558c8c616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4fd20 0x5558c8c58210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8a23920 con 0x5558c8c4fd20 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8c4a580 con 0x5558c8c5abd0 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8c537b0 con 0x5558c8c4f400 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4f400 0x5558c8c616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55580/0 (socket says 172.21.15.5:55580) 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.330+0000 7f0a3f7fe700 1 -- 172.21.15.5:0/2001920684 learned_addr learned my addr 172.21.15.5:0/2001920684 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 -- 172.21.15.5:0/2001920684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4f400 msgr2=0x5558c8c616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4f400 0x5558c8c616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 -- 172.21.15.5:0/2001920684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4fd20 msgr2=0x5558c8c58210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4fd20 0x5558c8c58210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 -- 172.21.15.5:0/2001920684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558c89b6c60 con 0x5558c8c5abd0 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.331+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0a38007810 tx=0x7f0a38007840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32debf24b061095 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.332+0000 7f0a25ffb700 1 -- 172.21.15.5:0/2001920684 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800e040 con 0x5558c8c5abd0 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.332+0000 7f0a25ffb700 1 -- 172.21.15.5:0/2001920684 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3800f040 con 0x5558c8c5abd0 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.332+0000 7f0a25ffb700 1 -- 172.21.15.5:0/2001920684 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a380086b0 con 0x5558c8c5abd0 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2001920684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 msgr2=0x5558c8c54bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0a38007810 tx=0x7f0a38007840 comp rx=0 tx=0).stop 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2001920684 shutdown_connections 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c5abd0 0x5558c8c54bd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4f400 0x5558c8c616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/2001920684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c4fd20 0x5558c8c58210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2001920684 >> 172.21.15.5:0/2001920684 conn(0x5558c8bc4260 msgr2=0x5558c8c52c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2001920684 shutdown_connections 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- 172.21.15.5:0/2001920684 wait complete. 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.333+0000 7f0a58c49b80 1 -- start start 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 Processor -- start 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8c603d0 con 0x5558c8c5abd0 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8c60540 con 0x5558c8c4f400 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.334+0000 7f0a58c49b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c8cbe460 con 0x5558c8c4fd20 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a58c49b80 10 client.0 _ll_register_callbacks cb 0x7f0a3401cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a3effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55590/0 (socket says 172.21.15.5:55590) 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a3effd700 1 -- 172.21.15.5:0/233305786 learned_addr learned my addr 172.21.15.5:0/233305786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a3f7fe700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.335+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.336+0000 7f0a3effd700 1 -- 172.21.15.5:0/233305786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 msgr2=0x5558c8c5f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.336+0000 7f0a3effd700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.336+0000 7f0a3effd700 1 -- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 msgr2=0x5558c8c5fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.336+0000 7f0a3effd700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.336+0000 7f0a3effd700 1 -- 172.21.15.5:0/233305786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a380074c0 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a3effd700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a300968b0 tx=0x7f0a3009a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a9021c96f192b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300b8040 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f0a300998a0 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a300a2740 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a30099a60 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a58c49b80 10 client.27677 fetch_fsmap learned FSMap version 68 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.337+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5558c89f4250 con 0x5558c8c4f400 2023-09-23T05:01:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.338+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a300a2740 con 0x5558c8c4f400 2023-09-23T05:01:06.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 68) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f0a3009a610 con 0x5558c8c4f400 2023-09-23T05:01:06.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 10 client.27677 fetch_fsmap finished waiting for FSMap version 68 2023-09-23T05:01:06.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 10 client.27677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:01:06.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 20 client.27677 populate_metadata read hostname 'smithi005' 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8cbf950 0x5558c8cc1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 4 client.27677 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558c8b9bb30 con 0x5558c8cbf950 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.340+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8cbf950 0x5558c8cc1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.341+0000 7f0a3e7fc700 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8cbf950 0x5558c8cc1d70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4baea9124136b9c server_cookie=42697d5e2da6a1e4 in_seq=0 out_seq=0 2023-09-23T05:01:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.341+0000 7f0a22ff5700 10 client.27677 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.343+0000 7f0a22ff5700 1 -- 172.21.15.5:0/233305786 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5558c8b9bb30 con 0x5558c8cbf950 2023-09-23T05:01:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.343+0000 7f0a22ff5700 10 client.27677 handle_command_reply: tid=0 2023-09-23T05:01:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.343+0000 7f0a58c49b80 1 client.27677 shutdown 2023-09-23T05:01:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.344+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 msgr2=0x5558c8c61540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.344+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a300968b0 tx=0x7f0a3009a2b0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 shutdown_connections 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5558c8cbf950 0x5558c8cc1d70 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558c8c5abd0 0x5558c8c5fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5558c8c4f400 0x5558c8c61540 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 --2- 172.21.15.5:0/233305786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5558c8c4fd20 0x5558c8c5f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 >> 172.21.15.5:0/233305786 conn(0x5558c8bc4260 msgr2=0x5558c8c51320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a22ff5700 0 client.27677 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 shutdown_connections 2023-09-23T05:01:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.345+0000 7f0a58c49b80 1 -- 172.21.15.5:0/233305786 wait complete. 2023-09-23T05:01:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.346+0000 7f0a58c49b80 20 client.27677 trim_cache size 0 max 16384 2023-09-23T05:01:06.347 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:01:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:01:06.348 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:01:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 msgr2=0x7f0a280a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 0x7f0a280a3da0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a34000c00 tx=0x7f0a34005fd0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 msgr2=0x7f0a4019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000e9a0 tx=0x7f0a3000ed60 comp rx=0 tx=0).stop 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 shutdown_connections 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f0a280a18e0 0x7f0a280a3da0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a34000c00 tx=0x7f0a34005fd0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a4010f9d0 0x7f0a401a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f0a40107e20 0x7f0a4019d600 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000e9a0 tx=0x7f0a3000ed60 comp rx=0 tx=0).stop 2023-09-23T05:01:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 --2- 172.21.15.5:0/260721937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4010f5f0 0x7f0a4019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 >> 172.21.15.5:0/260721937 conn(0x7f0a400725f0 msgr2=0x7f0a40109ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 shutdown_connections 2023-09-23T05:01:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:06.347+0000 7f0a453ff700 1 -- 172.21.15.5:0/260721937 wait complete. 2023-09-23T05:01:06.375 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:01:06.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54611 30.83 MB/sec warmup 101 sec latency 94.320 ms 2023-09-23T05:01:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:06 smithi005 ceph-mon[97638]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-23T05:01:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:06 smithi005 ceph-mon[97638]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T05:01:07.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:06 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:07.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:06 smithi118 ceph-mon[134297]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-23T05:01:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:06 smithi118 ceph-mon[134297]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T05:01:07.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:06 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:06 smithi069 ceph-mon[133931]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-23T05:01:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:06 smithi069 ceph-mon[133931]: osdmap e109: 12 total, 12 up, 12 in 2023-09-23T05:01:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:06 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:07.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55398 30.95 MB/sec warmup 102 sec latency 81.018 ms 2023-09-23T05:01:08.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56194 31.10 MB/sec warmup 103 sec latency 55.675 ms 2023-09-23T05:01:09.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:08 smithi005 ceph-mon[97638]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-09-23T05:01:09.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:08 smithi118 ceph-mon[134297]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-09-23T05:01:09.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:08 smithi069 ceph-mon[133931]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-09-23T05:01:09.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56857 31.13 MB/sec warmup 104 sec latency 122.756 ms 2023-09-23T05:01:10.468 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57628 31.22 MB/sec warmup 105 sec latency 83.750 ms 2023-09-23T05:01:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:10 smithi005 ceph-mon[97638]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 734 op/s 2023-09-23T05:01:11.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:10 smithi005 ceph-mon[97638]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T05:01:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:10 smithi118 ceph-mon[134297]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 734 op/s 2023-09-23T05:01:11.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:10 smithi118 ceph-mon[134297]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T05:01:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:10 smithi069 ceph-mon[133931]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 734 op/s 2023-09-23T05:01:11.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:10 smithi069 ceph-mon[133931]: osdmap e110: 12 total, 12 up, 12 in 2023-09-23T05:01:11.469 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58335 31.25 MB/sec warmup 106 sec latency 72.075 ms 2023-09-23T05:01:12.469 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59097 31.38 MB/sec warmup 107 sec latency 75.698 ms 2023-09-23T05:01:13.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:12 smithi005 ceph-mon[97638]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 623 op/s 2023-09-23T05:01:13.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:12 smithi118 ceph-mon[134297]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 623 op/s 2023-09-23T05:01:13.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:12 smithi069 ceph-mon[133931]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 623 op/s 2023-09-23T05:01:13.469 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60031 31.54 MB/sec warmup 108 sec latency 63.098 ms 2023-09-23T05:01:14.469 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61022 31.76 MB/sec warmup 109 sec latency 37.767 ms 2023-09-23T05:01:15.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:14 smithi005 ceph-mon[97638]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-09-23T05:01:15.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:14 smithi118 ceph-mon[134297]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-09-23T05:01:15.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:14 smithi069 ceph-mon[133931]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-09-23T05:01:15.470 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61612 31.81 MB/sec warmup 110 sec latency 94.558 ms 2023-09-23T05:01:16.470 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62324 31.80 MB/sec warmup 111 sec latency 60.001 ms 2023-09-23T05:01:17.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:16 smithi005 ceph-mon[97638]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-09-23T05:01:17.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:16 smithi118 ceph-mon[134297]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-09-23T05:01:17.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:16 smithi069 ceph-mon[133931]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 744 op/s 2023-09-23T05:01:17.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63012 31.83 MB/sec warmup 112 sec latency 98.243 ms 2023-09-23T05:01:18.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:17 smithi005 ceph-mon[97638]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 673 op/s 2023-09-23T05:01:18.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:17 smithi118 ceph-mon[134297]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 673 op/s 2023-09-23T05:01:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:17 smithi069 ceph-mon[133931]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.7 MiB/s wr, 673 op/s 2023-09-23T05:01:18.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63976 32.01 MB/sec warmup 113 sec latency 67.992 ms 2023-09-23T05:01:19.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64885 32.19 MB/sec warmup 114 sec latency 58.567 ms 2023-09-23T05:01:20.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:19 smithi005 ceph-mon[97638]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 631 op/s 2023-09-23T05:01:20.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:19 smithi118 ceph-mon[134297]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 631 op/s 2023-09-23T05:01:20.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:19 smithi069 ceph-mon[133931]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 631 op/s 2023-09-23T05:01:20.473 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65659 32.24 MB/sec warmup 115 sec latency 69.755 ms 2023-09-23T05:01:21.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66420 32.40 MB/sec warmup 116 sec latency 79.718 ms 2023-09-23T05:01:22.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:21 smithi005 ceph-mon[97638]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 625 op/s 2023-09-23T05:01:22.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:21 smithi118 ceph-mon[134297]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 625 op/s 2023-09-23T05:01:22.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:21 smithi069 ceph-mon[133931]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 625 op/s 2023-09-23T05:01:22.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67246 32.45 MB/sec warmup 117 sec latency 60.438 ms 2023-09-23T05:01:23.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:22 smithi069 ceph-mon[133931]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 776 op/s 2023-09-23T05:01:23.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:22 smithi005 ceph-mon[97638]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 776 op/s 2023-09-23T05:01:23.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:22 smithi118 ceph-mon[134297]: pgmap v474: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 776 op/s 2023-09-23T05:01:23.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67910 32.50 MB/sec warmup 118 sec latency 67.781 ms 2023-09-23T05:01:24.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68842 32.59 MB/sec warmup 119 sec latency 59.798 ms 2023-09-23T05:01:25.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:24 smithi069 ceph-mon[133931]: pgmap v475: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-09-23T05:01:25.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:24 smithi005 ceph-mon[97638]: pgmap v475: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-09-23T05:01:25.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:24 smithi118 ceph-mon[134297]: pgmap v475: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 589 op/s 2023-09-23T05:01:26.471 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70627 55.61 MB/sec execute 1 sec latency 61.699 ms 2023-09-23T05:01:27.244 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:26 smithi069 ceph-mon[133931]: pgmap v476: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 531 op/s 2023-09-23T05:01:27.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:26 smithi005 ceph-mon[97638]: pgmap v476: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 531 op/s 2023-09-23T05:01:27.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:26 smithi118 ceph-mon[134297]: pgmap v476: 129 pgs: 129 active+clean; 1.2 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.2 MiB/s wr, 531 op/s 2023-09-23T05:01:27.472 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70938 36.51 MB/sec execute 2 sec latency 433.939 ms 2023-09-23T05:01:28.473 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70962 24.81 MB/sec execute 3 sec latency 1177.359 ms 2023-09-23T05:01:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:28 smithi069 ceph-mon[133931]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 894 op/s 2023-09-23T05:01:29.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:28 smithi005 ceph-mon[97638]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 894 op/s 2023-09-23T05:01:29.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:28 smithi118 ceph-mon[134297]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 894 op/s 2023-09-23T05:01:29.473 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71099 19.36 MB/sec execute 4 sec latency 1302.846 ms 2023-09-23T05:01:30.474 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71383 17.44 MB/sec execute 5 sec latency 298.649 ms 2023-09-23T05:01:31.476 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71686 17.01 MB/sec execute 6 sec latency 319.227 ms 2023-09-23T05:01:32.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:31 smithi005 ceph-mon[97638]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 641 op/s 2023-09-23T05:01:32.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:31 smithi118 ceph-mon[134297]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 641 op/s 2023-09-23T05:01:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:31 smithi069 ceph-mon[133931]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 641 op/s 2023-09-23T05:01:32.477 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72038 16.96 MB/sec execute 7 sec latency 224.105 ms 2023-09-23T05:01:33.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:32 smithi069 ceph-mon[133931]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 930 op/s 2023-09-23T05:01:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:32 smithi005 ceph-mon[97638]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 930 op/s 2023-09-23T05:01:33.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:32 smithi118 ceph-mon[134297]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 930 op/s 2023-09-23T05:01:33.477 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72753 20.24 MB/sec execute 8 sec latency 110.029 ms 2023-09-23T05:01:34.480 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73248 21.01 MB/sec execute 9 sec latency 150.559 ms 2023-09-23T05:01:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:34 smithi069 ceph-mon[133931]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-09-23T05:01:35.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:34 smithi005 ceph-mon[97638]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-09-23T05:01:35.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:34 smithi118 ceph-mon[134297]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 19 MiB/s wr, 799 op/s 2023-09-23T05:01:35.483 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73711 21.54 MB/sec execute 10 sec latency 358.336 ms 2023-09-23T05:01:36.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:01:36.795 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74384 22.99 MB/sec execute 11 sec latency 358.458 ms 2023-09-23T05:01:37.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:36 smithi069 ceph-mon[133931]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 771 op/s 2023-09-23T05:01:37.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:36 smithi005 ceph-mon[97638]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 771 op/s 2023-09-23T05:01:37.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:36 smithi118 ceph-mon[134297]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 771 op/s 2023-09-23T05:01:37.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75025 23.47 MB/sec execute 12 sec latency 78.897 ms 2023-09-23T05:01:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.323+0000 7fe0229ff700 1 -- 172.21.15.5:0/3727346567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 msgr2=0x7fe01c10dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.323+0000 7fe0229ff700 1 --2- 172.21.15.5:0/3727346567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 0x7fe01c10dd90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b0a0 tx=0x7fe01400b3b0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 -- 172.21.15.5:0/3727346567 shutdown_connections 2023-09-23T05:01:38.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 --2- 172.21.15.5:0/3727346567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 0x7fe01c10dd90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 --2- 172.21.15.5:0/3727346567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c1010f0 0x7fe01c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 --2- 172.21.15.5:0/3727346567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c101aa0 0x7fe01c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 -- 172.21.15.5:0/3727346567 >> 172.21.15.5:0/3727346567 conn(0x7fe01c0fc9d0 msgr2=0x7fe01c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.324+0000 7fe0229ff700 1 -- 172.21.15.5:0/3727346567 shutdown_connections 2023-09-23T05:01:38.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.325+0000 7fe0229ff700 1 -- 172.21.15.5:0/3727346567 wait complete. 2023-09-23T05:01:38.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.326+0000 7fe0229ff700 1 Processor -- start 2023-09-23T05:01:38.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.326+0000 7fe0229ff700 1 -- start start 2023-09-23T05:01:38.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c1010f0 0x7fe01c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 0x7fe01c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19e4a0 con 0x7fe01c1010f0 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19e610 con 0x7fe01c106210 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.327+0000 7fe0229ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19e7d0 con 0x7fe01c19da20 2023-09-23T05:01:38.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48972/0 (socket says 172.21.15.5:48972) 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 -- 172.21.15.5:0/2532689304 learned_addr learned my addr 172.21.15.5:0/2532689304 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 -- 172.21.15.5:0/2532689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 msgr2=0x7fe01c1995c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 0x7fe01c1995c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 -- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c1010f0 msgr2=0x7fe01c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c1010f0 0x7fe01c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.328+0000 7fe020f9c700 1 -- 172.21.15.5:0/2532689304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009d00 con 0x7fe01c19da20 2023-09-23T05:01:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.329+0000 7fe020f9c700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b830 tx=0x7fe014007bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.329+0000 7fe0197fa700 1 -- 172.21.15.5:0/2532689304 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400e040 con 0x7fe01c19da20 2023-09-23T05:01:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.329+0000 7fe0197fa700 1 -- 172.21.15.5:0/2532689304 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe014004c80 con 0x7fe01c19da20 2023-09-23T05:01:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.329+0000 7fe0197fa700 1 -- 172.21.15.5:0/2532689304 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014005550 con 0x7fe01c19da20 2023-09-23T05:01:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.330+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c1a5cd0 con 0x7fe01c19da20 2023-09-23T05:01:38.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.330+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c1a6100 con 0x7fe01c19da20 2023-09-23T05:01:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.337+0000 7fe0197fa700 1 -- 172.21.15.5:0/2532689304 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe014012bb0 con 0x7fe01c19da20 2023-09-23T05:01:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.338+0000 7fe0197fa700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 0x7fe0080a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.339+0000 7fe0197fa700 1 -- 172.21.15.5:0/2532689304 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe0140ceb60 con 0x7fe01c19da20 2023-09-23T05:01:38.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.341+0000 7fe01bfff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 0x7fe0080a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.359+0000 7fe01bfff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 0x7fe0080a3d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097b0 tx=0x7fe00c006d20 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 Processor -- start 2023-09-23T05:01:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 -- start start 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfd07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bf23460 con 0x55d32bfce3e0 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.360+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bf14c20 con 0x55d32bfc3530 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bf12540 con 0x55d32bfc2c10 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48988/0 (socket says 172.21.15.5:48988) 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 -- 172.21.15.5:0/4278098692 learned_addr learned my addr 172.21.15.5:0/4278098692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 -- 172.21.15.5:0/4278098692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 msgr2=0x55d32bfcdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 -- 172.21.15.5:0/4278098692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 msgr2=0x55d32bfd07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfd07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.361+0000 7fe01bfff700 1 -- 172.21.15.5:0/4278098692 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32bf708d0 con 0x55d32bfc2c10 2023-09-23T05:01:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.362+0000 7fe01bfff700 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0158f0 tx=0x7fe00c006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964b67f808025b65 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.363+0000 7fe0127fc700 1 -- 172.21.15.5:0/4278098692 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01a070 con 0x55d32bfc2c10 2023-09-23T05:01:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.363+0000 7fe0127fc700 1 -- 172.21.15.5:0/4278098692 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe00c0128f0 con 0x55d32bfc2c10 2023-09-23T05:01:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.363+0000 7fe0127fc700 1 -- 172.21.15.5:0/4278098692 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c013840 con 0x55d32bfc2c10 2023-09-23T05:01:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 -- 172.21.15.5:0/4278098692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 msgr2=0x55d32bfc2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0158f0 tx=0x7fe00c006f90 comp rx=0 tx=0).stop 2023-09-23T05:01:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 -- 172.21.15.5:0/4278098692 shutdown_connections 2023-09-23T05:01:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfd07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc2ff0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/4278098692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.364+0000 7fe0363a4b80 1 -- 172.21.15.5:0/4278098692 >> 172.21.15.5:0/4278098692 conn(0x55d32bf2f260 msgr2=0x55d32bfb6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.365+0000 7fe0363a4b80 1 -- 172.21.15.5:0/4278098692 shutdown_connections 2023-09-23T05:01:38.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.367+0000 7fe0363a4b80 1 -- 172.21.15.5:0/4278098692 wait complete. 2023-09-23T05:01:38.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.367+0000 7fe0363a4b80 1 -- start start 2023-09-23T05:01:38.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.368+0000 7fe0363a4b80 1 Processor -- start 2023-09-23T05:01:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.368+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc2c10 0x55d32bfcb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.368+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc3530 0x55d32bfcb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.368+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.368+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bf0de30 con 0x55d32bfc2c10 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32beaa710 con 0x55d32bfce3e0 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32beaa3b0 con 0x55d32bfc3530 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe0363a4b80 10 client.0 _ll_register_callbacks cb 0x55d32bd07800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:50114/0 (socket says 172.21.15.5:50114) 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 -- 172.21.15.5:0/1038407889 learned_addr learned my addr 172.21.15.5:0/1038407889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 -- 172.21.15.5:0/1038407889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc3530 msgr2=0x55d32bfcb7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T05:01:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc3530 0x55d32bfcb7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 -- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc2c10 msgr2=0x55d32bfcb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc2c10 0x55d32bfcb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.369+0000 7fe020f9c700 1 -- 172.21.15.5:0/1038407889 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c0155d0 con 0x55d32bfce3e0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fe020f9c700 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe014098140 tx=0x7fe01409b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a154d31b0a8e897b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fdffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01409bb30 con 0x55d32bfce3e0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe01409b6d0 con 0x55d32bfce3e0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d32bfae250 con 0x55d32bfce3e0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0140bb070 con 0x55d32bfce3e0 2023-09-23T05:01:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fe0363a4b80 10 client.27686 fetch_fsmap learned FSMap version 68 2023-09-23T05:01:38.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.370+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55d32bd38de0 con 0x55d32bfce3e0 2023-09-23T05:01:38.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d32bfae250 con 0x55d32bfce3e0 2023-09-23T05:01:38.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 68) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe0140a3540 con 0x55d32bfce3e0 2023-09-23T05:01:38.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 10 client.27686 fetch_fsmap finished waiting for FSMap version 68 2023-09-23T05:01:38.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 10 client.27686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:01:38.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 20 client.27686 populate_metadata read hostname 'smithi005' 2023-09-23T05:01:38.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32bfeeea0 0x55d32bff12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 4 client.27686 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.371+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d32bf06e10 con 0x55d32bfeeea0 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.372+0000 7fe01bfff700 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32bfeeea0 0x55d32bff12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.372+0000 7fdffb7fe700 10 client.27686 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.372+0000 7fe01bfff700 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32bfeeea0 0x55d32bff12c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2620d6eed37ad6d9 server_cookie=3d7630e63f30d879 in_seq=0 out_seq=0 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.373+0000 7fdffb7fe700 1 -- 172.21.15.5:0/1038407889 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d32bf06e10 con 0x55d32bfeeea0 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.373+0000 7fdffb7fe700 10 client.27686 handle_command_reply: tid=0 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.373+0000 7fe0363a4b80 1 client.27686 shutdown 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 msgr2=0x55d32bfcbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe014098140 tx=0x7fe01409b340 comp rx=0 tx=0).stop 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 shutdown_connections 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32bfeeea0 0x55d32bff12c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfcbce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc2c10 0x55d32bfcb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/1038407889 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc3530 0x55d32bfcb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fdffb7fe700 0 client.27686 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 >> 172.21.15.5:0/1038407889 conn(0x55d32bf2f260 msgr2=0x55d32bfb6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 shutdown_connections 2023-09-23T05:01:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.374+0000 7fe0363a4b80 1 -- 172.21.15.5:0/1038407889 wait complete. 2023-09-23T05:01:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.375+0000 7fe0363a4b80 20 client.27686 trim_cache size 0 max 16384 2023-09-23T05:01:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 Processor -- start 2023-09-23T05:01:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 -- start start 2023-09-23T05:01:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bd8e920 con 0x55d32bfce3e0 2023-09-23T05:01:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bfc4400 con 0x55d32bfc3530 2023-09-23T05:01:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.404+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bfc97c0 con 0x55d32bfc2c10 2023-09-23T05:01:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe01bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48998/0 (socket says 172.21.15.5:48998) 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39824/0 (socket says 172.21.15.5:39824) 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe01bfff700 1 -- 172.21.15.5:0/8756092 learned_addr learned my addr 172.21.15.5:0/8756092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.405+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 -- 172.21.15.5:0/8756092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 msgr2=0x55d32bfc5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 -- 172.21.15.5:0/8756092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 msgr2=0x55d32bfcaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 -- 172.21.15.5:0/8756092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32bd21c60 con 0x55d32bfce3e0 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:01:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.406+0000 7fe020f9c700 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0140a34a0 tx=0x7fe014007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a88211b547e4c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.407+0000 7fe012ffd700 1 -- 172.21.15.5:0/8756092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140b9bb0 con 0x55d32bfce3e0 2023-09-23T05:01:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.407+0000 7fe012ffd700 1 -- 172.21.15.5:0/8756092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe01409abc0 con 0x55d32bfce3e0 2023-09-23T05:01:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.407+0000 7fe012ffd700 1 -- 172.21.15.5:0/8756092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01409b930 con 0x55d32bfce3e0 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 -- 172.21.15.5:0/8756092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 msgr2=0x55d32bfcb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0140a34a0 tx=0x7fe014007c60 comp rx=0 tx=0).stop 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 -- 172.21.15.5:0/8756092 shutdown_connections 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfce3e0 0x55d32bfcb930 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfc5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/8756092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfc3530 0x55d32bfcaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.408+0000 7fe0363a4b80 1 -- 172.21.15.5:0/8756092 >> 172.21.15.5:0/8756092 conn(0x55d32bf2f260 msgr2=0x55d32bfcb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.410+0000 7fe0363a4b80 1 -- 172.21.15.5:0/8756092 shutdown_connections 2023-09-23T05:01:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.411+0000 7fe0363a4b80 1 -- 172.21.15.5:0/8756092 wait complete. 2023-09-23T05:01:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.412+0000 7fe0363a4b80 1 -- start start 2023-09-23T05:01:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.412+0000 7fe0363a4b80 1 Processor -- start 2023-09-23T05:01:38.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfe5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfe41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bfe4730 con 0x55d32bfc3530 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bfe48a0 con 0x55d32bfce3e0 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32bfe4a60 con 0x55d32bfc2c10 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe0363a4b80 10 client.0 _ll_register_callbacks cb 0x55d32bd07800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe01b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe020f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfe41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe01b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39828/0 (socket says 172.21.15.5:39828) 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe01b7fe700 1 -- 172.21.15.5:0/297114775 learned_addr learned my addr 172.21.15.5:0/297114775 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:01:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.413+0000 7fe01bfff700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfe5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 -- 172.21.15.5:0/297114775 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 msgr2=0x55d32bfe5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfe5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 -- 172.21.15.5:0/297114775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 msgr2=0x55d32bfe41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfe41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 -- 172.21.15.5:0/297114775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00400a7d0 con 0x55d32bfc3530 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fe01b7fe700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c19a690 tx=0x7fe004010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c1b74eb4cdc96d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.414+0000 7fdffb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:01:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.415+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004015070 con 0x55d32bfc3530 2023-09-23T05:01:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.415+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe00400fb70 con 0x55d32bfc3530 2023-09-23T05:01:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.415+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe00c0128f0 con 0x55d32bfc3530 2023-09-23T05:01:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.416+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe00400fd30 con 0x55d32bfc3530 2023-09-23T05:01:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.416+0000 7fe0363a4b80 10 client.17661 fetch_fsmap learned FSMap version 68 2023-09-23T05:01:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.416+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55d32bd38de0 con 0x55d32bfc3530 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe00c0128f0 con 0x55d32bfc3530 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fe004021070 con 0x55d32bfc3530 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 10 client.17661 fetch_fsmap finished waiting for FSMap version 68 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 20 client.17661 populate_metadata read hostname 'smithi005' 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32c04cbd0 0x55d32c04f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:01:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 4 client.17661 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:01:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.417+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d32bf06b30 con 0x55d32c04cbd0 2023-09-23T05:01:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.419+0000 7fe01bfff700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32c04cbd0 0x55d32c04f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:01:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.420+0000 7fe01bfff700 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32c04cbd0 0x55d32c04f090 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64cdc734706d2252 server_cookie=35f854e0ad3866b3 in_seq=0 out_seq=0 2023-09-23T05:01:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.420+0000 7fdffb7fe700 10 client.17661 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.421+0000 7fdffb7fe700 1 -- 172.21.15.5:0/297114775 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d32bf06b30 con 0x55d32c04cbd0 2023-09-23T05:01:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.421+0000 7fdffb7fe700 10 client.17661 handle_command_reply: tid=0 2023-09-23T05:01:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.421+0000 7fe0363a4b80 1 client.17661 shutdown 2023-09-23T05:01:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.423+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 msgr2=0x55d32bfc00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.423+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c19a690 tx=0x7fe004010ce0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 shutdown_connections 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55d32c04cbd0 0x55d32c04f090 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d32bfce3e0 0x55d32bfe41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55d32bfc2c10 0x55d32bfe5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 --2- 172.21.15.5:0/297114775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d32bfc3530 0x55d32bfc00e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 >> 172.21.15.5:0/297114775 conn(0x55d32bf2f260 msgr2=0x55d32bfcd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.425+0000 7fdffb7fe700 0 client.17661 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:01:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.426+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 shutdown_connections 2023-09-23T05:01:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.426+0000 7fe0363a4b80 1 -- 172.21.15.5:0/297114775 wait complete. 2023-09-23T05:01:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.426+0000 7fe0363a4b80 20 client.17661 trim_cache size 0 max 16384 2023-09-23T05:01:38.427 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:01:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:01:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:01:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:01:38.428 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.428+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 msgr2=0x7fe0080a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.428+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 0x7fe0080a3d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097b0 tx=0x7fe00c006d20 comp rx=0 tx=0).stop 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.428+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 msgr2=0x7fe01c19dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.428+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b830 tx=0x7fe014007bf0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 shutdown_connections 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0080a1860 0x7fe0080a3d20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe01c19da20 0x7fe01c19dea0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c1010f0 0x7fe01c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 --2- 172.21.15.5:0/2532689304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe01c106210 0x7fe01c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:01:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 >> 172.21.15.5:0/2532689304 conn(0x7fe01c0fc9d0 msgr2=0x7fe01c10d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:01:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 shutdown_connections 2023-09-23T05:01:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:01:38.429+0000 7fe0229ff700 1 -- 172.21.15.5:0/2532689304 wait complete. 2023-09-23T05:01:38.460 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:01:38.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75717 24.63 MB/sec execute 13 sec latency 70.168 ms 2023-09-23T05:01:39.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:38 smithi069 ceph-mon[133931]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.05k op/s 2023-09-23T05:01:39.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:38 smithi005 ceph-mon[97638]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.05k op/s 2023-09-23T05:01:39.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:38 smithi118 ceph-mon[134297]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 23 MiB/s wr, 1.05k op/s 2023-09-23T05:01:39.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76483 25.85 MB/sec execute 14 sec latency 54.173 ms 2023-09-23T05:01:40.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77137 26.75 MB/sec execute 15 sec latency 58.566 ms 2023-09-23T05:01:41.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77941 27.33 MB/sec execute 16 sec latency 159.521 ms 2023-09-23T05:01:42.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:41 smithi005 ceph-mon[97638]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 683 op/s 2023-09-23T05:01:42.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:41 smithi118 ceph-mon[134297]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 683 op/s 2023-09-23T05:01:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:41 smithi069 ceph-mon[133931]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 683 op/s 2023-09-23T05:01:42.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78756 28.48 MB/sec execute 17 sec latency 67.439 ms 2023-09-23T05:01:43.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:42 smithi069 ceph-mon[133931]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-09-23T05:01:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:42 smithi005 ceph-mon[97638]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-09-23T05:01:43.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:42 smithi118 ceph-mon[134297]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-09-23T05:01:43.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79392 28.85 MB/sec execute 18 sec latency 113.955 ms 2023-09-23T05:01:44.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80155 29.49 MB/sec execute 19 sec latency 65.361 ms 2023-09-23T05:01:45.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:44 smithi069 ceph-mon[133931]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T05:01:45.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:44 smithi005 ceph-mon[97638]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T05:01:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:44 smithi118 ceph-mon[134297]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 682 op/s 2023-09-23T05:01:45.484 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80776 29.92 MB/sec execute 20 sec latency 259.365 ms 2023-09-23T05:01:46.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81176 29.49 MB/sec execute 21 sec latency 78.479 ms 2023-09-23T05:01:47.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:46 smithi069 ceph-mon[133931]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 562 op/s 2023-09-23T05:01:47.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:46 smithi005 ceph-mon[97638]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 562 op/s 2023-09-23T05:01:47.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:46 smithi118 ceph-mon[134297]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 562 op/s 2023-09-23T05:01:47.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82026 30.05 MB/sec execute 22 sec latency 56.599 ms 2023-09-23T05:01:48.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82799 30.42 MB/sec execute 23 sec latency 61.231 ms 2023-09-23T05:01:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:48 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:49.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:48 smithi069 ceph-mon[133931]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 795 op/s 2023-09-23T05:01:49.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:48 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:49.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:48 smithi005 ceph-mon[97638]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 795 op/s 2023-09-23T05:01:49.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:48 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:01:49.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:48 smithi118 ceph-mon[134297]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 795 op/s 2023-09-23T05:01:49.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83555 31.16 MB/sec execute 24 sec latency 59.311 ms 2023-09-23T05:01:50.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84267 31.49 MB/sec execute 25 sec latency 113.289 ms 2023-09-23T05:01:51.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84891 31.15 MB/sec execute 26 sec latency 113.875 ms 2023-09-23T05:01:52.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:51 smithi005 ceph-mon[97638]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 520 op/s 2023-09-23T05:01:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:51 smithi118 ceph-mon[134297]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 520 op/s 2023-09-23T05:01:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:51 smithi069 ceph-mon[133931]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 520 op/s 2023-09-23T05:01:52.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85659 31.92 MB/sec execute 27 sec latency 52.818 ms 2023-09-23T05:01:53.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:52 smithi069 ceph-mon[133931]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-23T05:01:53.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:52 smithi005 ceph-mon[97638]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-23T05:01:53.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:52 smithi118 ceph-mon[134297]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 673 op/s 2023-09-23T05:01:53.486 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86313 31.70 MB/sec execute 28 sec latency 71.827 ms 2023-09-23T05:01:54.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:53 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:01:54.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:53 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:01:54.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:53 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:01:54.488 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86455 30.90 MB/sec execute 29 sec latency 817.721 ms 2023-09-23T05:01:55.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:55 smithi005 ceph-mon[97638]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 593 op/s 2023-09-23T05:01:55.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:55 smithi118 ceph-mon[134297]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 593 op/s 2023-09-23T05:01:55.487 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86554 30.07 MB/sec execute 30 sec latency 1549.887 ms 2023-09-23T05:01:55.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:55 smithi069 ceph-mon[133931]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 593 op/s 2023-09-23T05:01:56.490 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86789 29.60 MB/sec execute 31 sec latency 243.771 ms 2023-09-23T05:01:57.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:56 smithi005 ceph-mon[97638]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 497 op/s 2023-09-23T05:01:57.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:57 smithi118 ceph-mon[134297]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 497 op/s 2023-09-23T05:01:57.492 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87092 29.27 MB/sec execute 32 sec latency 127.157 ms 2023-09-23T05:01:57.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:56 smithi069 ceph-mon[133931]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 497 op/s 2023-09-23T05:01:58.492 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87592 29.16 MB/sec execute 33 sec latency 166.170 ms 2023-09-23T05:01:59.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:01:58 smithi005 ceph-mon[97638]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 847 op/s 2023-09-23T05:01:59.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:01:58 smithi118 ceph-mon[134297]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 847 op/s 2023-09-23T05:01:59.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:01:58 smithi069 ceph-mon[133931]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 847 op/s 2023-09-23T05:01:59.492 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88228 29.29 MB/sec execute 34 sec latency 77.966 ms 2023-09-23T05:02:00.493 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88731 29.17 MB/sec execute 35 sec latency 264.995 ms 2023-09-23T05:02:01.495 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88920 28.63 MB/sec execute 36 sec latency 420.913 ms 2023-09-23T05:02:02.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:01 smithi005 ceph-mon[97638]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 660 op/s 2023-09-23T05:02:02.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:02:02.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:02:02.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:02:02.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:01 smithi118 ceph-mon[134297]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 660 op/s 2023-09-23T05:02:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:02:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:02:02.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:02:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:01 smithi069 ceph-mon[133931]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 660 op/s 2023-09-23T05:02:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:02:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:02:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:02:02.495 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89348 28.54 MB/sec execute 37 sec latency 177.698 ms 2023-09-23T05:02:03.495 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90183 28.84 MB/sec execute 38 sec latency 68.659 ms 2023-09-23T05:02:04.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:03 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:04.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:03 smithi005 ceph-mon[97638]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 835 op/s 2023-09-23T05:02:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:03 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:04.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:03 smithi118 ceph-mon[134297]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 835 op/s 2023-09-23T05:02:04.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:03 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:04.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:03 smithi069 ceph-mon[133931]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 835 op/s 2023-09-23T05:02:04.496 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90983 29.47 MB/sec execute 39 sec latency 156.464 ms 2023-09-23T05:02:05.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:05 smithi005 ceph-mon[97638]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 781 op/s 2023-09-23T05:02:05.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:05 smithi118 ceph-mon[134297]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 781 op/s 2023-09-23T05:02:05.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:05 smithi069 ceph-mon[133931]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 781 op/s 2023-09-23T05:02:05.496 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91458 29.33 MB/sec execute 40 sec latency 306.025 ms 2023-09-23T05:02:06.496 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91956 29.12 MB/sec execute 41 sec latency 368.239 ms 2023-09-23T05:02:07.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:07 smithi005 ceph-mon[97638]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T05:02:07.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:07 smithi005 ceph-mon[97638]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 804 op/s 2023-09-23T05:02:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:07 smithi118 ceph-mon[134297]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T05:02:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:07 smithi118 ceph-mon[134297]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 804 op/s 2023-09-23T05:02:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:07 smithi069 ceph-mon[133931]: osdmap e111: 12 total, 12 up, 12 in 2023-09-23T05:02:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:07 smithi069 ceph-mon[133931]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 804 op/s 2023-09-23T05:02:07.496 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92577 29.31 MB/sec execute 42 sec latency 120.636 ms 2023-09-23T05:02:08.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:08 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:08.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:08 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:08.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:02:08.490 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:08 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:08.496 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93464 29.70 MB/sec execute 43 sec latency 93.654 ms 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 -- 172.21.15.5:0/3959920073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9238101d70 msgr2=0x7f9238102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 --2- 172.21.15.5:0/3959920073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9238101d70 0x7f9238102150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9234009990 tx=0x7f9234009ca0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 -- 172.21.15.5:0/3959920073 shutdown_connections 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 --2- 172.21.15.5:0/3959920073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f923810b190 0x7f923810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 --2- 172.21.15.5:0/3959920073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9238101d70 0x7f9238102150 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 --2- 172.21.15.5:0/3959920073 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9238102690 0x7f923810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 -- 172.21.15.5:0/3959920073 >> 172.21.15.5:0/3959920073 conn(0x7f92380fb570 msgr2=0x7f92380fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 -- 172.21.15.5:0/3959920073 shutdown_connections 2023-09-23T05:02:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.795+0000 7f923e79d700 1 -- 172.21.15.5:0/3959920073 wait complete. 2023-09-23T05:02:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.796+0000 7f923e79d700 1 Processor -- start 2023-09-23T05:02:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.796+0000 7f923e79d700 1 -- start start 2023-09-23T05:02:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9238101d70 0x7f9238198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9238102690 0x7f9238199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923819def0 con 0x7f9238102690 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923819e060 con 0x7f923810b190 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923e79d700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923819e220 con 0x7f9238101d70 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33490/0 (socket says 172.21.15.5:33490) 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923df9c700 1 -- 172.21.15.5:0/3294533107 learned_addr learned my addr 172.21.15.5:0/3294533107 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.797+0000 7f923cf9a700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9238102690 0x7f9238199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 -- 172.21.15.5:0/3294533107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9238101d70 msgr2=0x7f9238198f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9238101d70 0x7f9238198f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 -- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9238102690 msgr2=0x7f9238199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9238102690 0x7f9238199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 -- 172.21.15.5:0/3294533107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92340095f0 con 0x7f923810b190 2023-09-23T05:02:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f923df9c700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f923000ef30 tx=0x7f923000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f922e7fc700 1 -- 172.21.15.5:0/3294533107 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230018070 con 0x7f923810b190 2023-09-23T05:02:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f922e7fc700 1 -- 172.21.15.5:0/3294533107 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f923000f040 con 0x7f923810b190 2023-09-23T05:02:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.798+0000 7f922e7fc700 1 -- 172.21.15.5:0/3294533107 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230003dc0 con 0x7f923810b190 2023-09-23T05:02:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.799+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923819e4a0 con 0x7f923810b190 2023-09-23T05:02:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.799+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9238102cf0 con 0x7f923810b190 2023-09-23T05:02:08.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.805+0000 7f9253343b80 1 Processor -- start 2023-09-23T05:02:08.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.806+0000 7f922e7fc700 1 -- 172.21.15.5:0/3294533107 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f923000caa0 con 0x7f923810b190 2023-09-23T05:02:08.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.806+0000 7f9253343b80 1 -- start start 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6b5c460 con 0x55a8f6bf3e90 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6b4dc20 con 0x55a8f6bfed40 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f9253343b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6b4b540 con 0x55a8f6bf3570 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f923cf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f923cf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56728/0 (socket says 172.21.15.5:56728) 2023-09-23T05:02:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.807+0000 7f923cf9a700 1 -- 172.21.15.5:0/2805928988 learned_addr learned my addr 172.21.15.5:0/2805928988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.808+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.808+0000 7f923d79b700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 -- 172.21.15.5:0/2805928988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 msgr2=0x55a8f6c01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 -- 172.21.15.5:0/2805928988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 msgr2=0x55a8f6bfe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 -- 172.21.15.5:0/2805928988 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8f6ba98d0 con 0x55a8f6bf3570 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923d79b700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92280146a0 tx=0x7f92280042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef22a258d91a4028 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f92237fe700 1 -- 172.21.15.5:0/2805928988 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228014cc0 con 0x55a8f6bf3570 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f92237fe700 1 -- 172.21.15.5:0/2805928988 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9228014e20 con 0x55a8f6bf3570 2023-09-23T05:02:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.809+0000 7f92237fe700 1 -- 172.21.15.5:0/2805928988 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228011400 con 0x55a8f6bf3570 2023-09-23T05:02:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.810+0000 7f922e7fc700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 0x7f92240a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.810+0000 7f922e7fc700 1 -- 172.21.15.5:0/3294533107 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f92300cd3b0 con 0x7f923810b190 2023-09-23T05:02:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 -- 172.21.15.5:0/2805928988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 msgr2=0x55a8f6bf3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92280146a0 tx=0x7f92280042d0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f923d79b700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 0x7f92240a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 -- 172.21.15.5:0/2805928988 shutdown_connections 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6bf3950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 --2- 172.21.15.5:0/2805928988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.811+0000 7f9253343b80 1 -- 172.21.15.5:0/2805928988 >> 172.21.15.5:0/2805928988 conn(0x55a8f6b68260 msgr2=0x55a8f6bef9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.812+0000 7f9253343b80 1 -- 172.21.15.5:0/2805928988 shutdown_connections 2023-09-23T05:02:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.812+0000 7f9253343b80 1 -- 172.21.15.5:0/2805928988 wait complete. 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.812+0000 7f9253343b80 1 -- start start 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 Processor -- start 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6bfe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bfbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6b46e30 con 0x55a8f6bf3570 2023-09-23T05:02:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.813+0000 7f9253343b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6ae3710 con 0x55a8f6bfed40 2023-09-23T05:02:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f9253343b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6ae33b0 con 0x55a8f6bf3e90 2023-09-23T05:02:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f9253343b80 10 client.0 _ll_register_callbacks cb 0x55a8f6940800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923d79b700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 0x7f92240a3cd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f923400b2e0 tx=0x7f9234005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56736/0 (socket says 172.21.15.5:56736) 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 -- 172.21.15.5:0/2253453004 learned_addr learned my addr 172.21.15.5:0/2253453004 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923df9c700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bfbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923d79b700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6bfe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 -- 172.21.15.5:0/2253453004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 msgr2=0x55a8f6bfbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bfbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 -- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 msgr2=0x55a8f6bfe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6bfe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.814+0000 7f923cf9a700 1 -- 172.21.15.5:0/2253453004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9228014410 con 0x55a8f6bf3e90 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.815+0000 7f923cf9a700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9228010b40 tx=0x7f9228014db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c982efc4c08e3b7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.815+0000 7f9220ff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.815+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228010e40 con 0x55a8f6bf3e90 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.815+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8f6be74a0 con 0x55a8f6bf3e90 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.815+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f9228010450 con 0x55a8f6bf3e90 2023-09-23T05:02:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9228010610 con 0x55a8f6bf3e90 2023-09-23T05:02:08.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9253343b80 10 client.27751 fetch_fsmap learned FSMap version 71 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55a8f6971de0 con 0x55a8f6bf3e90 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8f6be74a0 con 0x55a8f6bf3e90 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 71) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f9228038020 con 0x55a8f6bf3e90 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9253343b80 10 client.27751 fetch_fsmap finished waiting for FSMap version 71 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.816+0000 7f9253343b80 10 client.27751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.817+0000 7f9253343b80 20 client.27751 populate_metadata read hostname 'smithi005' 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.817+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c0e750 0x55a8f6c10b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.817+0000 7f9253343b80 4 client.27751 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.817+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8f6b3fe10 con 0x55a8f6c0e750 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.817+0000 7f923d79b700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c0e750 0x55a8f6c10b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.818+0000 7f923d79b700 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c0e750 0x55a8f6c10b70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733c4f1c2bebf08a server_cookie=eedd7d1ee310cf1d in_seq=0 out_seq=0 2023-09-23T05:02:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.818+0000 7f9220ff9700 10 client.27751 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:08.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.818+0000 7f9220ff9700 1 -- 172.21.15.5:0/2253453004 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8f6b3fe10 con 0x55a8f6c0e750 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.818+0000 7f9220ff9700 10 client.27751 handle_command_reply: tid=0 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.818+0000 7f9253343b80 1 client.27751 shutdown 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 msgr2=0x55a8f6bfb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9228010b40 tx=0x7f9228014db0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 shutdown_connections 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c0e750 0x55a8f6c10b70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bfbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6bfe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 --2- 172.21.15.5:0/2253453004 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bfb930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.819+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 >> 172.21.15.5:0/2253453004 conn(0x55a8f6b68260 msgr2=0x55a8f6bef540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.820+0000 7f9220ff9700 0 client.27751 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.820+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 shutdown_connections 2023-09-23T05:02:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.820+0000 7f9253343b80 1 -- 172.21.15.5:0/2253453004 wait complete. 2023-09-23T05:02:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.822+0000 7f9253343b80 20 client.27751 trim_cache size 0 max 16384 2023-09-23T05:02:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.850+0000 7f9253343b80 1 Processor -- start 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.850+0000 7f9253343b80 1 -- start start 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6c0a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bf6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f69c7920 con 0x55a8f6bf3e90 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6c0b180 con 0x55a8f6bfed40 2023-09-23T05:02:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f9253343b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6bf9ed0 con 0x55a8f6bf3570 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.851+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.852+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33516/0 (socket says 172.21.15.5:33516) 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.852+0000 7f923df9c700 1 -- 172.21.15.5:0/3350742335 learned_addr learned my addr 172.21.15.5:0/3350742335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 -- 172.21.15.5:0/3350742335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 msgr2=0x55a8f6c0a020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6c0a020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 -- 172.21.15.5:0/3350742335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 msgr2=0x55a8f6bf6ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T05:02:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923cf9a700 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bf6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bf6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 -- 172.21.15.5:0/3350742335 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8f695ac60 con 0x55a8f6bfed40 2023-09-23T05:02:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923cf9a700 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bf6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:02:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.853+0000 7f923df9c700 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92300090a0 tx=0x7f9230099820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2d4e0ee00aebe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.854+0000 7f9223fff700 1 -- 172.21.15.5:0/3350742335 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230099ec0 con 0x55a8f6bfed40 2023-09-23T05:02:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.854+0000 7f9223fff700 1 -- 172.21.15.5:0/3350742335 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f923009e010 con 0x55a8f6bfed40 2023-09-23T05:02:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.854+0000 7f9223fff700 1 -- 172.21.15.5:0/3350742335 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000ce70 con 0x55a8f6bfed40 2023-09-23T05:02:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 -- 172.21.15.5:0/3350742335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 msgr2=0x55a8f6bf71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92300090a0 tx=0x7f9230099820 comp rx=0 tx=0).stop 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 -- 172.21.15.5:0/3350742335 shutdown_connections 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6bf71e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3570 0x55a8f6c0a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 --2- 172.21.15.5:0/3350742335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3e90 0x55a8f6bf6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.855+0000 7f9253343b80 1 -- 172.21.15.5:0/3350742335 >> 172.21.15.5:0/3350742335 conn(0x55a8f6b68260 msgr2=0x55a8f6c10760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.856+0000 7f9253343b80 1 -- 172.21.15.5:0/3350742335 shutdown_connections 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.856+0000 7f9253343b80 1 -- 172.21.15.5:0/3350742335 wait complete. 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.856+0000 7f9253343b80 1 -- start start 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 Processor -- start 2023-09-23T05:02:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6c03aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6c03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6c5daa0 con 0x55a8f6bf3570 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6c5dc10 con 0x55a8f6bfed40 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6c5ddd0 con 0x55a8f6bf3e90 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.857+0000 7f9253343b80 10 client.0 _ll_register_callbacks cb 0x7f9228003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.858+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.858+0000 7f923df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33526/0 (socket says 172.21.15.5:33526) 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.858+0000 7f923df9c700 1 -- 172.21.15.5:0/2403039191 learned_addr learned my addr 172.21.15.5:0/2403039191 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923d79b700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6c03aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 -- 172.21.15.5:0/2403039191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 msgr2=0x55a8f6c03fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6c03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 -- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 msgr2=0x55a8f6c03aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6c03aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 -- 172.21.15.5:0/2403039191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9238102cf0 con 0x55a8f6bfed40 2023-09-23T05:02:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f923df9c700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9230096930 tx=0x7f92300a2270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a3eca75b7e2dcb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.859+0000 7f9220ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:02:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.860+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300b81c0 con 0x55a8f6bfed40 2023-09-23T05:02:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.860+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f92300cd0b0 con 0x55a8f6bfed40 2023-09-23T05:02:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.860+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9228014e20 con 0x55a8f6bfed40 2023-09-23T05:02:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.866+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923009a120 con 0x55a8f6bfed40 2023-09-23T05:02:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.871+0000 7f9253343b80 10 client.27710 fetch_fsmap learned FSMap version 71 2023-09-23T05:02:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.871+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55a8f6998250 con 0x55a8f6bfed40 2023-09-23T05:02:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.871+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9228014e20 con 0x55a8f6bfed40 2023-09-23T05:02:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.872+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 71) v1 ==== 7938+0+0 (secure 0 0 0) 0x7f92300a1d30 con 0x55a8f6bfed40 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.872+0000 7f9253343b80 10 client.27710 fetch_fsmap finished waiting for FSMap version 71 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.872+0000 7f9253343b80 10 client.27710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.872+0000 7f9253343b80 20 client.27710 populate_metadata read hostname 'smithi005' 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.873+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c5efa0 0x55a8f6c613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.873+0000 7f9253343b80 4 client.27710 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:02:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.873+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8f6b3fb30 con 0x55a8f6c5efa0 2023-09-23T05:02:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.881+0000 7f923d79b700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c5efa0 0x55a8f6c613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.885+0000 7f923d79b700 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c5efa0 0x55a8f6c613c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927a606ef6bde11 server_cookie=6367fb01f1820124 in_seq=0 out_seq=0 2023-09-23T05:02:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.885+0000 7f9220ff9700 10 client.27710 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.888+0000 7f9220ff9700 1 -- 172.21.15.5:0/2403039191 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a8f6b3fb30 con 0x55a8f6c5efa0 2023-09-23T05:02:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.888+0000 7f9220ff9700 10 client.27710 handle_command_reply: tid=0 2023-09-23T05:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.888+0000 7f9253343b80 1 client.27710 shutdown 2023-09-23T05:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.888+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 msgr2=0x55a8f6c5d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.888+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9230096930 tx=0x7f92300a2270 comp rx=0 tx=0).stop 2023-09-23T05:02:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 shutdown_connections 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a8f6c5efa0 0x55a8f6c613c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8f6bfed40 0x55a8f6c5d560 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8f6bf3570 0x55a8f6c03aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 --2- 172.21.15.5:0/2403039191 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a8f6bf3e90 0x55a8f6c03fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 >> 172.21.15.5:0/2403039191 conn(0x55a8f6b68260 msgr2=0x55a8f6c0a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9220ff9700 0 client.27710 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 shutdown_connections 2023-09-23T05:02:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.889+0000 7f9253343b80 1 -- 172.21.15.5:0/2403039191 wait complete. 2023-09-23T05:02:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.895+0000 7f9253343b80 20 client.27710 trim_cache size 0 max 16384 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:02:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:02:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:02:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:02:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:02:08.898 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:02:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.905+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 msgr2=0x7f92240a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.905+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 0x7f92240a3cd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f923400b2e0 tx=0x7f9234005fd0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.905+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 msgr2=0x7f923819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.905+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f923000ef30 tx=0x7f923000c450 comp rx=0 tx=0).stop 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 shutdown_connections 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f92240a1810 0x7f92240a3cd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f923810b190 0x7f923819d8b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9238101d70 0x7f9238198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 --2- 172.21.15.5:0/3294533107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9238102690 0x7f9238199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 >> 172.21.15.5:0/3294533107 conn(0x7f92380fb570 msgr2=0x7f923810d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 shutdown_connections 2023-09-23T05:02:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:08.906+0000 7f923e79d700 1 -- 172.21.15.5:0/3294533107 wait complete. 2023-09-23T05:02:08.961 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:02:09.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:09 smithi005 ceph-mon[97638]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:02:09.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:09 smithi118 ceph-mon[134297]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:02:09.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:09 smithi069 ceph-mon[133931]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:02:09.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93783 29.44 MB/sec execute 44 sec latency 427.909 ms 2023-09-23T05:02:10.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94089 29.23 MB/sec execute 45 sec latency 621.382 ms 2023-09-23T05:02:11.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94629 29.19 MB/sec execute 46 sec latency 182.238 ms 2023-09-23T05:02:12.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:11 smithi005 ceph-mon[97638]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-23T05:02:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:11 smithi118 ceph-mon[134297]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-23T05:02:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:11 smithi069 ceph-mon[133931]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-23T05:02:12.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95114 29.18 MB/sec execute 47 sec latency 166.901 ms 2023-09-23T05:02:13.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:13 smithi005 ceph-mon[97638]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 685 op/s 2023-09-23T05:02:13.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:13 smithi118 ceph-mon[134297]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 685 op/s 2023-09-23T05:02:13.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:13 smithi069 ceph-mon[133931]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 685 op/s 2023-09-23T05:02:13.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95715 29.10 MB/sec execute 48 sec latency 248.254 ms 2023-09-23T05:02:14.499 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96579 29.47 MB/sec execute 49 sec latency 68.689 ms 2023-09-23T05:02:15.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:15 smithi005 ceph-mon[97638]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:15.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:15 smithi118 ceph-mon[134297]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:15.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:15 smithi069 ceph-mon[133931]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:15.500 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97202 29.57 MB/sec execute 50 sec latency 78.551 ms 2023-09-23T05:02:16.500 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97955 29.94 MB/sec execute 51 sec latency 69.002 ms 2023-09-23T05:02:17.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:16 smithi005 ceph-mon[97638]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T05:02:17.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:16 smithi118 ceph-mon[134297]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T05:02:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:16 smithi069 ceph-mon[133931]: osdmap e112: 12 total, 12 up, 12 in 2023-09-23T05:02:17.501 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98458 29.77 MB/sec execute 52 sec latency 129.508 ms 2023-09-23T05:02:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:17 smithi005 ceph-mon[97638]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:17 smithi118 ceph-mon[134297]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:18.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:17 smithi069 ceph-mon[133931]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:02:18.501 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99159 29.81 MB/sec execute 53 sec latency 62.141 ms 2023-09-23T05:02:19.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:19 smithi005 ceph-mon[97638]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 688 op/s 2023-09-23T05:02:19.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:19 smithi118 ceph-mon[134297]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 688 op/s 2023-09-23T05:02:19.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:19 smithi069 ceph-mon[133931]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 688 op/s 2023-09-23T05:02:19.501 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99929 30.04 MB/sec execute 54 sec latency 90.118 ms 2023-09-23T05:02:20.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100877 30.55 MB/sec execute 55 sec latency 66.380 ms 2023-09-23T05:02:21.502 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101532 30.71 MB/sec execute 56 sec latency 86.493 ms 2023-09-23T05:02:22.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:21 smithi005 ceph-mon[97638]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 695 op/s 2023-09-23T05:02:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:21 smithi118 ceph-mon[134297]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 695 op/s 2023-09-23T05:02:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:21 smithi069 ceph-mon[133931]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 695 op/s 2023-09-23T05:02:22.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102096 30.68 MB/sec execute 57 sec latency 74.910 ms 2023-09-23T05:02:23.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:23 smithi005 ceph-mon[97638]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 647 op/s 2023-09-23T05:02:23.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:23 smithi118 ceph-mon[134297]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 647 op/s 2023-09-23T05:02:23.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:23 smithi069 ceph-mon[133931]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 647 op/s 2023-09-23T05:02:23.502 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102641 30.49 MB/sec execute 58 sec latency 63.459 ms 2023-09-23T05:02:24.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103311 30.54 MB/sec execute 59 sec latency 63.331 ms 2023-09-23T05:02:25.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104329 31.12 MB/sec execute 60 sec latency 49.092 ms 2023-09-23T05:02:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:25 smithi069 ceph-mon[133931]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 647 op/s 2023-09-23T05:02:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:25 smithi005 ceph-mon[97638]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 647 op/s 2023-09-23T05:02:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:25 smithi118 ceph-mon[134297]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 647 op/s 2023-09-23T05:02:26.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104975 31.15 MB/sec execute 61 sec latency 66.430 ms 2023-09-23T05:02:27.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:26 smithi069 ceph-mon[133931]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 583 op/s 2023-09-23T05:02:27.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:26 smithi005 ceph-mon[97638]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 583 op/s 2023-09-23T05:02:27.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:26 smithi118 ceph-mon[134297]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 583 op/s 2023-09-23T05:02:27.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105564 31.15 MB/sec execute 62 sec latency 68.075 ms 2023-09-23T05:02:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:27 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:28.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:27 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:27 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:28.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106486 31.28 MB/sec execute 63 sec latency 54.283 ms 2023-09-23T05:02:29.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:28 smithi005 ceph-mon[97638]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 756 op/s 2023-09-23T05:02:29.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:28 smithi118 ceph-mon[134297]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 756 op/s 2023-09-23T05:02:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:28 smithi069 ceph-mon[133931]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.7 MiB/s wr, 756 op/s 2023-09-23T05:02:29.503 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107115 31.56 MB/sec execute 64 sec latency 139.637 ms 2023-09-23T05:02:30.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107742 31.56 MB/sec execute 65 sec latency 51.736 ms 2023-09-23T05:02:31.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108348 31.48 MB/sec execute 66 sec latency 72.357 ms 2023-09-23T05:02:31.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:31 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:31.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:31 smithi005 ceph-mon[97638]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 540 op/s 2023-09-23T05:02:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:31 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:31.910 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:31 smithi118 ceph-mon[134297]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 540 op/s 2023-09-23T05:02:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:31 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:31.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:31 smithi069 ceph-mon[133931]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 540 op/s 2023-09-23T05:02:32.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109098 31.59 MB/sec execute 67 sec latency 51.518 ms 2023-09-23T05:02:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:33 smithi005 ceph-mon[97638]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-23T05:02:33.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:33 smithi118 ceph-mon[134297]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-23T05:02:33.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:33 smithi069 ceph-mon[133931]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 655 op/s 2023-09-23T05:02:33.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109899 31.84 MB/sec execute 68 sec latency 74.237 ms 2023-09-23T05:02:34.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110542 31.88 MB/sec execute 69 sec latency 86.861 ms 2023-09-23T05:02:35.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:35 smithi005 ceph-mon[97638]: pgmap v512: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 584 op/s 2023-09-23T05:02:35.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:35 smithi118 ceph-mon[134297]: pgmap v512: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 584 op/s 2023-09-23T05:02:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:35 smithi069 ceph-mon[133931]: pgmap v512: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.4 MiB/s wr, 584 op/s 2023-09-23T05:02:35.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111064 31.72 MB/sec execute 70 sec latency 107.676 ms 2023-09-23T05:02:36.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111756 31.93 MB/sec execute 71 sec latency 275.019 ms 2023-09-23T05:02:37.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:37 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:37.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:37 smithi005 ceph-mon[97638]: pgmap v513: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-09-23T05:02:37.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:37 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:37.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:37 smithi118 ceph-mon[134297]: pgmap v513: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-09-23T05:02:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:37 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:37 smithi069 ceph-mon[133931]: pgmap v513: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-09-23T05:02:37.504 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112552 32.03 MB/sec execute 72 sec latency 82.698 ms 2023-09-23T05:02:38.508 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112961 31.93 MB/sec execute 73 sec latency 294.525 ms 2023-09-23T05:02:38.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:02:39.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:39 smithi118 ceph-mon[134297]: pgmap v514: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 621 op/s 2023-09-23T05:02:39.413 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:39 smithi005 ceph-mon[97638]: pgmap v514: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 621 op/s 2023-09-23T05:02:39.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:39 smithi069 ceph-mon[133931]: pgmap v514: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 621 op/s 2023-09-23T05:02:39.521 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113004 31.52 MB/sec execute 74 sec latency 1056.775 ms 2023-09-23T05:02:40.520 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113188 31.25 MB/sec execute 75 sec latency 1341.197 ms 2023-09-23T05:02:41.520 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113364 31.00 MB/sec execute 76 sec latency 319.430 ms 2023-09-23T05:02:41.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:41 smithi005 ceph-mon[97638]: pgmap v515: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2023-09-23T05:02:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:41 smithi118 ceph-mon[134297]: pgmap v515: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2023-09-23T05:02:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:41 smithi069 ceph-mon[133931]: pgmap v515: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 531 op/s 2023-09-23T05:02:42.521 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113457 30.65 MB/sec execute 77 sec latency 552.861 ms 2023-09-23T05:02:43.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:43 smithi005 ceph-mon[97638]: pgmap v516: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 781 op/s 2023-09-23T05:02:43.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:43 smithi118 ceph-mon[134297]: pgmap v516: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 781 op/s 2023-09-23T05:02:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:43 smithi069 ceph-mon[133931]: pgmap v516: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 781 op/s 2023-09-23T05:02:43.522 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113692 30.38 MB/sec execute 78 sec latency 251.467 ms 2023-09-23T05:02:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 -- 172.21.15.5:0/3867557177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b40107e20 msgr2=0x7f2b401082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 --2- 172.21.15.5:0/3867557177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b40107e20 0x7f2b401082a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b3001a9a0 tx=0x7f2b3001acb0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 -- 172.21.15.5:0/3867557177 shutdown_connections 2023-09-23T05:02:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 --2- 172.21.15.5:0/3867557177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b40107e20 0x7f2b401082a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 --2- 172.21.15.5:0/3867557177 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b4010f5f0 0x7f2b4010d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 --2- 172.21.15.5:0/3867557177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f9d0 0x7f2b4010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.522+0000 7f2b463ff700 1 -- 172.21.15.5:0/3867557177 >> 172.21.15.5:0/3867557177 conn(0x7f2b400725f0 msgr2=0x7f2b4006d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.523+0000 7f2b463ff700 1 -- 172.21.15.5:0/3867557177 shutdown_connections 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.523+0000 7f2b463ff700 1 -- 172.21.15.5:0/3867557177 wait complete. 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.525+0000 7f2b463ff700 1 Processor -- start 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.525+0000 7f2b463ff700 1 -- start start 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.525+0000 7f2b463ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.525+0000 7f2b463ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 0x7f2b4007dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b463ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b4010f9d0 0x7f2b40083690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b463ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40083e60 con 0x7f2b4010f5f0 2023-09-23T05:02:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b463ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40078990 con 0x7f2b4010f9d0 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b463ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40078b50 con 0x7f2b40107e20 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45128/0 (socket says 172.21.15.5:45128) 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b3ffff700 1 -- 172.21.15.5:0/1123433759 learned_addr learned my addr 172.21.15.5:0/1123433759 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b4499c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b4010f9d0 0x7f2b40083690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3ffff700 1 -- 172.21.15.5:0/1123433759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b4010f9d0 msgr2=0x7f2b40083690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b4010f9d0 0x7f2b40083690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3ffff700 1 -- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 msgr2=0x7f2b4007dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.526+0000 7f2b3f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 0x7f2b4007dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 0x7f2b4007dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3ffff700 1 -- 172.21.15.5:0/1123433759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3001a600 con 0x7f2b40107e20 2023-09-23T05:02:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.527+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 0x7f2b4007dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.528+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2b34051960 tx=0x7f2b34051d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.529+0000 7f2b3d7fa700 1 -- 172.21.15.5:0/1123433759 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34009b40 con 0x7f2b40107e20 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.529+0000 7f2b3d7fa700 1 -- 172.21.15.5:0/1123433759 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b34009ca0 con 0x7f2b40107e20 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.529+0000 7f2b3d7fa700 1 -- 172.21.15.5:0/1123433759 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340536c0 con 0x7f2b40107e20 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.529+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b40078dd0 con 0x7f2b40107e20 2023-09-23T05:02:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.529+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b40079260 con 0x7f2b40107e20 2023-09-23T05:02:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.548+0000 7f2b59de6b80 1 Processor -- start 2023-09-23T05:02:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 -- start start 2023-09-23T05:02:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11af47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11affbd0 0x564c11b01fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11a5d460 con 0x564c11af4400 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11a4ec20 con 0x564c11affbd0 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.549+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11a4c540 con 0x564c11af4d20 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.550+0000 7f2b3d7fa700 1 -- 172.21.15.5:0/1123433759 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f2b34054470 con 0x7f2b40107e20 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.550+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.550+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45140/0 (socket says 172.21.15.5:45140) 2023-09-23T05:02:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.550+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3389756501 learned_addr learned my addr 172.21.15.5:0/3389756501 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b4499c700 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11af47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3389756501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11affbd0 msgr2=0x564c11b01fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11affbd0 0x564c11b01fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3389756501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 msgr2=0x564c11af47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11af47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.551+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3389756501 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c11aaa8d0 con 0x564c11af4d20 2023-09-23T05:02:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.552+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b3410e840 tx=0x7f2b34117820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f272c78ed15172c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.553+0000 7f2b3d7fa700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 0x7f2b280a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.553+0000 7f2b3d7fa700 1 -- 172.21.15.5:0/1123433759 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f2b34111640 con 0x7f2b40107e20 2023-09-23T05:02:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.553+0000 7f2b4499c700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 0x7f2b280a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.554+0000 7f2b267fc700 1 -- 172.21.15.5:0/3389756501 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34111e40 con 0x564c11af4d20 2023-09-23T05:02:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.554+0000 7f2b267fc700 1 -- 172.21.15.5:0/3389756501 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b34113590 con 0x564c11af4d20 2023-09-23T05:02:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.554+0000 7f2b267fc700 1 -- 172.21.15.5:0/3389756501 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3411ea00 con 0x564c11af4d20 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.555+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3389756501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 msgr2=0x564c11aff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.555+0000 7f2b4499c700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 0x7f2b280a3de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b30003820 tx=0x7f2b3002a040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.555+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b3410e840 tx=0x7f2b34117820 comp rx=0 tx=0).stop 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.556+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3389756501 shutdown_connections 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.556+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11affbd0 0x564c11b01fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.556+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11af47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.556+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3389756501 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11aff690 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.556+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3389756501 >> 172.21.15.5:0/3389756501 conn(0x564c11a69260 msgr2=0x564c11af0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.567+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3389756501 shutdown_connections 2023-09-23T05:02:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.576+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3389756501 wait complete. 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.577+0000 7f2b59de6b80 1 -- start start 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.577+0000 7f2b59de6b80 1 Processor -- start 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11afc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11afcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11a47e30 con 0x564c11affbd0 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c119e4710 con 0x564c11af4400 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c119e43b0 con 0x564c11af4d20 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.578+0000 7f2b59de6b80 10 client.0 _ll_register_callbacks cb 0x564c11841800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:02:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.579+0000 7f2b4499c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.579+0000 7f2b4499c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40162/0 (socket says 172.21.15.5:40162) 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.579+0000 7f2b4499c700 1 -- 172.21.15.5:0/3317347383 learned_addr learned my addr 172.21.15.5:0/3317347383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.579+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11afc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.580+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3317347383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 msgr2=0x564c11afcef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.580+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11afcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.580+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 msgr2=0x564c11afd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.580+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.580+0000 7f2b3ffff700 1 -- 172.21.15.5:0/3317347383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b40079260 con 0x564c11af4400 2023-09-23T05:02:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.581+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11afc9b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b34117300 tx=0x7f2b34054d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa49762cd0b768a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.581+0000 7f2b1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:02:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.581+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3410e870 con 0x564c11af4400 2023-09-23T05:02:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.581+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f2b340de820 con 0x564c11af4400 2023-09-23T05:02:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.581+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c11ae8250 con 0x564c11af4400 2023-09-23T05:02:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.582+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b3411e360 con 0x564c11af4400 2023-09-23T05:02:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.582+0000 7f2b59de6b80 10 client.27719 fetch_fsmap learned FSMap version 74 2023-09-23T05:02:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.582+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x564c11872de0 con 0x564c11af4400 2023-09-23T05:02:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c11ae8250 con 0x564c11af4400 2023-09-23T05:02:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 74) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2b340de380 con 0x564c11af4400 2023-09-23T05:02:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 10 client.27719 fetch_fsmap finished waiting for FSMap version 74 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 10 client.27719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 20 client.27719 populate_metadata read hostname 'smithi005' 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b0f240 0x564c11b11630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 4 client.27719 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.585+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c11a40e10 con 0x564c11b0f240 2023-09-23T05:02:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.587+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b0f240 0x564c11b11630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.588+0000 7f2b1f7fe700 10 client.27719 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.588+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b0f240 0x564c11b11630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1c132989ec74fd server_cookie=1ac3178c1171cf92 in_seq=0 out_seq=0 2023-09-23T05:02:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.591+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/3317347383 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c11a40e10 con 0x564c11b0f240 2023-09-23T05:02:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.591+0000 7f2b1f7fe700 10 client.27719 handle_command_reply: tid=0 2023-09-23T05:02:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.591+0000 7f2b59de6b80 1 client.27719 shutdown 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.596+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 msgr2=0x564c11afc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.596+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11afc9b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b34117300 tx=0x7f2b34054d30 comp rx=0 tx=0).stop 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 shutdown_connections 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b0f240 0x564c11b11630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11afc9b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/3317347383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11afcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.600+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 >> 172.21.15.5:0/3317347383 conn(0x564c11a69260 msgr2=0x564c11af0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.601+0000 7f2b1f7fe700 0 client.27719 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.601+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 shutdown_connections 2023-09-23T05:02:43.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.606+0000 7f2b59de6b80 1 -- 172.21.15.5:0/3317347383 wait complete. 2023-09-23T05:02:43.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.607+0000 7f2b59de6b80 20 client.27719 trim_cache size 0 max 16384 2023-09-23T05:02:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.658+0000 7f2b59de6b80 1 Processor -- start 2023-09-23T05:02:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.658+0000 7f2b59de6b80 1 -- start start 2023-09-23T05:02:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11b06710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c118c8920 con 0x564c11affbd0 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11b06c50 con 0x564c11af4400 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11af55b0 con 0x564c11af4d20 2023-09-23T05:02:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33108/0 (socket says 172.21.15.5:33108) 2023-09-23T05:02:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.659+0000 7f2b3ffff700 1 -- 172.21.15.5:0/2404586228 learned_addr learned my addr 172.21.15.5:0/2404586228 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b4499c700 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 -- 172.21.15.5:0/2404586228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 msgr2=0x564c11b06710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11b06710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 -- 172.21.15.5:0/2404586228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 msgr2=0x564c11afd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 -- 172.21.15.5:0/2404586228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c11aec480 con 0x564c11af4400 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.660+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b340de0b0 tx=0x7f2b340dc6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c272658abf30c72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.661+0000 7f2b26ffd700 1 -- 172.21.15.5:0/2404586228 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b341171d0 con 0x564c11af4400 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.661+0000 7f2b26ffd700 1 -- 172.21.15.5:0/2404586228 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b340da460 con 0x564c11af4400 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.661+0000 7f2b26ffd700 1 -- 172.21.15.5:0/2404586228 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340ddef0 con 0x564c11af4400 2023-09-23T05:02:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2404586228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 msgr2=0x564c11af73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b340de0b0 tx=0x7f2b340dc6d0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2404586228 shutdown_connections 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11affbd0 0x564c11afd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4400 0x564c11af73a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2404586228 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11af4d20 0x564c11b06710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.662+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2404586228 >> 172.21.15.5:0/2404586228 conn(0x564c11a69260 msgr2=0x564c11af7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.663+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2404586228 shutdown_connections 2023-09-23T05:02:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.663+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2404586228 wait complete. 2023-09-23T05:02:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.663+0000 7f2b59de6b80 1 -- start start 2023-09-23T05:02:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.664+0000 7f2b59de6b80 1 Processor -- start 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.664+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11b043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.664+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.664+0000 7f2b59de6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 0x564c11b61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.664+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11b05050 con 0x564c11af4400 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11b62530 con 0x564c11af4d20 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b59de6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c11b626f0 con 0x564c11affbd0 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b59de6b80 10 client.0 _ll_register_callbacks cb 0x7f2b3001ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b4499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 0x564c11b61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b4499c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 0x564c11b61ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45162/0 (socket says 172.21.15.5:45162) 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b3f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:33110/0 (socket says 172.21.15.5:33110) 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b4499c700 1 -- 172.21.15.5:0/2142386407 learned_addr learned my addr 172.21.15.5:0/2142386407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.665+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11b043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 -- 172.21.15.5:0/2142386407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 msgr2=0x564c11b61ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 0x564c11b61ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 -- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 msgr2=0x564c11b043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11b043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 -- 172.21.15.5:0/2142386407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c11872de0 con 0x564c11af4d20 2023-09-23T05:02:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b3f7fe700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b4007a230 tx=0x7f2b38015a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad47cfc5d61a340 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:02:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.666+0000 7f2b1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:02:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.667+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38015070 con 0x564c11af4d20 2023-09-23T05:02:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.667+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f2b3801e420 con 0x564c11af4d20 2023-09-23T05:02:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.672+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b34113590 con 0x564c11af4d20 2023-09-23T05:02:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.672+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b38015e50 con 0x564c11af4d20 2023-09-23T05:02:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.673+0000 7f2b59de6b80 10 client.27725 fetch_fsmap learned FSMap version 74 2023-09-23T05:02:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.673+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x564c1185bc60 con 0x564c11af4d20 2023-09-23T05:02:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.682+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b34113590 con 0x564c11af4d20 2023-09-23T05:02:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.682+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 74) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2b380286b0 con 0x564c11af4d20 2023-09-23T05:02:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 10 client.27725 fetch_fsmap finished waiting for FSMap version 74 2023-09-23T05:02:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 10 client.27725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:02:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 20 client.27725 populate_metadata read hostname 'smithi005' 2023-09-23T05:02:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b63f80 0x564c11b663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:02:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 4 client.27725 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:02:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.687+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c11a40b30 con 0x564c11b63f80 2023-09-23T05:02:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.694+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b63f80 0x564c11b663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:02:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.698+0000 7f2b3ffff700 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b63f80 0x564c11b663a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9caa63667aebb5 server_cookie=9cd994cc2e0c150c in_seq=0 out_seq=0 2023-09-23T05:02:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.698+0000 7f2b1f7fe700 10 client.27725 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.698+0000 7f2b1f7fe700 1 -- 172.21.15.5:0/2142386407 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c11a40b30 con 0x564c11b63f80 2023-09-23T05:02:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.698+0000 7f2b1f7fe700 10 client.27725 handle_command_reply: tid=0 2023-09-23T05:02:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.698+0000 7f2b59de6b80 1 client.27725 shutdown 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 msgr2=0x564c11b048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b4007a230 tx=0x7f2b38015a20 comp rx=0 tx=0).stop 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 shutdown_connections 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564c11b63f80 0x564c11b663a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564c11affbd0 0x564c11b61ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c11af4400 0x564c11b043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 --2- 172.21.15.5:0/2142386407 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564c11af4d20 0x564c11b048e0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b4007a230 tx=0x7f2b38015a20 comp rx=0 tx=0).stop 2023-09-23T05:02:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.708+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 >> 172.21.15.5:0/2142386407 conn(0x564c11a69260 msgr2=0x564c11afcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.711+0000 7f2b1f7fe700 0 client.27725 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:02:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.712+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 shutdown_connections 2023-09-23T05:02:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.713+0000 7f2b59de6b80 1 -- 172.21.15.5:0/2142386407 wait complete. 2023-09-23T05:02:43.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.719+0000 7f2b59de6b80 20 client.27725 trim_cache size 0 max 16384 2023-09-23T05:02:43.727 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:02:43.727 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.726+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 msgr2=0x7f2b280a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.726+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 0x7f2b280a3de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b30003820 tx=0x7f2b3002a040 comp rx=0 tx=0).stop 2023-09-23T05:02:43.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.726+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 msgr2=0x7f2b4007d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:02:43.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.726+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2b34051960 tx=0x7f2b34051d20 comp rx=0 tx=0).stop 2023-09-23T05:02:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 shutdown_connections 2023-09-23T05:02:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f2b280a1920 0x7f2b280a3de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b4010f9d0 0x7f2b40083690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2b40107e20 0x7f2b4007d960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 --2- 172.21.15.5:0/1123433759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b4010f5f0 0x7f2b4007dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:02:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.733+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 >> 172.21.15.5:0/1123433759 conn(0x7f2b400725f0 msgr2=0x7f2b40109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:02:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.751+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 shutdown_connections 2023-09-23T05:02:43.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:02:43.756+0000 7f2b463ff700 1 -- 172.21.15.5:0/1123433759 wait complete. 2023-09-23T05:02:43.809 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:02:44.522 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113949 30.12 MB/sec execute 79 sec latency 209.998 ms 2023-09-23T05:02:45.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:45 smithi005 ceph-mon[97638]: pgmap v517: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 20 MiB/s wr, 755 op/s 2023-09-23T05:02:45.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:45 smithi118 ceph-mon[134297]: pgmap v517: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 20 MiB/s wr, 755 op/s 2023-09-23T05:02:45.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:45 smithi069 ceph-mon[133931]: pgmap v517: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 20 MiB/s wr, 755 op/s 2023-09-23T05:02:45.522 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114315 29.99 MB/sec execute 80 sec latency 161.027 ms 2023-09-23T05:02:46.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114997 30.06 MB/sec execute 81 sec latency 90.017 ms 2023-09-23T05:02:47.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:47 smithi005 ceph-mon[97638]: pgmap v518: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 650 op/s 2023-09-23T05:02:47.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:47 smithi118 ceph-mon[134297]: pgmap v518: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 650 op/s 2023-09-23T05:02:47.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:47 smithi069 ceph-mon[133931]: pgmap v518: 129 pgs: 129 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 650 op/s 2023-09-23T05:02:47.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 115909 30.33 MB/sec execute 82 sec latency 78.600 ms 2023-09-23T05:02:48.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 116363 30.23 MB/sec execute 83 sec latency 78.891 ms 2023-09-23T05:02:49.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:49 smithi005 ceph-mon[97638]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 20 MiB/s wr, 822 op/s 2023-09-23T05:02:49.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:49 smithi118 ceph-mon[134297]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 20 MiB/s wr, 822 op/s 2023-09-23T05:02:49.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:49 smithi069 ceph-mon[133931]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 20 MiB/s wr, 822 op/s 2023-09-23T05:02:49.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 117054 30.31 MB/sec execute 84 sec latency 122.317 ms 2023-09-23T05:02:50.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 117618 30.31 MB/sec execute 85 sec latency 76.424 ms 2023-09-23T05:02:51.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 118217 30.38 MB/sec execute 86 sec latency 123.160 ms 2023-09-23T05:02:52.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:51 smithi005 ceph-mon[97638]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 20 MiB/s wr, 841 op/s 2023-09-23T05:02:52.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:51 smithi118 ceph-mon[134297]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 20 MiB/s wr, 841 op/s 2023-09-23T05:02:52.240 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:51 smithi069 ceph-mon[133931]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 20 MiB/s wr, 841 op/s 2023-09-23T05:02:52.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 118928 30.49 MB/sec execute 87 sec latency 76.034 ms 2023-09-23T05:02:53.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:53 smithi005 ceph-mon[97638]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 808 op/s 2023-09-23T05:02:53.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:53 smithi118 ceph-mon[134297]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 808 op/s 2023-09-23T05:02:53.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:53 smithi069 ceph-mon[133931]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 808 op/s 2023-09-23T05:02:53.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 119649 30.54 MB/sec execute 88 sec latency 100.380 ms 2023-09-23T05:02:54.523 INFO:tasks.workunit.client.0.smithi005.stdout: 10 120325 30.64 MB/sec execute 89 sec latency 90.736 ms 2023-09-23T05:02:55.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:55 smithi005 ceph-mon[97638]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2023-09-23T05:02:55.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:55 smithi118 ceph-mon[134297]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2023-09-23T05:02:55.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:55 smithi069 ceph-mon[133931]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2023-09-23T05:02:55.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 121044 30.66 MB/sec execute 90 sec latency 61.058 ms 2023-09-23T05:02:56.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 121667 30.71 MB/sec execute 91 sec latency 228.916 ms 2023-09-23T05:02:57.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:57 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:57.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:57 smithi005 ceph-mon[97638]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-09-23T05:02:57.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:57 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:57.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:57 smithi118 ceph-mon[134297]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-09-23T05:02:57.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:57 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:02:57.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:57 smithi069 ceph-mon[133931]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-09-23T05:02:57.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 122322 30.77 MB/sec execute 92 sec latency 79.792 ms 2023-09-23T05:02:58.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 123167 31.02 MB/sec execute 93 sec latency 45.346 ms 2023-09-23T05:02:59.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:02:59 smithi005 ceph-mon[97638]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 700 op/s 2023-09-23T05:02:59.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:02:59 smithi118 ceph-mon[134297]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 700 op/s 2023-09-23T05:02:59.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:02:59 smithi069 ceph-mon[133931]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 700 op/s 2023-09-23T05:02:59.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 123840 31.05 MB/sec execute 94 sec latency 84.656 ms 2023-09-23T05:03:00.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124412 31.02 MB/sec execute 95 sec latency 81.565 ms 2023-09-23T05:03:01.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124585 30.79 MB/sec execute 96 sec latency 750.153 ms 2023-09-23T05:03:02.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:01 smithi005 ceph-mon[97638]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-23T05:03:02.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:01 smithi118 ceph-mon[134297]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-23T05:03:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:01 smithi069 ceph-mon[133931]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-23T05:03:02.524 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124585 30.47 MB/sec execute 97 sec latency 1750.203 ms 2023-09-23T05:03:03.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:02 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:03:03.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:02 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:03:03.420 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:02 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:03:03.526 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124598 30.16 MB/sec execute 98 sec latency 2438.283 ms 2023-09-23T05:03:04.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:03 smithi069 ceph-mon[133931]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.7 MiB/s wr, 498 op/s 2023-09-23T05:03:04.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:03 smithi005 ceph-mon[97638]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.7 MiB/s wr, 498 op/s 2023-09-23T05:03:04.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:03 smithi118 ceph-mon[134297]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.7 MiB/s wr, 498 op/s 2023-09-23T05:03:04.526 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124666 29.88 MB/sec execute 99 sec latency 1218.776 ms 2023-09-23T05:03:05.527 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124961 29.69 MB/sec execute 100 sec latency 193.242 ms 2023-09-23T05:03:06.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:05 smithi069 ceph-mon[133931]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 536 op/s 2023-09-23T05:03:06.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:05 smithi005 ceph-mon[97638]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 536 op/s 2023-09-23T05:03:06.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:05 smithi118 ceph-mon[134297]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.8 MiB/s wr, 536 op/s 2023-09-23T05:03:06.528 INFO:tasks.workunit.client.0.smithi005.stdout: 10 125223 29.62 MB/sec execute 101 sec latency 201.347 ms 2023-09-23T05:03:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:06 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:07.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:06 smithi069 ceph-mon[133931]: osdmap e113: 12 total, 12 up, 12 in 2023-09-23T05:03:07.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:06 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:07.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:06 smithi118 ceph-mon[134297]: osdmap e113: 12 total, 12 up, 12 in 2023-09-23T05:03:07.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:06 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:07.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:06 smithi005 ceph-mon[97638]: osdmap e113: 12 total, 12 up, 12 in 2023-09-23T05:03:07.703 INFO:tasks.workunit.client.0.smithi005.stdout: 10 125710 29.61 MB/sec execute 102 sec latency 172.278 ms 2023-09-23T05:03:08.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:07 smithi069 ceph-mon[133931]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 510 op/s 2023-09-23T05:03:08.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:07 smithi005 ceph-mon[97638]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 510 op/s 2023-09-23T05:03:08.416 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:07 smithi118 ceph-mon[134297]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 510 op/s 2023-09-23T05:03:08.530 INFO:tasks.workunit.client.0.smithi005.stdout: 10 126109 29.50 MB/sec execute 103 sec latency 239.836 ms 2023-09-23T05:03:09.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:09 smithi005 ceph-mon[97638]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:03:09.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:09 smithi118 ceph-mon[134297]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:03:09.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:09 smithi069 ceph-mon[133931]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:03:09.534 INFO:tasks.workunit.client.0.smithi005.stdout: 10 126630 29.43 MB/sec execute 104 sec latency 118.735 ms 2023-09-23T05:03:10.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:10 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:03:10.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:10 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:03:10.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:10 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:03:10.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:10 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:03:10.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:10 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:03:10.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:10 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:03:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:10 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:03:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:10 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:03:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:10 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:03:10.534 INFO:tasks.workunit.client.0.smithi005.stdout: 10 127327 29.55 MB/sec execute 105 sec latency 239.348 ms 2023-09-23T05:03:11.534 INFO:tasks.workunit.client.0.smithi005.stdout: 10 127928 29.60 MB/sec execute 106 sec latency 1239.415 ms 2023-09-23T05:03:12.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:11 smithi005 ceph-mon[97638]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:03:12.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:11 smithi118 ceph-mon[134297]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:03:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:11 smithi069 ceph-mon[133931]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 681 op/s 2023-09-23T05:03:12.534 INFO:tasks.workunit.client.0.smithi005.stdout: 10 128416 29.62 MB/sec execute 107 sec latency 2239.464 ms 2023-09-23T05:03:13.536 INFO:tasks.workunit.client.0.smithi005.stdout: 10 128709 29.47 MB/sec execute 108 sec latency 3239.798 ms 2023-09-23T05:03:13.810 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:03:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.162+0000 7f6a96bff700 1 -- 172.21.15.5:0/1036927491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a901083f0 msgr2=0x7f6a9010ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.162+0000 7f6a96bff700 1 --2- 172.21.15.5:0/1036927491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a901083f0 0x7f6a9010ff70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009f90 tx=0x7f6a8800b330 comp rx=0 tx=0).stop 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.163+0000 7f6a96bff700 1 -- 172.21.15.5:0/1036927491 shutdown_connections 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.163+0000 7f6a96bff700 1 --2- 172.21.15.5:0/1036927491 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a901083f0 0x7f6a9010ff70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.163+0000 7f6a96bff700 1 --2- 172.21.15.5:0/1036927491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a901032d0 0x7f6a901036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.163+0000 7f6a96bff700 1 --2- 172.21.15.5:0/1036927491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a90103c80 0x7f6a90107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.163+0000 7f6a96bff700 1 -- 172.21.15.5:0/1036927491 >> 172.21.15.5:0/1036927491 conn(0x7f6a900feb50 msgr2=0x7f6a90100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.166+0000 7f6a96bff700 1 -- 172.21.15.5:0/1036927491 shutdown_connections 2023-09-23T05:03:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.166+0000 7f6a96bff700 1 -- 172.21.15.5:0/1036927491 wait complete. 2023-09-23T05:03:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.167+0000 7f6a96bff700 1 Processor -- start 2023-09-23T05:03:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.167+0000 7f6a96bff700 1 -- start start 2023-09-23T05:03:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a901083f0 0x7f6a9010cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9010cff0 0x7f6a9010d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901102a0 con 0x7f6a901083f0 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019f6e0 con 0x7f6a901032d0 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a96bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019f8a0 con 0x7f6a9010cff0 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a9499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a9499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55014/0 (socket says 172.21.15.5:55014) 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a9499b700 1 -- 172.21.15.5:0/3046703284 learned_addr learned my addr 172.21.15.5:0/3046703284 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.168+0000 7f6a9519c700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9010cff0 0x7f6a9010d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 -- 172.21.15.5:0/3046703284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9010cff0 msgr2=0x7f6a9010d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9010cff0 0x7f6a9010d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 -- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a901083f0 msgr2=0x7f6a9010cab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a901083f0 0x7f6a9010cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 -- 172.21.15.5:0/3046703284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84009520 con 0x7f6a901032d0 2023-09-23T05:03:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.169+0000 7f6a9499b700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400eb30 tx=0x7f6a8400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.170+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3046703284 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84009880 con 0x7f6a901032d0 2023-09-23T05:03:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.170+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a88009c40 con 0x7f6a901032d0 2023-09-23T05:03:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.170+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9019ff50 con 0x7f6a901032d0 2023-09-23T05:03:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.170+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3046703284 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a840044e0 con 0x7f6a901032d0 2023-09-23T05:03:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.170+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3046703284 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84010430 con 0x7f6a901032d0 2023-09-23T05:03:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.176+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3046703284 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f6a84004020 con 0x7f6a901032d0 2023-09-23T05:03:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.178+0000 7f6aaa5a3b80 1 Processor -- start 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.178+0000 7f6aaa5a3b80 1 -- start start 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a49805690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a49807fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a49763460 con 0x564a497fad20 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a49754c20 con 0x564a497fa400 2023-09-23T05:03:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a49752540 con 0x564a49805bd0 2023-09-23T05:03:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55016/0 (socket says 172.21.15.5:55016) 2023-09-23T05:03:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.179+0000 7f6a9519c700 1 -- 172.21.15.5:0/572933370 learned_addr learned my addr 172.21.15.5:0/572933370 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9499b700 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a49805690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a8dffb700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 0x7f6a800a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a49807fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9519c700 1 -- 172.21.15.5:0/572933370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 msgr2=0x564a49807fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9519c700 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a49807fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9519c700 1 -- 172.21.15.5:0/572933370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 msgr2=0x564a49805690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9519c700 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a49805690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a9519c700 1 -- 172.21.15.5:0/572933370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a497b08d0 con 0x564a497fa400 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.180+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3046703284 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f6a84014070 con 0x7f6a901032d0 2023-09-23T05:03:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.182+0000 7f6a9519c700 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009320 tx=0x7f6a88009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdde942cc82cabf5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.182+0000 7f6a9519c700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 0x7f6a800a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.182+0000 7f6a7affd700 1 -- 172.21.15.5:0/572933370 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800e070 con 0x564a497fa400 2023-09-23T05:03:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.183+0000 7f6a7affd700 1 -- 172.21.15.5:0/572933370 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a88007e40 con 0x564a497fa400 2023-09-23T05:03:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.183+0000 7f6a7affd700 1 -- 172.21.15.5:0/572933370 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880052a0 con 0x564a497fa400 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/572933370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 msgr2=0x564a497fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009320 tx=0x7f6a88009350 comp rx=0 tx=0).stop 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6a9519c700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 0x7f6a800a3cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a88026000 tx=0x7f6a880055f0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/572933370 shutdown_connections 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a49807fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a497fa7e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/572933370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a49805690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/572933370 >> 172.21.15.5:0/572933370 conn(0x564a4976f260 msgr2=0x564a497f6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/572933370 shutdown_connections 2023-09-23T05:03:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.184+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/572933370 wait complete. 2023-09-23T05:03:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.185+0000 7f6aaa5a3b80 1 -- start start 2023-09-23T05:03:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.185+0000 7f6aaa5a3b80 1 Processor -- start 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fa400 0x564a49805650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a498028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4974de30 con 0x564a497fad20 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a496ea710 con 0x564a49805bd0 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a496ea3b0 con 0x564a497fa400 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.186+0000 7f6aaa5a3b80 10 client.0 _ll_register_callbacks cb 0x564a49547800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:03:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.187+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.187+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55032/0 (socket says 172.21.15.5:55032) 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.187+0000 7f6a9519c700 1 -- 172.21.15.5:0/1792479450 learned_addr learned my addr 172.21.15.5:0/1792479450 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.187+0000 7f6a9499b700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fa400 0x564a49805650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.187+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a498028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 -- 172.21.15.5:0/1792479450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fa400 msgr2=0x564a49805650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fa400 0x564a49805650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 -- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 msgr2=0x564a498028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a498028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 -- 172.21.15.5:0/1792479450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a901107e0 con 0x564a49805bd0 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a498028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800bda0 tx=0x7f6a8801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1f900b1bd95ca1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.188+0000 7f6a6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.189+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88005210 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.189+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a497ee250 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.189+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f6a88005370 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.189+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8800b990 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 10 client.27737 fetch_fsmap learned FSMap version 76 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x564a49578de0 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a497ee250 con 0x564a49805bd0 2023-09-23T05:03:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6a880397e0 con 0x564a49805bd0 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 10 client.27737 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 10 client.27737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 20 client.27737 populate_metadata read hostname 'smithi005' 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a49815520 0x564a49817940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 4 client.27737 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.190+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a49746e10 con 0x564a49815520 2023-09-23T05:03:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.191+0000 7f6a9499b700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a49815520 0x564a49817940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.192+0000 7f6a9499b700 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a49815520 0x564a49817940 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81d647542e9f483 server_cookie=fb9f26c62cdc9d1e in_seq=0 out_seq=0 2023-09-23T05:03:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.192+0000 7f6a6ffff700 10 client.27737 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6a6ffff700 1 -- 172.21.15.5:0/1792479450 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a49746e10 con 0x564a49815520 2023-09-23T05:03:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6a6ffff700 10 client.27737 handle_command_reply: tid=0 2023-09-23T05:03:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 client.27737 shutdown 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 msgr2=0x564a49802fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800bda0 tx=0x7f6a8801e000 comp rx=0 tx=0).stop 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 shutdown_connections 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a49815520 0x564a49817940 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a49805bd0 0x564a49802fc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fa400 0x564a49805650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6a6ffff700 0 client.27737 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1792479450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fad20 0x564a498028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.193+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 >> 172.21.15.5:0/1792479450 conn(0x564a4976f260 msgr2=0x564a497f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.194+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 shutdown_connections 2023-09-23T05:03:14.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.194+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1792479450 wait complete. 2023-09-23T05:03:14.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.194+0000 7f6aaa5a3b80 20 client.27737 trim_cache size 0 max 16384 2023-09-23T05:03:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.228+0000 7f6aaa5a3b80 1 Processor -- start 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.228+0000 7f6aaa5a3b80 1 -- start start 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fa400 0x564a4980c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 0x564a497fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a495ce920 con 0x564a497fa400 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a497f2d60 con 0x564a497fad20 2023-09-23T05:03:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a49803cc0 con 0x564a49805bd0 2023-09-23T05:03:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.229+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40056/0 (socket says 172.21.15.5:40056) 2023-09-23T05:03:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 -- 172.21.15.5:0/1169901964 learned_addr learned my addr 172.21.15.5:0/1169901964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 0x564a497fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 -- 172.21.15.5:0/1169901964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 msgr2=0x564a497fdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 0x564a497fdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 -- 172.21.15.5:0/1169901964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fa400 msgr2=0x564a4980c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fa400 0x564a4980c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a9519c700 1 -- 172.21.15.5:0/1169901964 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a49561c60 con 0x564a49805bd0 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.230+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 0x564a497fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.231+0000 7f6a9519c700 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800b370 tx=0x7f6a88009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11a76f2dc87d029 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.231+0000 7f6a7b7fe700 1 -- 172.21.15.5:0/1169901964 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800e070 con 0x564a49805bd0 2023-09-23T05:03:14.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.231+0000 7f6a7b7fe700 1 -- 172.21.15.5:0/1169901964 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a880092e0 con 0x564a49805bd0 2023-09-23T05:03:14.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.231+0000 7f6a7b7fe700 1 -- 172.21.15.5:0/1169901964 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880396d0 con 0x564a49805bd0 2023-09-23T05:03:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1169901964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 msgr2=0x564a497fe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800b370 tx=0x7f6a88009f50 comp rx=0 tx=0).stop 2023-09-23T05:03:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1169901964 shutdown_connections 2023-09-23T05:03:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a49805bd0 0x564a497fe170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a497fa400 0x564a4980c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/1169901964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fad20 0x564a497fdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.232+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1169901964 >> 172.21.15.5:0/1169901964 conn(0x564a4976f260 msgr2=0x564a49810ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.235+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1169901964 shutdown_connections 2023-09-23T05:03:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.235+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/1169901964 wait complete. 2023-09-23T05:03:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.236+0000 7f6aaa5a3b80 1 -- start start 2023-09-23T05:03:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.236+0000 7f6aaa5a3b80 1 Processor -- start 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fad20 0x564a49864320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a49805bd0 0x564a49864860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4980c9f0 con 0x564a49805bd0 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4980cb60 con 0x564a497fa400 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4980cd20 con 0x564a497fad20 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.237+0000 7f6aaa5a3b80 10 client.0 _ll_register_callbacks cb 0x7f6a84005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:03:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:55048/0 (socket says 172.21.15.5:55048) 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 -- 172.21.15.5:0/4104249778 learned_addr learned my addr 172.21.15.5:0/4104249778 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fad20 0x564a49864320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 -- 172.21.15.5:0/4104249778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fad20 msgr2=0x564a49864320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fad20 0x564a49864320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 -- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a49805bd0 msgr2=0x564a49864860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a49805bd0 0x564a49864860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.238+0000 7f6a9499b700 1 -- 172.21.15.5:0/4104249778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a49578de0 con 0x564a497fa400 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.239+0000 7f6a9499b700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a840ba160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abaf3ff44e6515d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.239+0000 7f6a6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:03:14.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:13 smithi069 ceph-mon[133931]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 774 op/s 2023-09-23T05:03:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.240+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8409a6b0 con 0x564a497fa400 2023-09-23T05:03:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.240+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f6a8409e800 con 0x564a497fa400 2023-09-23T05:03:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.240+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a49808450 con 0x564a497fa400 2023-09-23T05:03:14.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.243+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a840a2a40 con 0x564a497fa400 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.243+0000 7f6aaa5a3b80 10 client.27740 fetch_fsmap learned FSMap version 76 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.244+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x564a4959f250 con 0x564a497fa400 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.244+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a49808450 con 0x564a497fa400 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.244+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 76) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6a84003680 con 0x564a497fa400 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 10 client.27740 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 10 client.27740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:03:14.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 20 client.27740 populate_metadata read hostname 'smithi005' 2023-09-23T05:03:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a4980b780 0x564a4980bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 4 client.27740 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:03:14.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.245+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f6a88007e40 con 0x564a4980b780 2023-09-23T05:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.270+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a4980b780 0x564a4980bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:14.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.320+0000 7f6a8ffff700 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a4980b780 0x564a4980bb90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f043daa22ecde6 server_cookie=6ea7cfdbdbe16358 in_seq=0 out_seq=0 2023-09-23T05:03:14.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.324+0000 7f6a6ffff700 10 client.27740 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:14.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.330+0000 7f6a6ffff700 1 -- 172.21.15.5:0/4104249778 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f6a88007e40 con 0x564a4980b780 2023-09-23T05:03:14.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.330+0000 7f6a6ffff700 10 client.27740 handle_command_reply: tid=0 2023-09-23T05:03:14.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.330+0000 7f6aaa5a3b80 1 client.27740 shutdown 2023-09-23T05:03:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.341+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 msgr2=0x564a49800190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.341+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a840ba160 comp rx=0 tx=0).stop 2023-09-23T05:03:14.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.346+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 shutdown_connections 2023-09-23T05:03:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.346+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564a4980b780 0x564a4980bb90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.346+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a49805bd0 0x564a49864860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.346+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564a497fa400 0x564a49800190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.346+0000 7f6aaa5a3b80 1 --2- 172.21.15.5:0/4104249778 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564a497fad20 0x564a49864320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.347+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 >> 172.21.15.5:0/4104249778 conn(0x564a4976f260 msgr2=0x564a497fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.350+0000 7f6a6ffff700 0 client.27740 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:14.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.350+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 shutdown_connections 2023-09-23T05:03:14.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.350+0000 7f6aaa5a3b80 1 -- 172.21.15.5:0/4104249778 wait complete. 2023-09-23T05:03:14.361 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:03:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:03:14.361 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:03:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.360+0000 7f6aaa5a3b80 20 client.27740 trim_cache size 0 max 16384 2023-09-23T05:03:14.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.365+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 msgr2=0x7f6a800a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.365+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 0x7f6a800a3cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a88026000 tx=0x7f6a880055f0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.365+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 msgr2=0x7f6a9010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:14.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.365+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400eb30 tx=0x7f6a8400eef0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 shutdown_connections 2023-09-23T05:03:14.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f6a800a1810 0x7f6a800a3cd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f6a9010cff0 0x7f6a9010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a901032d0 0x7f6a9010faa0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 --2- 172.21.15.5:0/3046703284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a901083f0 0x7f6a9010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:14.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.369+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 >> 172.21.15.5:0/3046703284 conn(0x7f6a900feb50 msgr2=0x7f6a90100600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.372+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 shutdown_connections 2023-09-23T05:03:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:14.373+0000 7f6a96bff700 1 -- 172.21.15.5:0/3046703284 wait complete. 2023-09-23T05:03:14.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:13 smithi005 ceph-mon[97638]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 774 op/s 2023-09-23T05:03:14.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:13 smithi118 ceph-mon[134297]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 774 op/s 2023-09-23T05:03:14.429 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:03:14.535 INFO:tasks.workunit.client.0.smithi005.stdout: 10 128937 29.27 MB/sec execute 109 sec latency 4240.343 ms 2023-09-23T05:03:15.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:15 smithi005 ceph-mon[97638]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:15.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:15 smithi118 ceph-mon[134297]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:15.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:15 smithi069 ceph-mon[133931]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:15.535 INFO:tasks.workunit.client.0.smithi005.stdout: 10 129399 29.22 MB/sec execute 110 sec latency 5240.407 ms 2023-09-23T05:03:16.535 INFO:tasks.workunit.client.0.smithi005.stdout: 10 129838 29.16 MB/sec execute 111 sec latency 6240.467 ms 2023-09-23T05:03:17.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:16 smithi005 ceph-mon[97638]: osdmap e114: 12 total, 12 up, 12 in 2023-09-23T05:03:17.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:16 smithi118 ceph-mon[134297]: osdmap e114: 12 total, 12 up, 12 in 2023-09-23T05:03:17.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:16 smithi069 ceph-mon[133931]: osdmap e114: 12 total, 12 up, 12 in 2023-09-23T05:03:17.536 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130314 29.17 MB/sec execute 112 sec latency 6688.660 ms 2023-09-23T05:03:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:17 smithi005 ceph-mon[97638]: pgmap v535: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:18.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:17 smithi118 ceph-mon[134297]: pgmap v535: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:18.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:17 smithi069 ceph-mon[133931]: pgmap v535: 129 pgs: 129 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 15 MiB/s wr, 711 op/s 2023-09-23T05:03:18.536 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130587 29.03 MB/sec execute 113 sec latency 6907.814 ms 2023-09-23T05:03:19.536 INFO:tasks.workunit.client.0.smithi005.stdout: 10 131008 28.99 MB/sec execute 114 sec latency 7907.875 ms 2023-09-23T05:03:20.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:19 smithi069 ceph-mon[133931]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 569 op/s 2023-09-23T05:03:20.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:19 smithi005 ceph-mon[97638]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 569 op/s 2023-09-23T05:03:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:19 smithi118 ceph-mon[134297]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 569 op/s 2023-09-23T05:03:20.537 INFO:tasks.workunit.client.0.smithi005.stdout: 10 131414 28.87 MB/sec execute 115 sec latency 8908.283 ms 2023-09-23T05:03:21.537 INFO:tasks.workunit.client.0.smithi005.stdout: 10 132008 28.92 MB/sec execute 116 sec latency 9908.352 ms 2023-09-23T05:03:22.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:21 smithi005 ceph-mon[97638]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 542 op/s 2023-09-23T05:03:22.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:21 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:21 smithi118 ceph-mon[134297]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 542 op/s 2023-09-23T05:03:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:21 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:21 smithi069 ceph-mon[133931]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 542 op/s 2023-09-23T05:03:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:21 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:03:22.538 INFO:tasks.workunit.client.0.smithi005.stdout: 10 132601 28.96 MB/sec execute 117 sec latency 10705.254 ms 2023-09-23T05:03:23.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:23 smithi005 ceph-mon[97638]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 497 op/s 2023-09-23T05:03:23.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:23 smithi118 ceph-mon[134297]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 497 op/s 2023-09-23T05:03:23.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:23 smithi069 ceph-mon[133931]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 497 op/s 2023-09-23T05:03:23.539 INFO:tasks.workunit.client.0.smithi005.stdout: 10 133162 28.98 MB/sec execute 118 sec latency 71.151 ms 2023-09-23T05:03:24.539 INFO:tasks.workunit.client.0.smithi005.stdout: 10 133699 28.97 MB/sec execute 119 sec latency 103.854 ms 2023-09-23T05:03:25.539 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134367 29.01 MB/sec execute 120 sec latency 83.783 ms 2023-09-23T05:03:26.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:25 smithi069 ceph-mon[133931]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 513 op/s 2023-09-23T05:03:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:25 smithi005 ceph-mon[97638]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 513 op/s 2023-09-23T05:03:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:25 smithi118 ceph-mon[134297]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 513 op/s 2023-09-23T05:03:26.539 INFO:tasks.workunit.client.0.smithi005.stdout: 10 135057 29.05 MB/sec execute 121 sec latency 76.532 ms 2023-09-23T05:03:27.539 INFO:tasks.workunit.client.0.smithi005.stdout: 10 135818 29.21 MB/sec execute 122 sec latency 70.213 ms 2023-09-23T05:03:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:27 smithi005 ceph-mon[97638]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2023-09-23T05:03:28.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:27 smithi118 ceph-mon[134297]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2023-09-23T05:03:28.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:27 smithi069 ceph-mon[133931]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 462 op/s 2023-09-23T05:03:28.540 INFO:tasks.workunit.client.0.smithi005.stdout: 10 136308 29.17 MB/sec execute 123 sec latency 132.914 ms 2023-09-23T05:03:29.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:28 smithi005 ceph-mon[97638]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 549 op/s 2023-09-23T05:03:29.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:29 smithi118 ceph-mon[134297]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 549 op/s 2023-09-23T05:03:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:29 smithi069 ceph-mon[133931]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 549 op/s 2023-09-23T05:03:29.540 INFO:tasks.workunit.client.0.smithi005.stdout: 10 136939 29.25 MB/sec execute 124 sec latency 103.492 ms 2023-09-23T05:03:30.540 INFO:tasks.workunit.client.0.smithi005.stdout: 10 137425 29.19 MB/sec execute 125 sec latency 90.855 ms 2023-09-23T05:03:31.541 INFO:tasks.workunit.client.0.smithi005.stdout: 10 137946 29.15 MB/sec execute 126 sec latency 114.031 ms 2023-09-23T05:03:32.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:31 smithi005 ceph-mon[97638]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-09-23T05:03:32.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:31 smithi118 ceph-mon[134297]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-09-23T05:03:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:31 smithi069 ceph-mon[133931]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-09-23T05:03:32.541 INFO:tasks.workunit.client.0.smithi005.stdout: 10 138588 29.23 MB/sec execute 127 sec latency 70.070 ms 2023-09-23T05:03:33.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:33 smithi005 ceph-mon[97638]: pgmap v543: 129 pgs: 129 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 548 op/s 2023-09-23T05:03:33.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:33 smithi118 ceph-mon[134297]: pgmap v543: 129 pgs: 129 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 548 op/s 2023-09-23T05:03:33.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:33 smithi069 ceph-mon[133931]: pgmap v543: 129 pgs: 129 active+clean; 1.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 548 op/s 2023-09-23T05:03:33.541 INFO:tasks.workunit.client.0.smithi005.stdout: 10 139205 29.19 MB/sec execute 128 sec latency 95.875 ms 2023-09-23T05:03:34.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 139942 29.34 MB/sec execute 129 sec latency 81.705 ms 2023-09-23T05:03:35.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 140369 29.29 MB/sec execute 130 sec latency 120.154 ms 2023-09-23T05:03:36.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:36 smithi005 ceph-mon[97638]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2023-09-23T05:03:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:36 smithi118 ceph-mon[134297]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2023-09-23T05:03:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:36 smithi069 ceph-mon[133931]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2023-09-23T05:03:36.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 140997 29.32 MB/sec execute 131 sec latency 115.744 ms 2023-09-23T05:03:37.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:37 smithi005 ceph-mon[97638]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-09-23T05:03:37.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:37 smithi118 ceph-mon[134297]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-09-23T05:03:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:37 smithi069 ceph-mon[133931]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 437 op/s 2023-09-23T05:03:37.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 141688 29.37 MB/sec execute 132 sec latency 64.465 ms 2023-09-23T05:03:38.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 142188 29.31 MB/sec execute 133 sec latency 325.139 ms 2023-09-23T05:03:39.542 INFO:tasks.workunit.client.0.smithi005.stdout: 10 142787 29.40 MB/sec execute 134 sec latency 161.900 ms 2023-09-23T05:03:40.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:39 smithi118 ceph-mon[134297]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.5 MiB/s wr, 556 op/s 2023-09-23T05:03:40.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:39 smithi005 ceph-mon[97638]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.5 MiB/s wr, 556 op/s 2023-09-23T05:03:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:39 smithi069 ceph-mon[133931]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.5 MiB/s wr, 556 op/s 2023-09-23T05:03:40.543 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143185 29.33 MB/sec execute 135 sec latency 451.550 ms 2023-09-23T05:03:41.544 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143242 29.14 MB/sec execute 136 sec latency 1381.281 ms 2023-09-23T05:03:42.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:41 smithi005 ceph-mon[97638]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2023-09-23T05:03:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:41 smithi118 ceph-mon[134297]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2023-09-23T05:03:42.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:41 smithi069 ceph-mon[133931]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 523 op/s 2023-09-23T05:03:42.545 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143478 29.01 MB/sec execute 137 sec latency 1486.436 ms 2023-09-23T05:03:43.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:43 smithi005 ceph-mon[97638]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 590 op/s 2023-09-23T05:03:43.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:43 smithi118 ceph-mon[134297]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 590 op/s 2023-09-23T05:03:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:43 smithi069 ceph-mon[133931]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 590 op/s 2023-09-23T05:03:43.547 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143696 28.87 MB/sec execute 138 sec latency 324.060 ms 2023-09-23T05:03:44.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:03:44.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 144054 28.80 MB/sec execute 139 sec latency 126.807 ms 2023-09-23T05:03:45.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:45 smithi005 ceph-mon[97638]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 620 op/s 2023-09-23T05:03:45.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:45 smithi118 ceph-mon[134297]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 620 op/s 2023-09-23T05:03:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:45 smithi069 ceph-mon[133931]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 620 op/s 2023-09-23T05:03:45.548 INFO:tasks.workunit.client.0.smithi005.stdout: 10 144674 28.88 MB/sec execute 140 sec latency 89.422 ms 2023-09-23T05:03:46.549 INFO:tasks.workunit.client.0.smithi005.stdout: 10 144778 28.69 MB/sec execute 141 sec latency 566.856 ms 2023-09-23T05:03:46.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.570+0000 7f3ef8bff700 1 -- 172.21.15.5:0/338744666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef4100990 msgr2=0x7f3ef41049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.570+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/338744666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef4100990 0x7f3ef41049d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009990 tx=0x7f3ee8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 -- 172.21.15.5:0/338744666 shutdown_connections 2023-09-23T05:03:46.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/338744666 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef4105100 0x7f3ef410cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/338744666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef40fffe0 0x7f3ef41003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/338744666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef4100990 0x7f3ef41049d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 -- 172.21.15.5:0/338744666 >> 172.21.15.5:0/338744666 conn(0x7f3ef4076f30 msgr2=0x7f3ef4077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.575+0000 7f3ef8bff700 1 -- 172.21.15.5:0/338744666 shutdown_connections 2023-09-23T05:03:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.578+0000 7f3ef8bff700 1 -- 172.21.15.5:0/338744666 wait complete. 2023-09-23T05:03:46.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.579+0000 7f3ef8bff700 1 Processor -- start 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.579+0000 7f3ef8bff700 1 -- start start 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef40fffe0 0x7f3ef406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef4105100 0x7f3ef406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41a5e40 con 0x7f3ef40fffe0 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41a5fb0 con 0x7f3ef4105100 2023-09-23T05:03:46.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef8bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41a6170 con 0x7f3ef41a53c0 2023-09-23T05:03:46.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.580+0000 7f3ef2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42208/0 (socket says 172.21.15.5:42208) 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 -- 172.21.15.5:0/1967748744 learned_addr learned my addr 172.21.15.5:0/1967748744 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef4105100 0x7f3ef406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 -- 172.21.15.5:0/1967748744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef4105100 msgr2=0x7f3ef406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef4105100 0x7f3ef406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 -- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef40fffe0 msgr2=0x7f3ef406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef40fffe0 0x7f3ef406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.581+0000 7f3ef2d9d700 1 -- 172.21.15.5:0/1967748744 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80095f0 con 0x7f3ef41a53c0 2023-09-23T05:03:46.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ef2d9d700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ada0 tx=0x7f3eec00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ee37fe700 1 -- 172.21.15.5:0/1967748744 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00f070 con 0x7f3ef41a53c0 2023-09-23T05:03:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ee37fe700 1 -- 172.21.15.5:0/1967748744 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eec007b30 con 0x7f3ef41a53c0 2023-09-23T05:03:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4106cb0 con 0x7f3ef41a53c0 2023-09-23T05:03:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ee37fe700 1 -- 172.21.15.5:0/1967748744 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0053e0 con 0x7f3ef41a53c0 2023-09-23T05:03:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.582+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef41071c0 con 0x7f3ef41a53c0 2023-09-23T05:03:46.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.591+0000 7f3f0c47fb80 1 Processor -- start 2023-09-23T05:03:46.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.591+0000 7f3ee37fe700 1 -- 172.21.15.5:0/1967748744 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3eec00c980 con 0x7f3ef41a53c0 2023-09-23T05:03:46.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.592+0000 7f3f0c47fb80 1 -- start start 2023-09-23T05:03:46.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5d20 0x56310ffd0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ff2e460 con 0x56310ffd0bd0 2023-09-23T05:03:46.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ff1fc20 con 0x56310ffc5400 2023-09-23T05:03:46.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.594+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ff1d540 con 0x56310ffc5d20 2023-09-23T05:03:46.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.596+0000 7f3ee37fe700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 0x7f3edc0a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.596+0000 7f3ee37fe700 1 -- 172.21.15.5:0/1967748744 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3eec0ccc40 con 0x7f3ef41a53c0 2023-09-23T05:03:46.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.596+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.596+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:38718/0 (socket says 172.21.15.5:38718) 2023-09-23T05:03:46.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.596+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3712673134 learned_addr learned my addr 172.21.15.5:0/3712673134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3712673134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5d20 msgr2=0x56310ffd0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5d20 0x56310ffd0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3712673134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 msgr2=0x56310ffd2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.597+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3712673134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56310ff7b8d0 con 0x56310ffc5400 2023-09-23T05:03:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.598+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009990 tx=0x7f3ee801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b41814506437ec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.598+0000 7f3ef2d9d700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 0x7f3edc0a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.599+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/3712673134 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80050f0 con 0x56310ffc5400 2023-09-23T05:03:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.599+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/3712673134 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee802b070 con 0x56310ffc5400 2023-09-23T05:03:46.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.599+0000 7f3ee0ff9700 1 -- 172.21.15.5:0/3712673134 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800bb90 con 0x56310ffc5400 2023-09-23T05:03:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.600+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3712673134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 msgr2=0x56310ffc57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.600+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009990 tx=0x7f3ee801a040 comp rx=0 tx=0).stop 2023-09-23T05:03:46.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.601+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3712673134 shutdown_connections 2023-09-23T05:03:46.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.601+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.601+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5400 0x56310ffc57e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.601+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3712673134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5d20 0x56310ffd0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.601+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3712673134 >> 172.21.15.5:0/3712673134 conn(0x56310ff3a260 msgr2=0x56310ffc1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.602+0000 7f3ef2d9d700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 0x7f3edc0a3c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3eec095d10 tx=0x7f3eec004cc0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.602+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3712673134 shutdown_connections 2023-09-23T05:03:46.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.604+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3712673134 wait complete. 2023-09-23T05:03:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.604+0000 7f3f0c47fb80 1 -- start start 2023-09-23T05:03:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.605+0000 7f3f0c47fb80 1 Processor -- start 2023-09-23T05:03:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.605+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.605+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffcdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ff18e30 con 0x56310ffd0bd0 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310feb5710 con 0x56310ffc5d20 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310feb53b0 con 0x56310ffc5400 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3f0c47fb80 10 client.0 _ll_register_callbacks cb 0x56310fd12800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:03:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.606+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.607+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42216/0 (socket says 172.21.15.5:42216) 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.607+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3336437770 learned_addr learned my addr 172.21.15.5:0/3336437770 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.608+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3336437770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 msgr2=0x56310ffcdea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.608+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffcdea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.608+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 msgr2=0x56310ffce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.608+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.608+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/3336437770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee802d040 con 0x56310ffc5400 2023-09-23T05:03:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.609+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005850 tx=0x7f3ee8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c40925bb597f60 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.609+0000 7f3ed9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.609+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800ba20 con 0x56310ffc5400 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.609+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3ee8025850 con 0x56310ffc5400 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.610+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56310ffb9250 con 0x56310ffc5400 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.611+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee800bb80 con 0x56310ffc5400 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.611+0000 7f3f0c47fb80 10 client.27787 fetch_fsmap learned FSMap version 77 2023-09-23T05:03:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.611+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x56310fd43de0 con 0x56310ffc5400 2023-09-23T05:03:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56310ffb9250 con 0x56310ffc5400 2023-09-23T05:03:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 77) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3ee8004060 con 0x56310ffc5400 2023-09-23T05:03:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 10 client.27787 fetch_fsmap finished waiting for FSMap version 77 2023-09-23T05:03:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 10 client.27787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:03:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 20 client.27787 populate_metadata read hostname 'smithi005' 2023-09-23T05:03:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56310ffe0140 0x56310ffe2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 4 client.27787 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:03:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.612+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56310ff11e10 con 0x56310ffe0140 2023-09-23T05:03:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.614+0000 7f3ef259c700 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56310ffe0140 0x56310ffe2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.615+0000 7f3ef259c700 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56310ffe0140 0x56310ffe2560 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4d4b69bbef6953 server_cookie=e792d6d31463d1aa in_seq=0 out_seq=0 2023-09-23T05:03:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.615+0000 7f3ed9ffb700 10 client.27787 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.623+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3336437770 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56310ff11e10 con 0x56310ffe0140 2023-09-23T05:03:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.623+0000 7f3ed9ffb700 10 client.27787 handle_command_reply: tid=0 2023-09-23T05:03:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.623+0000 7f3f0c47fb80 1 client.27787 shutdown 2023-09-23T05:03:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.626+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 msgr2=0x56310ffcd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.626+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005850 tx=0x7f3ee8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 shutdown_connections 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56310ffe0140 0x56310ffe2560 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffc5400 0x56310ffcd960 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005850 tx=0x7f3ee8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3336437770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffcdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 >> 172.21.15.5:0/3336437770 conn(0x56310ff3a260 msgr2=0x56310ffc1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.627+0000 7f3ed9ffb700 0 client.27787 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:46.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.629+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 shutdown_connections 2023-09-23T05:03:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.629+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3336437770 wait complete. 2023-09-23T05:03:46.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.633+0000 7f3f0c47fb80 20 client.27787 trim_cache size 0 max 16384 2023-09-23T05:03:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.668+0000 7f3f0c47fb80 1 Processor -- start 2023-09-23T05:03:46.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.668+0000 7f3f0c47fb80 1 -- start start 2023-09-23T05:03:46.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 0x56310ffc79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffe6ee0 0x56310ffcac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310fd99920 con 0x56310ffe6ee0 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ffc9810 con 0x56310ffc5d20 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310ffdcf90 con 0x56310ffd0bd0 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.669+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.670+0000 7f3ef259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 0x56310ffc79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.670+0000 7f3ef259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 0x56310ffc79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42220/0 (socket says 172.21.15.5:42220) 2023-09-23T05:03:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.670+0000 7f3ef259c700 1 -- 172.21.15.5:0/1922622600 learned_addr learned my addr 172.21.15.5:0/1922622600 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/1922622600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 msgr2=0x56310ffc79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 0x56310ffc79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/1922622600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffe6ee0 msgr2=0x56310ffcac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffe6ee0 0x56310ffcac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 -- 172.21.15.5:0/1922622600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56310fd2cc60 con 0x56310ffc5d20 2023-09-23T05:03:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffce240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004550 tx=0x7f3ee8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61dd6ccc98d7432 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.671+0000 7f3ee17fa700 1 -- 172.21.15.5:0/1922622600 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800ba20 con 0x56310ffc5d20 2023-09-23T05:03:46.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.674+0000 7f3ee17fa700 1 -- 172.21.15.5:0/1922622600 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee800bb80 con 0x56310ffc5d20 2023-09-23T05:03:46.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.674+0000 7f3ee17fa700 1 -- 172.21.15.5:0/1922622600 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8025c70 con 0x56310ffc5d20 2023-09-23T05:03:46.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/1922622600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 msgr2=0x56310ffce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffce240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004550 tx=0x7f3ee8005e50 comp rx=0 tx=0).stop 2023-09-23T05:03:46.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/1922622600 shutdown_connections 2023-09-23T05:03:46.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffe6ee0 0x56310ffcac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffc5d20 0x56310ffce240 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004550 tx=0x7f3ee8005e50 comp rx=0 tx=0).stop 2023-09-23T05:03:46.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/1922622600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffd0bd0 0x56310ffc79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.676+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/1922622600 >> 172.21.15.5:0/1922622600 conn(0x56310ff3a260 msgr2=0x56310ffc7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.684+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/1922622600 shutdown_connections 2023-09-23T05:03:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.684+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/1922622600 wait complete. 2023-09-23T05:03:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.684+0000 7f3f0c47fb80 1 -- start start 2023-09-23T05:03:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.685+0000 7f3f0c47fb80 1 Processor -- start 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffd60d0 0x56310ffd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563110034120 con 0x56310ffd0bd0 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563110034260 con 0x56310ffd60d0 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563110045430 con 0x56310ffe6ee0 2023-09-23T05:03:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.686+0000 7f3f0c47fb80 10 client.0 _ll_register_callbacks cb 0x7f3ee8005620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:03:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:42232/0 (socket says 172.21.15.5:42232) 2023-09-23T05:03:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 -- 172.21.15.5:0/3918742551 learned_addr learned my addr 172.21.15.5:0/3918742551 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:03:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffd60d0 0x56310ffd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 -- 172.21.15.5:0/3918742551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffd60d0 msgr2=0x56310ffd6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffd60d0 0x56310ffd6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 -- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 msgr2=0x56310ffd7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:03:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.687+0000 7f3ef259c700 1 -- 172.21.15.5:0/3918742551 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56310fd43de0 con 0x56310ffe6ee0 2023-09-23T05:03:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.688+0000 7f3ef259c700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x56310ffcefa0 tx=0x7f3ee4011710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89647e3723a4f95f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:03:46.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.694+0000 7f3ed9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:03:46.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.694+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4016040 con 0x56310ffe6ee0 2023-09-23T05:03:46.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.694+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3ee401e620 con 0x56310ffe6ee0 2023-09-23T05:03:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.698+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ee802b070 con 0x56310ffe6ee0 2023-09-23T05:03:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.699+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee401e7e0 con 0x56310ffe6ee0 2023-09-23T05:03:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.699+0000 7f3f0c47fb80 10 client.27793 fetch_fsmap learned FSMap version 77 2023-09-23T05:03:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.699+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x56310fd6a250 con 0x56310ffe6ee0 2023-09-23T05:03:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.702+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee802b070 con 0x56310ffe6ee0 2023-09-23T05:03:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.702+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 77) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3ee400fa70 con 0x56310ffe6ee0 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 10 client.27793 fetch_fsmap finished waiting for FSMap version 77 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 10 client.27793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 20 client.27793 populate_metadata read hostname 'smithi005' 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563110046bf0 0x563110049010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 4 client.27793 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.704+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56310ff11b30 con 0x563110046bf0 2023-09-23T05:03:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.705+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563110046bf0 0x563110049010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:03:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.705+0000 7f3ef1d9b700 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563110046bf0 0x563110049010 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582f6cd1c2664bc2 server_cookie=278f814c5ea5efd8 in_seq=0 out_seq=0 2023-09-23T05:03:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.706+0000 7f3ed9ffb700 10 client.27793 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.715+0000 7f3ed9ffb700 1 -- 172.21.15.5:0/3918742551 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56310ff11b30 con 0x563110046bf0 2023-09-23T05:03:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.715+0000 7f3ed9ffb700 10 client.27793 handle_command_reply: tid=0 2023-09-23T05:03:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.719+0000 7f3f0c47fb80 1 client.27793 shutdown 2023-09-23T05:03:46.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.727+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 msgr2=0x56310ffd5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.727+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x56310ffcefa0 tx=0x7f3ee4011710 comp rx=0 tx=0).stop 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 shutdown_connections 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563110046bf0 0x563110049010 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56310ffd60d0 0x56310ffd6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56310ffd0bd0 0x56310ffd7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 --2- 172.21.15.5:0/3918742551 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56310ffe6ee0 0x56310ffd5a80 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x56310ffcefa0 tx=0x7f3ee4011710 comp rx=0 tx=0).stop 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 >> 172.21.15.5:0/3918742551 conn(0x56310ff3a260 msgr2=0x56310ffc7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.729+0000 7f3ed9ffb700 0 client.27793 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:03:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.735+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 shutdown_connections 2023-09-23T05:03:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.739+0000 7f3f0c47fb80 1 -- 172.21.15.5:0/3918742551 wait complete. 2023-09-23T05:03:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.739+0000 7f3f0c47fb80 20 client.27793 trim_cache size 0 max 16384 2023-09-23T05:03:46.740 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:03:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:03:46.742 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:03:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.740+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 msgr2=0x7f3edc0a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.740+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 0x7f3edc0a3c90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3eec095d10 tx=0x7f3eec004cc0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.740+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 msgr2=0x7f3ef41a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.740+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ada0 tx=0x7f3eec00c330 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 shutdown_connections 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3edc0a17d0 0x7f3edc0a3c90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3ef41a53c0 0x7f3ef41a5870 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef40fffe0 0x7f3ef406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 --2- 172.21.15.5:0/1967748744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ef4105100 0x7f3ef406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 >> 172.21.15.5:0/1967748744 conn(0x7f3ef4076f30 msgr2=0x7f3ef41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 shutdown_connections 2023-09-23T05:03:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:03:46.741+0000 7f3ef8bff700 1 -- 172.21.15.5:0/1967748744 wait complete. 2023-09-23T05:03:46.782 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:03:47.078 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:47 smithi005 ceph-mon[97638]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 518 op/s 2023-09-23T05:03:47.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:47 smithi118 ceph-mon[134297]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 518 op/s 2023-09-23T05:03:47.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:47 smithi069 ceph-mon[133931]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 518 op/s 2023-09-23T05:03:47.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 144907 28.53 MB/sec execute 142 sec latency 407.361 ms 2023-09-23T05:03:48.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 145152 28.39 MB/sec execute 143 sec latency 287.671 ms 2023-09-23T05:03:49.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:49 smithi005 ceph-mon[97638]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 742 op/s 2023-09-23T05:03:49.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:49 smithi118 ceph-mon[134297]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 742 op/s 2023-09-23T05:03:49.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:49 smithi069 ceph-mon[133931]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 742 op/s 2023-09-23T05:03:49.553 INFO:tasks.workunit.client.0.smithi005.stdout: 10 145682 28.37 MB/sec execute 144 sec latency 258.185 ms 2023-09-23T05:03:50.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 146365 28.45 MB/sec execute 145 sec latency 82.195 ms 2023-09-23T05:03:51.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 146902 28.44 MB/sec execute 146 sec latency 88.298 ms 2023-09-23T05:03:52.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:51 smithi005 ceph-mon[97638]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-09-23T05:03:52.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:51 smithi118 ceph-mon[134297]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-09-23T05:03:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:51 smithi069 ceph-mon[133931]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 18 MiB/s wr, 715 op/s 2023-09-23T05:03:52.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 147576 28.57 MB/sec execute 147 sec latency 84.479 ms 2023-09-23T05:03:53.554 INFO:tasks.workunit.client.0.smithi005.stdout: 10 148169 28.57 MB/sec execute 148 sec latency 65.320 ms 2023-09-23T05:03:54.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:54 smithi005 ceph-mon[97638]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 779 op/s 2023-09-23T05:03:54.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:54 smithi118 ceph-mon[134297]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 779 op/s 2023-09-23T05:03:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:54 smithi069 ceph-mon[133931]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 779 op/s 2023-09-23T05:03:54.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 148751 28.60 MB/sec execute 149 sec latency 84.418 ms 2023-09-23T05:03:55.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:55 smithi005 ceph-mon[97638]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 695 op/s 2023-09-23T05:03:55.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:55 smithi118 ceph-mon[134297]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 695 op/s 2023-09-23T05:03:55.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:55 smithi069 ceph-mon[133931]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 695 op/s 2023-09-23T05:03:55.555 INFO:tasks.workunit.client.0.smithi005.stdout: 10 149250 28.54 MB/sec execute 150 sec latency 89.542 ms 2023-09-23T05:03:56.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 149826 28.56 MB/sec execute 151 sec latency 104.743 ms 2023-09-23T05:03:57.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 150329 28.58 MB/sec execute 152 sec latency 167.179 ms 2023-09-23T05:03:58.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:58 smithi005 ceph-mon[97638]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-09-23T05:03:58.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:58 smithi118 ceph-mon[134297]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-09-23T05:03:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:58 smithi069 ceph-mon[133931]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 11 MiB/s wr, 548 op/s 2023-09-23T05:03:58.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 150895 28.59 MB/sec execute 153 sec latency 128.911 ms 2023-09-23T05:03:59.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:03:59 smithi005 ceph-mon[97638]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:03:59.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:03:59 smithi118 ceph-mon[134297]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:03:59.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:03:59 smithi069 ceph-mon[133931]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 672 op/s 2023-09-23T05:03:59.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 151527 28.60 MB/sec execute 154 sec latency 71.367 ms 2023-09-23T05:04:00.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152236 28.71 MB/sec execute 155 sec latency 76.185 ms 2023-09-23T05:04:01.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:01 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:01.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:01 smithi069 ceph-mon[133931]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 525 op/s 2023-09-23T05:04:01.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152503 28.58 MB/sec execute 156 sec latency 516.664 ms 2023-09-23T05:04:01.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:01 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:01.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:01 smithi005 ceph-mon[97638]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 525 op/s 2023-09-23T05:04:01.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:01 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:01.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:01 smithi118 ceph-mon[134297]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 525 op/s 2023-09-23T05:04:02.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152555 28.40 MB/sec execute 157 sec latency 1303.056 ms 2023-09-23T05:04:03.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152792 28.25 MB/sec execute 158 sec latency 423.063 ms 2023-09-23T05:04:04.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:04 smithi005 ceph-mon[97638]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 561 op/s 2023-09-23T05:04:04.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:04 smithi118 ceph-mon[134297]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 561 op/s 2023-09-23T05:04:04.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:04 smithi069 ceph-mon[133931]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 561 op/s 2023-09-23T05:04:04.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152998 28.15 MB/sec execute 159 sec latency 324.019 ms 2023-09-23T05:04:05.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:05 smithi005 ceph-mon[97638]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 495 op/s 2023-09-23T05:04:05.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:05 smithi118 ceph-mon[134297]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 495 op/s 2023-09-23T05:04:05.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:05 smithi069 ceph-mon[133931]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.7 MiB/s wr, 495 op/s 2023-09-23T05:04:05.559 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153017 27.99 MB/sec execute 160 sec latency 319.084 ms 2023-09-23T05:04:06.560 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153102 27.87 MB/sec execute 161 sec latency 446.328 ms 2023-09-23T05:04:07.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:07 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:07.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:07 smithi005 ceph-mon[97638]: osdmap e115: 12 total, 12 up, 12 in 2023-09-23T05:04:07.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:07 smithi005 ceph-mon[97638]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 500 op/s 2023-09-23T05:04:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:07 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:07 smithi118 ceph-mon[134297]: osdmap e115: 12 total, 12 up, 12 in 2023-09-23T05:04:07.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:07 smithi118 ceph-mon[134297]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 500 op/s 2023-09-23T05:04:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:07 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:04:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:07 smithi069 ceph-mon[133931]: osdmap e115: 12 total, 12 up, 12 in 2023-09-23T05:04:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:07 smithi069 ceph-mon[133931]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.8 MiB/s wr, 500 op/s 2023-09-23T05:04:07.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153266 27.76 MB/sec execute 162 sec latency 332.369 ms 2023-09-23T05:04:08.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153805 27.80 MB/sec execute 163 sec latency 232.055 ms 2023-09-23T05:04:09.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 154446 27.88 MB/sec execute 164 sec latency 65.466 ms 2023-09-23T05:04:10.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:10 smithi005 ceph-mon[97638]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 577 op/s 2023-09-23T05:04:10.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:10 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:04:10.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:10 smithi118 ceph-mon[134297]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 577 op/s 2023-09-23T05:04:10.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:10 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:04:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:10 smithi069 ceph-mon[133931]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.0 MiB/s wr, 577 op/s 2023-09-23T05:04:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:10 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:04:10.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 154932 27.82 MB/sec execute 165 sec latency 252.566 ms 2023-09-23T05:04:11.563 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155085 27.70 MB/sec execute 166 sec latency 815.832 ms 2023-09-23T05:04:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:11 smithi069 ceph-mon[133931]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 581 op/s 2023-09-23T05:04:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:11 smithi005 ceph-mon[97638]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 581 op/s 2023-09-23T05:04:12.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:11 smithi118 ceph-mon[134297]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 581 op/s 2023-09-23T05:04:12.564 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155161 27.55 MB/sec execute 167 sec latency 1024.131 ms 2023-09-23T05:04:13.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155345 27.44 MB/sec execute 168 sec latency 313.598 ms 2023-09-23T05:04:14.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:14 smithi118 ceph-mon[134297]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-23T05:04:14.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:14 smithi005 ceph-mon[97638]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-23T05:04:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:14 smithi069 ceph-mon[133931]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-23T05:04:14.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155518 27.32 MB/sec execute 169 sec latency 209.402 ms 2023-09-23T05:04:15.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:15 smithi005 ceph-mon[97638]: pgmap v565: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:15.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:15 smithi118 ceph-mon[134297]: pgmap v565: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:15.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:15 smithi069 ceph-mon[133931]: pgmap v565: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:15.567 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155761 27.24 MB/sec execute 170 sec latency 295.285 ms 2023-09-23T05:04:16.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156423 27.29 MB/sec execute 171 sec latency 186.099 ms 2023-09-23T05:04:16.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:04:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2010968201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 msgr2=0x7fb444104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2010968201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb444104c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb4340098f0 tx=0x7fb434009c00 comp rx=0 tx=0).stop 2023-09-23T05:04:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2010968201 shutdown_connections 2023-09-23T05:04:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2010968201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4441052b0 0x7fb444105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2010968201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4440ffeb0 0x7fb444100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2010968201 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb444104c90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.130+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2010968201 >> 172.21.15.5:0/2010968201 conn(0x7fb444076f30 msgr2=0x7fb444077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.131+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2010968201 shutdown_connections 2023-09-23T05:04:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.131+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2010968201 wait complete. 2023-09-23T05:04:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.132+0000 7fb44c1ff700 1 Processor -- start 2023-09-23T05:04:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.133+0000 7fb44c1ff700 1 -- start start 2023-09-23T05:04:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.133+0000 7fb44c1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.133+0000 7fb44c1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb44419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb44c1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4441052b0 0x7fb4441a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb44c1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441a25f0 con 0x7fb4441052b0 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb44c1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441a2760 con 0x7fb4440ffeb0 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:37544/0 (socket says 172.21.15.5:37544) 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb449f9b700 1 -- 172.21.15.5:0/2472471134 learned_addr learned my addr 172.21.15.5:0/2472471134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441a2920 con 0x7fb4441007d0 2023-09-23T05:04:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.134+0000 7fb44979a700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb44419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb44a79c700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4441052b0 0x7fb4441a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 -- 172.21.15.5:0/2472471134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 msgr2=0x7fb44419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb44419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 -- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4441052b0 msgr2=0x7fb4441a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4441052b0 0x7fb4441a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 -- 172.21.15.5:0/2472471134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009520 con 0x7fb4440ffeb0 2023-09-23T05:04:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.135+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb44000d950 tx=0x7fb44000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.136+0000 7fb43affd700 1 -- 172.21.15.5:0/2472471134 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440009940 con 0x7fb4440ffeb0 2023-09-23T05:04:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.136+0000 7fb43affd700 1 -- 172.21.15.5:0/2472471134 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb440010460 con 0x7fb4440ffeb0 2023-09-23T05:04:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.136+0000 7fb43affd700 1 -- 172.21.15.5:0/2472471134 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000f620 con 0x7fb4440ffeb0 2023-09-23T05:04:17.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.136+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4441a2c00 con 0x7fb4440ffeb0 2023-09-23T05:04:17.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.137+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444106440 con 0x7fb4440ffeb0 2023-09-23T05:04:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.143+0000 7fb45fc1bb80 1 Processor -- start 2023-09-23T05:04:17.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 -- start start 2023-09-23T05:04:17.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda41fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd99d460 con 0x564bbda34400 2023-09-23T05:04:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd98ec20 con 0x564bbda3fbd0 2023-09-23T05:04:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.144+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd98c540 con 0x564bbda34d20 2023-09-23T05:04:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.145+0000 7fb44a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.145+0000 7fb44a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51388/0 (socket says 172.21.15.5:51388) 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.145+0000 7fb44a79c700 1 -- 172.21.15.5:0/3122903675 learned_addr learned my addr 172.21.15.5:0/3122903675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb44a79c700 1 -- 172.21.15.5:0/3122903675 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 msgr2=0x564bbda3f690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb44a79c700 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb44a79c700 1 -- 172.21.15.5:0/3122903675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 msgr2=0x564bbda41fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb44a79c700 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda41fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb44a79c700 1 -- 172.21.15.5:0/3122903675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bbd9ea8d0 con 0x564bbda34400 2023-09-23T05:04:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.146+0000 7fb43affd700 1 -- 172.21.15.5:0/2472471134 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb4400108d0 con 0x7fb4440ffeb0 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.147+0000 7fb44a79c700 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009410 tx=0x7fb43c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3feb4629df1f5ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.148+0000 7fb42ffff700 1 -- 172.21.15.5:0/3122903675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00e070 con 0x564bbda34400 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.148+0000 7fb42ffff700 1 -- 172.21.15.5:0/3122903675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43c00f040 con 0x564bbda34400 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.148+0000 7fb42ffff700 1 -- 172.21.15.5:0/3122903675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c015b20 con 0x564bbda34400 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.148+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3122903675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 msgr2=0x564bbda347e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.148+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb43c009410 tx=0x7fb43c0099e0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb43affd700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 0x7fb4300a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb43affd700 1 -- 172.21.15.5:0/2472471134 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb4400cd6f0 con 0x7fb4440ffeb0 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3122903675 shutdown_connections 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda41fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda34400 0x564bbda347e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3122903675 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.149+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3122903675 >> 172.21.15.5:0/3122903675 conn(0x564bbd9a9260 msgr2=0x564bbda30790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.150+0000 7fb44a79c700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 0x7fb4300a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.150+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3122903675 shutdown_connections 2023-09-23T05:04:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.150+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3122903675 wait complete. 2023-09-23T05:04:17.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.151+0000 7fb45fc1bb80 1 -- start start 2023-09-23T05:04:17.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.151+0000 7fb45fc1bb80 1 Processor -- start 2023-09-23T05:04:17.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.152+0000 7fb44a79c700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 0x7fb4300a3d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0061f0 tx=0x7fb43c0155e0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda3fbd0 0x564bbda3d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd987e30 con 0x564bbda3fbd0 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd924710 con 0x564bbda34400 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd9243b0 con 0x564bbda34d20 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.153+0000 7fb45fc1bb80 10 client.0 _ll_register_callbacks cb 0x564bbd781800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:37552/0 (socket says 172.21.15.5:37552) 2023-09-23T05:04:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 -- 172.21.15.5:0/3194566096 learned_addr learned my addr 172.21.15.5:0/3194566096 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb44a79c700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda3fbd0 0x564bbda3d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 -- 172.21.15.5:0/3194566096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 msgr2=0x564bbda3cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb44979a700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 -- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda3fbd0 msgr2=0x564bbda3d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda3fbd0 0x564bbda3d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.154+0000 7fb449f9b700 1 -- 172.21.15.5:0/3194566096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c0090f0 con 0x564bbda34400 2023-09-23T05:04:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.155+0000 7fb44979a700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:04:17.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:16 smithi005 ceph-mon[97638]: osdmap e116: 12 total, 12 up, 12 in 2023-09-23T05:04:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.155+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb4400966d0 tx=0x7fb44009aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64f35f5bb8649e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.155+0000 7fb42d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:04:17.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.158+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400cae30 con 0x564bbda34400 2023-09-23T05:04:17.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.158+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb44009a430 con 0x564bbda34400 2023-09-23T05:04:17.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.159+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bbda28250 con 0x564bbda34400 2023-09-23T05:04:17.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.159+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb44009a5f0 con 0x564bbda34400 2023-09-23T05:04:17.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.159+0000 7fb45fc1bb80 10 client.27761 fetch_fsmap learned FSMap version 79 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x564bbd7b2de0 con 0x564bbda34400 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bbda28250 con 0x564bbda34400 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb4400a20d0 con 0x564bbda34400 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 10 client.27761 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 10 client.27761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:04:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 20 client.27761 populate_metadata read hostname 'smithi005' 2023-09-23T05:04:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbda4f780 0x564bbda51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 4 client.27761 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:04:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.160+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bbd980e10 con 0x564bbda4f780 2023-09-23T05:04:17.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:16 smithi118 ceph-mon[134297]: osdmap e116: 12 total, 12 up, 12 in 2023-09-23T05:04:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.161+0000 7fb44979a700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbda4f780 0x564bbda51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.162+0000 7fb44979a700 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbda4f780 0x564bbda51ba0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e2686e78e31491 server_cookie=df73bbced092ac88 in_seq=0 out_seq=0 2023-09-23T05:04:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.162+0000 7fb42d7fa700 10 client.27761 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.163+0000 7fb42d7fa700 1 -- 172.21.15.5:0/3194566096 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564bbd980e10 con 0x564bbda4f780 2023-09-23T05:04:17.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.163+0000 7fb42d7fa700 10 client.27761 handle_command_reply: tid=0 2023-09-23T05:04:17.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.163+0000 7fb45fc1bb80 1 client.27761 shutdown 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 msgr2=0x564bbda3c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb4400966d0 tx=0x7fb44009aa00 comp rx=0 tx=0).stop 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 shutdown_connections 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbda4f780 0x564bbda51ba0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda3fbd0 0x564bbda3d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34400 0x564bbda3c960 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb4400966d0 tx=0x7fb44009aa00 comp rx=0 tx=0).stop 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3194566096 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb42d7fa700 0 client.27761 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 >> 172.21.15.5:0/3194566096 conn(0x564bbd9a9260 msgr2=0x564bbda30380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 shutdown_connections 2023-09-23T05:04:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.164+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3194566096 wait complete. 2023-09-23T05:04:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.165+0000 7fb45fc1bb80 20 client.27761 trim_cache size 0 max 16384 2023-09-23T05:04:17.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.192+0000 7fb45fc1bb80 1 Processor -- start 2023-09-23T05:04:17.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.192+0000 7fb45fc1bb80 1 -- start start 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda3fbd0 0x564bbda36b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda39c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbd808920 con 0x564bbda37050 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbda32080 con 0x564bbda34d20 2023-09-23T05:04:17.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbda35360 con 0x564bbda3fbd0 2023-09-23T05:04:17.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:37560/0 (socket says 172.21.15.5:37560) 2023-09-23T05:04:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.193+0000 7fb449f9b700 1 -- 172.21.15.5:0/3661438166 learned_addr learned my addr 172.21.15.5:0/3661438166 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb44a79c700 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda39c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb44979a700 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda3fbd0 0x564bbda36b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 -- 172.21.15.5:0/3661438166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda3fbd0 msgr2=0x564bbda36b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda3fbd0 0x564bbda36b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 -- 172.21.15.5:0/3661438166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 msgr2=0x564bbda39c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda39c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 -- 172.21.15.5:0/3661438166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bbda2f590 con 0x564bbda34d20 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.194+0000 7fb449f9b700 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb44009a160 tx=0x7fb440000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9daddbfc8bd67b6b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.195+0000 7fb438ff9700 1 -- 172.21.15.5:0/3661438166 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b8360 con 0x564bbda34d20 2023-09-23T05:04:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.195+0000 7fb438ff9700 1 -- 172.21.15.5:0/3661438166 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4400cac30 con 0x564bbda34d20 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.195+0000 7fb438ff9700 1 -- 172.21.15.5:0/3661438166 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44009a690 con 0x564bbda34d20 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3661438166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 msgr2=0x564bbda3d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb44009a160 tx=0x7fb440000bc0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3661438166 shutdown_connections 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda39c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda34d20 0x564bbda3d260 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/3661438166 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda3fbd0 0x564bbda36b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.196+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3661438166 >> 172.21.15.5:0/3661438166 conn(0x564bbd9a9260 msgr2=0x564bbda36400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.197+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3661438166 shutdown_connections 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.197+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/3661438166 wait complete. 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.197+0000 7fb45fc1bb80 1 -- start start 2023-09-23T05:04:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.198+0000 7fb45fc1bb80 1 Processor -- start 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.198+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda44ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda44ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbda45700 con 0x564bbda37050 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbda45870 con 0x564bbda3fbd0 2023-09-23T05:04:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bbdaa3580 con 0x564bbda34d20 2023-09-23T05:04:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb45fc1bb80 10 client.0 _ll_register_callbacks cb 0x7fb440099d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:04:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb449f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54092/0 (socket says 172.21.15.5:54092) 2023-09-23T05:04:17.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb449f9b700 1 -- 172.21.15.5:0/2351924662 learned_addr learned my addr 172.21.15.5:0/2351924662 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.199+0000 7fb44979a700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda44ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 -- 172.21.15.5:0/2351924662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 msgr2=0x564bbda44ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda44ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 -- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 msgr2=0x564bbda44ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda44ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 -- 172.21.15.5:0/2351924662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bbd7b2de0 con 0x564bbda34d20 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb449f9b700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb440010c80 tx=0x7fb44009aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b5c7264b77b055 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb42d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440096510 con 0x564bbda34d20 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb4400996c0 con 0x564bbda34d20 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.200+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb43c00f040 con 0x564bbda34d20 2023-09-23T05:04:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4400b8360 con 0x564bbda34d20 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 10 client.27805 fetch_fsmap learned FSMap version 79 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x564bbd79bc60 con 0x564bbda34d20 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43c00f040 con 0x564bbda34d20 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb440095fc0 con 0x564bbda34d20 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 10 client.27805 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 10 client.27805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 20 client.27805 populate_metadata read hostname 'smithi005' 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbdaa4700 0x564bbdaa6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 4 client.27805 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:04:17.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.201+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564bbd980b30 con 0x564bbdaa4700 2023-09-23T05:04:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.202+0000 7fb44979a700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbdaa4700 0x564bbdaa6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.203+0000 7fb42d7fa700 10 client.27805 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.203+0000 7fb44979a700 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbdaa4700 0x564bbdaa6af0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f239623561e446a server_cookie=6eaa341695dc4bc3 in_seq=0 out_seq=0 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.204+0000 7fb42d7fa700 1 -- 172.21.15.5:0/2351924662 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564bbd980b30 con 0x564bbdaa4700 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.204+0000 7fb42d7fa700 10 client.27805 handle_command_reply: tid=0 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.204+0000 7fb45fc1bb80 1 client.27805 shutdown 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.204+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 msgr2=0x564bbda46780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.204+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb440010c80 tx=0x7fb44009aa00 comp rx=0 tx=0).stop 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 shutdown_connections 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564bbdaa4700 0x564bbdaa6af0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564bbda3fbd0 0x564bbda44ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564bbda34d20 0x564bbda46780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb42d7fa700 0 client.27805 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 --2- 172.21.15.5:0/2351924662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bbda37050 0x564bbda44ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 >> 172.21.15.5:0/2351924662 conn(0x564bbd9a9260 msgr2=0x564bbda3cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 shutdown_connections 2023-09-23T05:04:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.205+0000 7fb45fc1bb80 1 -- 172.21.15.5:0/2351924662 wait complete. 2023-09-23T05:04:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.206+0000 7fb45fc1bb80 20 client.27805 trim_cache size 0 max 16384 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:04:17.207 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 msgr2=0x7fb4300a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 0x7fb4300a3d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0061f0 tx=0x7fb43c0155e0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 msgr2=0x7fb44419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb44000d950 tx=0x7fb44000dc60 comp rx=0 tx=0).stop 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 shutdown_connections 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb4300a1850 0x7fb4300a3d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4441052b0 0x7fb4441a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4440ffeb0 0x7fb44419d610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 --2- 172.21.15.5:0/2472471134 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb4441007d0 0x7fb44419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.207+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 >> 172.21.15.5:0/2472471134 conn(0x7fb444076f30 msgr2=0x7fb444102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.209+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 shutdown_connections 2023-09-23T05:04:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:17.212+0000 7fb44c1ff700 1 -- 172.21.15.5:0/2472471134 wait complete. 2023-09-23T05:04:17.241 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:04:17.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:16 smithi069 ceph-mon[133931]: osdmap e116: 12 total, 12 up, 12 in 2023-09-23T05:04:17.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 157086 27.37 MB/sec execute 172 sec latency 323.433 ms 2023-09-23T05:04:18.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:17 smithi005 ceph-mon[97638]: pgmap v567: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:18.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:17 smithi118 ceph-mon[134297]: pgmap v567: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:18.244 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:17 smithi069 ceph-mon[133931]: pgmap v567: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 15 MiB/s wr, 654 op/s 2023-09-23T05:04:18.565 INFO:tasks.workunit.client.0.smithi005.stdout: 10 157581 27.39 MB/sec execute 173 sec latency 91.944 ms 2023-09-23T05:04:19.566 INFO:tasks.workunit.client.0.smithi005.stdout: 10 158059 27.38 MB/sec execute 174 sec latency 108.587 ms 2023-09-23T05:04:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:20 smithi005 ceph-mon[97638]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 626 op/s 2023-09-23T05:04:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:04:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:04:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:20 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:04:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:20 smithi118 ceph-mon[134297]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 626 op/s 2023-09-23T05:04:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:04:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:04:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:20 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:04:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:20 smithi069 ceph-mon[133931]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 626 op/s 2023-09-23T05:04:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:04:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:04:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:20 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:04:20.572 INFO:tasks.workunit.client.0.smithi005.stdout: 10 158478 27.34 MB/sec execute 175 sec latency 140.940 ms 2023-09-23T05:04:21.570 INFO:tasks.workunit.client.0.smithi005.stdout: 10 158979 27.31 MB/sec execute 176 sec latency 91.762 ms 2023-09-23T05:04:22.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:21 smithi005 ceph-mon[97638]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:04:22.160 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:21 smithi118 ceph-mon[134297]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:04:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:21 smithi069 ceph-mon[133931]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 608 op/s 2023-09-23T05:04:22.569 INFO:tasks.workunit.client.0.smithi005.stdout: 10 159550 27.34 MB/sec execute 177 sec latency 107.347 ms 2023-09-23T05:04:23.571 INFO:tasks.workunit.client.0.smithi005.stdout: 10 160259 27.38 MB/sec execute 178 sec latency 88.569 ms 2023-09-23T05:04:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:24 smithi005 ceph-mon[97638]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-09-23T05:04:24.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:24 smithi118 ceph-mon[134297]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-09-23T05:04:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:24 smithi069 ceph-mon[133931]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 610 op/s 2023-09-23T05:04:24.572 INFO:tasks.workunit.client.0.smithi005.stdout: 10 160771 27.42 MB/sec execute 179 sec latency 119.519 ms 2023-09-23T05:04:25.573 INFO:tasks.workunit.client.0.smithi005.stdout: 10 161273 27.45 MB/sec execute 180 sec latency 92.199 ms 2023-09-23T05:04:26.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:26 smithi005 ceph-mon[97638]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 540 op/s 2023-09-23T05:04:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:26 smithi118 ceph-mon[134297]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 540 op/s 2023-09-23T05:04:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:26 smithi069 ceph-mon[133931]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.2 MiB/s wr, 540 op/s 2023-09-23T05:04:26.573 INFO:tasks.workunit.client.0.smithi005.stdout: 10 161752 27.43 MB/sec execute 181 sec latency 154.415 ms 2023-09-23T05:04:27.573 INFO:tasks.workunit.client.0.smithi005.stdout: 10 162260 27.35 MB/sec execute 182 sec latency 104.469 ms 2023-09-23T05:04:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:28 smithi005 ceph-mon[97638]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.5 MiB/s wr, 487 op/s 2023-09-23T05:04:28.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:28 smithi118 ceph-mon[134297]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.5 MiB/s wr, 487 op/s 2023-09-23T05:04:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:28 smithi069 ceph-mon[133931]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.5 MiB/s wr, 487 op/s 2023-09-23T05:04:28.573 INFO:tasks.workunit.client.0.smithi005.stdout: 10 162913 27.42 MB/sec execute 183 sec latency 91.331 ms 2023-09-23T05:04:29.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:29 smithi005 ceph-mon[97638]: pgmap v573: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 551 op/s 2023-09-23T05:04:29.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:29 smithi118 ceph-mon[134297]: pgmap v573: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 551 op/s 2023-09-23T05:04:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:29 smithi069 ceph-mon[133931]: pgmap v573: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.0 MiB/s wr, 551 op/s 2023-09-23T05:04:29.573 INFO:tasks.workunit.client.0.smithi005.stdout: 10 163649 27.54 MB/sec execute 184 sec latency 68.674 ms 2023-09-23T05:04:30.574 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164163 27.51 MB/sec execute 185 sec latency 117.315 ms 2023-09-23T05:04:31.574 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164487 27.45 MB/sec execute 186 sec latency 132.975 ms 2023-09-23T05:04:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:31 smithi069 ceph-mon[133931]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 487 op/s 2023-09-23T05:04:32.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:31 smithi005 ceph-mon[97638]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 487 op/s 2023-09-23T05:04:32.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:31 smithi118 ceph-mon[134297]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 487 op/s 2023-09-23T05:04:32.574 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164999 27.44 MB/sec execute 187 sec latency 82.803 ms 2023-09-23T05:04:33.574 INFO:tasks.workunit.client.0.smithi005.stdout: 10 165646 27.48 MB/sec execute 188 sec latency 69.418 ms 2023-09-23T05:04:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:34 smithi005 ceph-mon[97638]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 554 op/s 2023-09-23T05:04:34.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:34 smithi118 ceph-mon[134297]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 554 op/s 2023-09-23T05:04:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:34 smithi069 ceph-mon[133931]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 554 op/s 2023-09-23T05:04:34.574 INFO:tasks.workunit.client.0.smithi005.stdout: 10 166372 27.54 MB/sec execute 189 sec latency 73.143 ms 2023-09-23T05:04:35.575 INFO:tasks.workunit.client.0.smithi005.stdout: 10 166853 27.57 MB/sec execute 190 sec latency 147.694 ms 2023-09-23T05:04:36.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:36 smithi005 ceph-mon[97638]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:04:36.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:36 smithi118 ceph-mon[134297]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:04:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:36 smithi069 ceph-mon[133931]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:04:36.575 INFO:tasks.workunit.client.0.smithi005.stdout: 10 167255 27.56 MB/sec execute 191 sec latency 204.282 ms 2023-09-23T05:04:37.575 INFO:tasks.workunit.client.0.smithi005.stdout: 10 167630 27.46 MB/sec execute 192 sec latency 97.285 ms 2023-09-23T05:04:38.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:38 smithi005 ceph-mon[97638]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 389 op/s 2023-09-23T05:04:38.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:38 smithi118 ceph-mon[134297]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 389 op/s 2023-09-23T05:04:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:38 smithi069 ceph-mon[133931]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 389 op/s 2023-09-23T05:04:38.576 INFO:tasks.workunit.client.0.smithi005.stdout: 10 168284 27.50 MB/sec execute 193 sec latency 87.168 ms 2023-09-23T05:04:39.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:39 smithi005 ceph-mon[97638]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 555 op/s 2023-09-23T05:04:39.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:39 smithi118 ceph-mon[134297]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 555 op/s 2023-09-23T05:04:39.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:39 smithi069 ceph-mon[133931]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.3 MiB/s wr, 555 op/s 2023-09-23T05:04:39.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 168853 27.51 MB/sec execute 194 sec latency 83.224 ms 2023-09-23T05:04:40.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 169560 27.56 MB/sec execute 195 sec latency 72.146 ms 2023-09-23T05:04:41.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 170129 27.63 MB/sec execute 196 sec latency 133.167 ms 2023-09-23T05:04:42.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:41 smithi118 ceph-mon[134297]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 492 op/s 2023-09-23T05:04:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:41 smithi069 ceph-mon[133931]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 492 op/s 2023-09-23T05:04:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:41 smithi005 ceph-mon[97638]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 492 op/s 2023-09-23T05:04:42.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 170682 27.63 MB/sec execute 197 sec latency 99.095 ms 2023-09-23T05:04:43.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 171266 27.66 MB/sec execute 198 sec latency 94.804 ms 2023-09-23T05:04:44.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:44 smithi005 ceph-mon[97638]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 529 op/s 2023-09-23T05:04:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:44 smithi118 ceph-mon[134297]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 529 op/s 2023-09-23T05:04:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:44 smithi069 ceph-mon[133931]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.7 MiB/s wr, 529 op/s 2023-09-23T05:04:44.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 171820 27.66 MB/sec execute 199 sec latency 77.677 ms 2023-09-23T05:04:45.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 172370 27.61 MB/sec execute 200 sec latency 100.853 ms 2023-09-23T05:04:46.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:46 smithi005 ceph-mon[97638]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 475 op/s 2023-09-23T05:04:46.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:46 smithi118 ceph-mon[134297]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 475 op/s 2023-09-23T05:04:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:46 smithi069 ceph-mon[133931]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 475 op/s 2023-09-23T05:04:46.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 172826 27.60 MB/sec execute 201 sec latency 126.702 ms 2023-09-23T05:04:47.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:04:47.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:47 smithi005 ceph-mon[97638]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 420 op/s 2023-09-23T05:04:47.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:47 smithi118 ceph-mon[134297]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 420 op/s 2023-09-23T05:04:47.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:47 smithi069 ceph-mon[133931]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 420 op/s 2023-09-23T05:04:47.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 173307 27.63 MB/sec execute 202 sec latency 421.025 ms 2023-09-23T05:04:48.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 173918 27.69 MB/sec execute 203 sec latency 111.204 ms 2023-09-23T05:04:49.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 174453 27.65 MB/sec execute 204 sec latency 97.145 ms 2023-09-23T05:04:49.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 -- 172.21.15.5:0/783333959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101aa0 msgr2=0x7f2220105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 --2- 172.21.15.5:0/783333959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101aa0 0x7f2220105ae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2210009990 tx=0x7f2210009ca0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 -- 172.21.15.5:0/783333959 shutdown_connections 2023-09-23T05:04:49.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 --2- 172.21.15.5:0/783333959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2220106210 0x7f222010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 --2- 172.21.15.5:0/783333959 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f22201010f0 0x7f22201014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 --2- 172.21.15.5:0/783333959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220101aa0 0x7f2220105ae0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.625+0000 7f22273ff700 1 -- 172.21.15.5:0/783333959 >> 172.21.15.5:0/783333959 conn(0x7f22200fc9d0 msgr2=0x7f22200fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.626+0000 7f22273ff700 1 -- 172.21.15.5:0/783333959 shutdown_connections 2023-09-23T05:04:49.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.626+0000 7f22273ff700 1 -- 172.21.15.5:0/783333959 wait complete. 2023-09-23T05:04:49.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.627+0000 7f22273ff700 1 Processor -- start 2023-09-23T05:04:49.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 -- start start 2023-09-23T05:04:49.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 0x7f222010b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2220106c40 0x7f22201070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22201078c0 con 0x7f2220106210 2023-09-23T05:04:49.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2220107a00 con 0x7f22201010f0 2023-09-23T05:04:49.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.628+0000 7f22273ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2220111190 con 0x7f2220106c40 2023-09-23T05:04:49.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:42678/0 (socket says 172.21.15.5:42678) 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 -- 172.21.15.5:0/1499972960 learned_addr learned my addr 172.21.15.5:0/1499972960 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222499a700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 0x7f222010b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 -- 172.21.15.5:0/1499972960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2220106c40 msgr2=0x7f22201070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2220106c40 0x7f22201070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 -- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 msgr2=0x7f222010b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 0x7f222010b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.629+0000 7f222519b700 1 -- 172.21.15.5:0/1499972960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f22201010f0 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f222499a700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 0x7f222010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:04:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f222519b700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f221c00ba50 tx=0x7f221c00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f22167fc700 1 -- 172.21.15.5:0/1499972960 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00cab0 con 0x7f22201010f0 2023-09-23T05:04:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f22167fc700 1 -- 172.21.15.5:0/1499972960 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c00cc10 con 0x7f22201010f0 2023-09-23T05:04:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f22167fc700 1 -- 172.21.15.5:0/1499972960 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0075d0 con 0x7f22201010f0 2023-09-23T05:04:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2220111470 con 0x7f22201010f0 2023-09-23T05:04:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.630+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2220111900 con 0x7f22201010f0 2023-09-23T05:04:49.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.635+0000 7f22167fc700 1 -- 172.21.15.5:0/1499972960 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f221c007730 con 0x7f22201010f0 2023-09-23T05:04:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.637+0000 7f223ac62b80 1 Processor -- start 2023-09-23T05:04:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.638+0000 7f22167fc700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 0x7f220c0a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.638+0000 7f223ac62b80 1 -- start start 2023-09-23T05:04:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.638+0000 7f22167fc700 1 -- 172.21.15.5:0/1499972960 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f221c006e40 con 0x7f22201010f0 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.638+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb224e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb224ebd0 0x555fb2250fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f223ac62b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb21ac460 con 0x555fb2243d20 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f223ac62b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb219dc20 con 0x555fb224ebd0 2023-09-23T05:04:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f223ac62b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb219b540 con 0x555fb2243400 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f222599c700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 0x7f220c0a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f222519b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f222519b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34554/0 (socket says 172.21.15.5:34554) 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f222519b700 1 -- 172.21.15.5:0/2597057883 learned_addr learned my addr 172.21.15.5:0/2597057883 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.639+0000 7f222599c700 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb224ebd0 0x555fb2250fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.640+0000 7f222519b700 1 -- 172.21.15.5:0/2597057883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb224ebd0 msgr2=0x555fb2250fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.640+0000 7f222519b700 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb224ebd0 0x555fb2250fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.640+0000 7f222519b700 1 -- 172.21.15.5:0/2597057883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 msgr2=0x555fb224e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.640+0000 7f222519b700 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb224e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.640+0000 7f222519b700 1 -- 172.21.15.5:0/2597057883 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fb21f98d0 con 0x555fb2243400 2023-09-23T05:04:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.641+0000 7f222599c700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 0x7f220c0a3cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2218009e00 tx=0x7f2218009150 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.641+0000 7f222519b700 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f221c0a2800 tx=0x7f221c09b0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6227480e7aea3b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.641+0000 7f220b7fe700 1 -- 172.21.15.5:0/2597057883 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c09a9a0 con 0x555fb2243400 2023-09-23T05:04:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.641+0000 7f220b7fe700 1 -- 172.21.15.5:0/2597057883 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c09eaf0 con 0x555fb2243400 2023-09-23T05:04:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.641+0000 7f220b7fe700 1 -- 172.21.15.5:0/2597057883 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0a28a0 con 0x555fb2243400 2023-09-23T05:04:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.642+0000 7f223ac62b80 1 -- 172.21.15.5:0/2597057883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 msgr2=0x555fb22437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.642+0000 7f223ac62b80 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f221c0a2800 tx=0x7f221c09b0a0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 -- 172.21.15.5:0/2597057883 shutdown_connections 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb224ebd0 0x555fb2250fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb22437e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 --2- 172.21.15.5:0/2597057883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb224e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 -- 172.21.15.5:0/2597057883 >> 172.21.15.5:0/2597057883 conn(0x555fb21b8260 msgr2=0x555fb223f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 -- 172.21.15.5:0/2597057883 shutdown_connections 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.643+0000 7f223ac62b80 1 -- 172.21.15.5:0/2597057883 wait complete. 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.644+0000 7f223ac62b80 1 -- start start 2023-09-23T05:04:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.644+0000 7f223ac62b80 1 Processor -- start 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb224b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb224bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb2196e30 con 0x555fb2243400 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb2133710 con 0x555fb2243d20 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb21333b0 con 0x555fb224ebd0 2023-09-23T05:04:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.645+0000 7f223ac62b80 10 client.0 _ll_register_callbacks cb 0x555fb1f90800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.646+0000 7f222519b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.646+0000 7f222519b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40904/0 (socket says 172.21.15.5:40904) 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.646+0000 7f222519b700 1 -- 172.21.15.5:0/4149271836 learned_addr learned my addr 172.21.15.5:0/4149271836 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222599c700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb224bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222499a700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb224b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222519b700 1 -- 172.21.15.5:0/4149271836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 msgr2=0x555fb224bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222519b700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb224bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222519b700 1 -- 172.21.15.5:0/4149271836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 msgr2=0x555fb224b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222519b700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb224b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.647+0000 7f222519b700 1 -- 172.21.15.5:0/4149271836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2220111900 con 0x555fb2243400 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.648+0000 7f222519b700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f221c016000 tx=0x7f221c09aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fcfcdab288b20a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.648+0000 7f2208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.648+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c09aa20 con 0x555fb2243400 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.648+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f221c0a2ab0 con 0x555fb2243400 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.648+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fb2237250 con 0x555fb2243400 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221c0ba0b0 con 0x555fb2243400 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f223ac62b80 10 client.17730 fetch_fsmap learned FSMap version 79 2023-09-23T05:04:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x555fb1fc1de0 con 0x555fb2243400 2023-09-23T05:04:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fb2237250 con 0x555fb2243400 2023-09-23T05:04:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f221c09f830 con 0x555fb2243400 2023-09-23T05:04:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f223ac62b80 10 client.17730 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:04:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f223ac62b80 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.649+0000 7f223ac62b80 20 client.17730 populate_metadata read hostname 'smithi005' 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.650+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb225e1f0 0x555fb22605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.650+0000 7f223ac62b80 4 client.17730 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.650+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fb218fe10 con 0x555fb225e1f0 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.650+0000 7f222499a700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb225e1f0 0x555fb22605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.651+0000 7f222499a700 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb225e1f0 0x555fb22605e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f814f8ed71754dc2 server_cookie=dac469043e8719b1 in_seq=0 out_seq=0 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.651+0000 7f2208ff9700 10 client.17730 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.651+0000 7f2208ff9700 1 -- 172.21.15.5:0/4149271836 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555fb218fe10 con 0x555fb225e1f0 2023-09-23T05:04:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.651+0000 7f2208ff9700 10 client.17730 handle_command_reply: tid=0 2023-09-23T05:04:49.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.652+0000 7f223ac62b80 1 client.17730 shutdown 2023-09-23T05:04:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 msgr2=0x555fb224e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f221c016000 tx=0x7f221c09aec0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 shutdown_connections 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb225e1f0 0x555fb22605e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb224bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243400 0x555fb224e590 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 --2- 172.21.15.5:0/4149271836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb224b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.653+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 >> 172.21.15.5:0/4149271836 conn(0x555fb21b8260 msgr2=0x555fb223f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.654+0000 7f2208ff9700 0 client.17730 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.654+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 shutdown_connections 2023-09-23T05:04:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.654+0000 7f223ac62b80 1 -- 172.21.15.5:0/4149271836 wait complete. 2023-09-23T05:04:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.655+0000 7f223ac62b80 20 client.17730 trim_cache size 0 max 16384 2023-09-23T05:04:49.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.680+0000 7f223ac62b80 1 Processor -- start 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 -- start start 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243400 0x555fb224e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb2246b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb2017920 con 0x555fb2243d20 2023-09-23T05:04:49.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb224cdf0 con 0x555fb2243400 2023-09-23T05:04:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.681+0000 7f223ac62b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb2239860 con 0x555fb224ebd0 2023-09-23T05:04:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34578/0 (socket says 172.21.15.5:34578) 2023-09-23T05:04:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 -- 172.21.15.5:0/3158335333 learned_addr learned my addr 172.21.15.5:0/3158335333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:49.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222499a700 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb2246b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 -- 172.21.15.5:0/3158335333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243400 msgr2=0x555fb224e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243400 0x555fb224e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 -- 172.21.15.5:0/3158335333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 msgr2=0x555fb2246b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb2246b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.682+0000 7f222599c700 1 -- 172.21.15.5:0/3158335333 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fb1faac60 con 0x555fb224ebd0 2023-09-23T05:04:49.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.683+0000 7f222599c700 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22180092b0 tx=0x7f2218004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea99f134972bd15 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.683+0000 7f220bfff700 1 -- 172.21.15.5:0/3158335333 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221801e070 con 0x555fb224ebd0 2023-09-23T05:04:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.683+0000 7f220bfff700 1 -- 172.21.15.5:0/3158335333 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2218009a40 con 0x555fb224ebd0 2023-09-23T05:04:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.683+0000 7f220bfff700 1 -- 172.21.15.5:0/3158335333 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218022be0 con 0x555fb224ebd0 2023-09-23T05:04:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 -- 172.21.15.5:0/3158335333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 msgr2=0x555fb22470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22180092b0 tx=0x7f2218004870 comp rx=0 tx=0).stop 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 -- 172.21.15.5:0/3158335333 shutdown_connections 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb224ebd0 0x555fb22470d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243400 0x555fb224e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 --2- 172.21.15.5:0/3158335333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb2243d20 0x555fb2246b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 -- 172.21.15.5:0/3158335333 >> 172.21.15.5:0/3158335333 conn(0x555fb21b8260 msgr2=0x555fb225a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 -- 172.21.15.5:0/3158335333 shutdown_connections 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.684+0000 7f223ac62b80 1 -- 172.21.15.5:0/3158335333 wait complete. 2023-09-23T05:04:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.685+0000 7f223ac62b80 1 -- start start 2023-09-23T05:04:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.685+0000 7f223ac62b80 1 Processor -- start 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb2253440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb2253980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb224ebd0 0x555fb22b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb22b1540 con 0x555fb224ebd0 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb22b1680 con 0x555fb2243d20 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fb22b17c0 con 0x555fb2243400 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.686+0000 7f223ac62b80 10 client.0 _ll_register_callbacks cb 0x7f221c09a390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:04:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.687+0000 7f222499a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb2253980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.687+0000 7f222519b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb2253440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.687+0000 7f222519b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb2253440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34594/0 (socket says 172.21.15.5:34594) 2023-09-23T05:04:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222519b700 1 -- 172.21.15.5:0/672050829 learned_addr learned my addr 172.21.15.5:0/672050829 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:04:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222599c700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb224ebd0 0x555fb22b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222499a700 1 -- 172.21.15.5:0/672050829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 msgr2=0x555fb2253440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222499a700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb2253440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222499a700 1 -- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb224ebd0 msgr2=0x555fb22b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222499a700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb224ebd0 0x555fb22b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.688+0000 7f222499a700 1 -- 172.21.15.5:0/672050829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2218012930 con 0x555fb2243d20 2023-09-23T05:04:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.690+0000 7f222499a700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb2253980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2210003760 tx=0x7f221000b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749eaef6e96c0dc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:04:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.690+0000 7f2208ff9700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:04:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.690+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221000b9d0 con 0x555fb2243d20 2023-09-23T05:04:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.690+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f2210024980 con 0x555fb2243d20 2023-09-23T05:04:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.690+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f221c09eaf0 con 0x555fb2243d20 2023-09-23T05:04:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.691+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221000bb30 con 0x555fb2243d20 2023-09-23T05:04:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.691+0000 7f223ac62b80 10 client.27773 fetch_fsmap learned FSMap version 79 2023-09-23T05:04:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.691+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x555fb1fe8250 con 0x555fb2243d20 2023-09-23T05:04:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.691+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c09eaf0 con 0x555fb2243d20 2023-09-23T05:04:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f22100050d0 con 0x555fb2243d20 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 10 client.27773 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 20 client.27773 populate_metadata read hostname 'smithi005' 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb22b2f90 0x555fb22b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 4 client.27773 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555fb218fb30 con 0x555fb22b2f90 2023-09-23T05:04:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.692+0000 7f222519b700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb22b2f90 0x555fb22b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:04:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.693+0000 7f2208ff9700 10 client.27773 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.693+0000 7f222519b700 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb22b2f90 0x555fb22b53b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a770a97a8d4d65da server_cookie=d431f1ed653fe14d in_seq=0 out_seq=0 2023-09-23T05:04:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.694+0000 7f2208ff9700 1 -- 172.21.15.5:0/672050829 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555fb218fb30 con 0x555fb22b2f90 2023-09-23T05:04:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.694+0000 7f2208ff9700 10 client.27773 handle_command_reply: tid=0 2023-09-23T05:04:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.694+0000 7f223ac62b80 1 client.27773 shutdown 2023-09-23T05:04:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.694+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 msgr2=0x555fb2253980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.694+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb2253980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2210003760 tx=0x7f221000b540 comp rx=0 tx=0).stop 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 shutdown_connections 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x555fb22b2f90 0x555fb22b53b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb224ebd0 0x555fb22b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x555fb2243400 0x555fb2253440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 --2- 172.21.15.5:0/672050829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555fb2243d20 0x555fb2253980 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 >> 172.21.15.5:0/672050829 conn(0x555fb21b8260 msgr2=0x555fb2246b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f2208ff9700 0 client.27773 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 shutdown_connections 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.695+0000 7f223ac62b80 1 -- 172.21.15.5:0/672050829 wait complete. 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:04:49.697 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:04:49.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:04:49.698 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:04:49.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:04:49.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:04:49.698 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.696+0000 7f223ac62b80 20 client.27773 trim_cache size 0 max 16384 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.696+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 msgr2=0x7f220c0a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.696+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 0x7f220c0a3cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2218009e00 tx=0x7f2218009150 comp rx=0 tx=0).stop 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.696+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 msgr2=0x7f222010b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.696+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f221c00ba50 tx=0x7f221c00be10 comp rx=0 tx=0).stop 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 shutdown_connections 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f220c0a1810 0x7f220c0a3cd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f2220106c40 0x7f22201070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22201010f0 0x7f222010b480 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 --2- 172.21.15.5:0/1499972960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2220106210 0x7f222010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:04:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 >> 172.21.15.5:0/1499972960 conn(0x7f22200fc9d0 msgr2=0x7f2220104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:04:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 shutdown_connections 2023-09-23T05:04:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:04:49.697+0000 7f22273ff700 1 -- 172.21.15.5:0/1499972960 wait complete. 2023-09-23T05:04:49.726 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:04:50.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:50 smithi005 ceph-mon[97638]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 594 op/s 2023-09-23T05:04:50.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:50 smithi118 ceph-mon[134297]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 594 op/s 2023-09-23T05:04:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:50 smithi069 ceph-mon[133931]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.7 MiB/s wr, 594 op/s 2023-09-23T05:04:50.577 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175006 27.65 MB/sec execute 205 sec latency 100.696 ms 2023-09-23T05:04:51.579 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175119 27.56 MB/sec execute 206 sec latency 870.494 ms 2023-09-23T05:04:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:51 smithi069 ceph-mon[133931]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 447 op/s 2023-09-23T05:04:52.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:51 smithi005 ceph-mon[97638]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 447 op/s 2023-09-23T05:04:52.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:51 smithi118 ceph-mon[134297]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 447 op/s 2023-09-23T05:04:52.582 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175124 27.43 MB/sec execute 207 sec latency 1786.725 ms 2023-09-23T05:04:53.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:53 smithi005 ceph-mon[97638]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 623 op/s 2023-09-23T05:04:53.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:53 smithi118 ceph-mon[134297]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 623 op/s 2023-09-23T05:04:53.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:53 smithi069 ceph-mon[133931]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 623 op/s 2023-09-23T05:04:53.585 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175210 27.32 MB/sec execute 208 sec latency 2035.964 ms 2023-09-23T05:04:54.584 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175333 27.20 MB/sec execute 209 sec latency 440.903 ms 2023-09-23T05:04:55.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:55 smithi005 ceph-mon[97638]: pgmap v586: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 629 op/s 2023-09-23T05:04:55.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:55 smithi118 ceph-mon[134297]: pgmap v586: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 629 op/s 2023-09-23T05:04:55.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:55 smithi069 ceph-mon[133931]: pgmap v586: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 629 op/s 2023-09-23T05:04:55.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175595 27.13 MB/sec execute 210 sec latency 226.427 ms 2023-09-23T05:04:56.587 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175863 27.09 MB/sec execute 211 sec latency 245.040 ms 2023-09-23T05:04:57.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:04:57 smithi069 ceph-mon[133931]: pgmap v587: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 549 op/s 2023-09-23T05:04:57.590 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176057 27.01 MB/sec execute 212 sec latency 171.794 ms 2023-09-23T05:04:57.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:04:57 smithi005 ceph-mon[97638]: pgmap v587: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 549 op/s 2023-09-23T05:04:57.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:04:57 smithi118 ceph-mon[134297]: pgmap v587: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 549 op/s 2023-09-23T05:04:58.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176349 26.97 MB/sec execute 213 sec latency 182.169 ms 2023-09-23T05:04:59.592 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176909 27.02 MB/sec execute 214 sec latency 137.127 ms 2023-09-23T05:05:00.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:00 smithi005 ceph-mon[97638]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-09-23T05:05:00.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:00 smithi118 ceph-mon[134297]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-09-23T05:05:00.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:00 smithi069 ceph-mon[133931]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-09-23T05:05:00.593 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177361 26.97 MB/sec execute 215 sec latency 255.456 ms 2023-09-23T05:05:01.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177557 26.87 MB/sec execute 216 sec latency 410.475 ms 2023-09-23T05:05:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:01 smithi069 ceph-mon[133931]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 689 op/s 2023-09-23T05:05:02.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:01 smithi005 ceph-mon[97638]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 689 op/s 2023-09-23T05:05:02.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:01 smithi118 ceph-mon[134297]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 689 op/s 2023-09-23T05:05:02.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177671 26.76 MB/sec execute 217 sec latency 469.959 ms 2023-09-23T05:05:03.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177678 26.65 MB/sec execute 218 sec latency 414.595 ms 2023-09-23T05:05:04.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:04 smithi005 ceph-mon[97638]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 757 op/s 2023-09-23T05:05:04.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:04 smithi118 ceph-mon[134297]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 757 op/s 2023-09-23T05:05:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:04 smithi069 ceph-mon[133931]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 19 MiB/s wr, 757 op/s 2023-09-23T05:05:04.594 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177708 26.54 MB/sec execute 219 sec latency 378.207 ms 2023-09-23T05:05:05.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177887 26.49 MB/sec execute 220 sec latency 318.089 ms 2023-09-23T05:05:06.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:06 smithi005 ceph-mon[97638]: pgmap v591: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:05:06.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:06 smithi118 ceph-mon[134297]: pgmap v591: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:05:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:06 smithi069 ceph-mon[133931]: pgmap v591: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 665 op/s 2023-09-23T05:05:06.596 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178282 26.47 MB/sec execute 221 sec latency 323.548 ms 2023-09-23T05:05:07.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:07 smithi005 ceph-mon[97638]: pgmap v592: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-23T05:05:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:07 smithi118 ceph-mon[134297]: pgmap v592: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-23T05:05:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:07 smithi069 ceph-mon[133931]: pgmap v592: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 523 op/s 2023-09-23T05:05:07.597 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178820 26.46 MB/sec execute 222 sec latency 174.702 ms 2023-09-23T05:05:08.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 179411 26.48 MB/sec execute 223 sec latency 129.804 ms 2023-09-23T05:05:09.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 180023 26.54 MB/sec execute 224 sec latency 110.756 ms 2023-09-23T05:05:10.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:10 smithi005 ceph-mon[97638]: pgmap v593: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.9 MiB/s wr, 724 op/s 2023-09-23T05:05:10.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:10 smithi118 ceph-mon[134297]: pgmap v593: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.9 MiB/s wr, 724 op/s 2023-09-23T05:05:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:10 smithi069 ceph-mon[133931]: pgmap v593: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.9 MiB/s wr, 724 op/s 2023-09-23T05:05:10.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 180694 26.60 MB/sec execute 225 sec latency 80.787 ms 2023-09-23T05:05:11.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 181172 26.58 MB/sec execute 226 sec latency 86.764 ms 2023-09-23T05:05:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:11 smithi069 ceph-mon[133931]: pgmap v594: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 546 op/s 2023-09-23T05:05:12.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:11 smithi005 ceph-mon[97638]: pgmap v594: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 546 op/s 2023-09-23T05:05:12.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:11 smithi118 ceph-mon[134297]: pgmap v594: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 546 op/s 2023-09-23T05:05:12.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 181644 26.57 MB/sec execute 227 sec latency 102.959 ms 2023-09-23T05:05:13.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:13 smithi005 ceph-mon[97638]: pgmap v595: 129 pgs: 129 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.9 MiB/s wr, 573 op/s 2023-09-23T05:05:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:13 smithi118 ceph-mon[134297]: pgmap v595: 129 pgs: 129 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.9 MiB/s wr, 573 op/s 2023-09-23T05:05:13.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:13 smithi069 ceph-mon[133931]: pgmap v595: 129 pgs: 129 active+clean; 2.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.9 MiB/s wr, 573 op/s 2023-09-23T05:05:13.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 182137 26.58 MB/sec execute 228 sec latency 99.106 ms 2023-09-23T05:05:14.598 INFO:tasks.workunit.client.0.smithi005.stdout: 10 182612 26.55 MB/sec execute 229 sec latency 94.906 ms 2023-09-23T05:05:15.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 183160 26.55 MB/sec execute 230 sec latency 112.161 ms 2023-09-23T05:05:16.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:16 smithi005 ceph-mon[97638]: pgmap v596: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 575 op/s 2023-09-23T05:05:16.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:16 smithi118 ceph-mon[134297]: pgmap v596: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 575 op/s 2023-09-23T05:05:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:16 smithi069 ceph-mon[133931]: pgmap v596: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 575 op/s 2023-09-23T05:05:16.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 183785 26.62 MB/sec execute 231 sec latency 99.023 ms 2023-09-23T05:05:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:17 smithi005 ceph-mon[97638]: pgmap v597: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-09-23T05:05:17.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:17 smithi118 ceph-mon[134297]: pgmap v597: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-09-23T05:05:17.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:17 smithi069 ceph-mon[133931]: pgmap v597: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.4 MiB/s wr, 452 op/s 2023-09-23T05:05:17.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 184270 26.60 MB/sec execute 232 sec latency 89.721 ms 2023-09-23T05:05:18.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 184763 26.59 MB/sec execute 233 sec latency 215.939 ms 2023-09-23T05:05:19.599 INFO:tasks.workunit.client.0.smithi005.stdout: 10 185348 26.63 MB/sec execute 234 sec latency 94.978 ms 2023-09-23T05:05:19.727 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:05:20.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:20 smithi005 ceph-mon[97638]: pgmap v598: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 587 op/s 2023-09-23T05:05:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:20 smithi118 ceph-mon[134297]: pgmap v598: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 587 op/s 2023-09-23T05:05:20.500 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:20 smithi069 ceph-mon[133931]: pgmap v598: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.6 MiB/s wr, 587 op/s 2023-09-23T05:05:20.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 185716 26.58 MB/sec execute 235 sec latency 131.515 ms 2023-09-23T05:05:21.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:21 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:05:21.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:21 smithi005 ceph-mon[97638]: pgmap v599: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 464 op/s 2023-09-23T05:05:21.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:21 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:05:21.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:21 smithi118 ceph-mon[134297]: pgmap v599: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 464 op/s 2023-09-23T05:05:21.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:21 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:05:21.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:21 smithi069 ceph-mon[133931]: pgmap v599: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.1 MiB/s wr, 464 op/s 2023-09-23T05:05:21.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 186375 26.62 MB/sec execute 236 sec latency 138.455 ms 2023-09-23T05:05:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.797+0000 7fcf747ff700 1 -- 172.21.15.5:0/2247927279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 msgr2=0x7fcf6c1082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.797+0000 7fcf747ff700 1 --2- 172.21.15.5:0/2247927279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c1082a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009990 tx=0x7fcf5c009ca0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 -- 172.21.15.5:0/2247927279 shutdown_connections 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 --2- 172.21.15.5:0/2247927279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 0x7fcf6c079080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 --2- 172.21.15.5:0/2247927279 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 --2- 172.21.15.5:0/2247927279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c1082a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 -- 172.21.15.5:0/2247927279 >> 172.21.15.5:0/2247927279 conn(0x7fcf6c0725f0 msgr2=0x7fcf6c06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 -- 172.21.15.5:0/2247927279 shutdown_connections 2023-09-23T05:05:21.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.798+0000 7fcf747ff700 1 -- 172.21.15.5:0/2247927279 wait complete. 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 Processor -- start 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 -- start start 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c083420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 0x7fcf6c083960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c07dc50 con 0x7fcf6c1087e0 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c07ddc0 con 0x7fcf6c107e20 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.800+0000 7fcf747ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c07df80 con 0x7fcf6c10d490 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.801+0000 7fcf72d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.801+0000 7fcf72d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48220/0 (socket says 172.21.15.5:48220) 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.801+0000 7fcf72d9c700 1 -- 172.21.15.5:0/3024921220 learned_addr learned my addr 172.21.15.5:0/3024921220 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:21.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.801+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c083420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 0x7fcf6c083960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf72d9c700 1 -- 172.21.15.5:0/3024921220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 msgr2=0x7fcf6c083420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf72d9c700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c083420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf72d9c700 1 -- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 msgr2=0x7fcf6c083960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf72d9c700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 0x7fcf6c083960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.802+0000 7fcf72d9c700 1 -- 172.21.15.5:0/3024921220 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0095f0 con 0x7fcf6c10d490 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.803+0000 7fcf72d9c700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400b970 tx=0x7fcf6400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.803+0000 7fcf637fe700 1 -- 172.21.15.5:0/3024921220 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400c930 con 0x7fcf6c10d490 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.803+0000 7fcf637fe700 1 -- 172.21.15.5:0/3024921220 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf64014440 con 0x7fcf6c10d490 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.803+0000 7fcf637fe700 1 -- 172.21.15.5:0/3024921220 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640075d0 con 0x7fcf6c10d490 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.803+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c07e260 con 0x7fcf6c10d490 2023-09-23T05:05:21.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.804+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c136610 con 0x7fcf6c10d490 2023-09-23T05:05:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.812+0000 7fcf881beb80 1 Processor -- start 2023-09-23T05:05:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.814+0000 7fcf637fe700 1 -- 172.21.15.5:0/3024921220 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fcf640145b0 con 0x7fcf6c10d490 2023-09-23T05:05:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 -- start start 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a79da460 con 0x55c6a7a71d20 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a79cbc20 con 0x55c6a7a7cbd0 2023-09-23T05:05:21.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.815+0000 7fcf881beb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a79c9540 con 0x55c6a7a71400 2023-09-23T05:05:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.817+0000 7fcf637fe700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 0x7fcf580a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.817+0000 7fcf637fe700 1 -- 172.21.15.5:0/3024921220 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fcf64006e40 con 0x7fcf6c10d490 2023-09-23T05:05:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.818+0000 7fcf7259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.818+0000 7fcf7259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50868/0 (socket says 172.21.15.5:50868) 2023-09-23T05:05:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.818+0000 7fcf7259b700 1 -- 172.21.15.5:0/3489361876 learned_addr learned my addr 172.21.15.5:0/3489361876 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.819+0000 7fcf7259b700 1 -- 172.21.15.5:0/3489361876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 msgr2=0x55c6a7a717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.819+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.819+0000 7fcf7259b700 1 -- 172.21.15.5:0/3489361876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a7cbd0 msgr2=0x55c6a7a7efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.819+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.819+0000 7fcf7259b700 1 -- 172.21.15.5:0/3489361876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a7a278d0 con 0x55c6a7a71d20 2023-09-23T05:05:21.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.820+0000 7fcf72d9c700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 0x7fcf580a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.821+0000 7fcf72d9c700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 0x7fcf580a3d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf64095d20 tx=0x7fcf64004be0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.822+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ad70 tx=0x7fcf6800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c448cae7feaaac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.822+0000 7fcf60ff9700 1 -- 172.21.15.5:0/3489361876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800a870 con 0x55c6a7a71d20 2023-09-23T05:05:21.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.822+0000 7fcf60ff9700 1 -- 172.21.15.5:0/3489361876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6800fec0 con 0x55c6a7a71d20 2023-09-23T05:05:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.822+0000 7fcf60ff9700 1 -- 172.21.15.5:0/3489361876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680186a0 con 0x55c6a7a71d20 2023-09-23T05:05:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.823+0000 7fcf881beb80 1 -- 172.21.15.5:0/3489361876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 msgr2=0x55c6a7a7c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.823+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ad70 tx=0x7fcf6800f6a0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 -- 172.21.15.5:0/3489361876 shutdown_connections 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a7efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3489361876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c690 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 -- 172.21.15.5:0/3489361876 >> 172.21.15.5:0/3489361876 conn(0x55c6a79e6260 msgr2=0x55c6a7a6d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 -- 172.21.15.5:0/3489361876 shutdown_connections 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.824+0000 7fcf881beb80 1 -- 172.21.15.5:0/3489361876 wait complete. 2023-09-23T05:05:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.825+0000 7fcf881beb80 1 -- start start 2023-09-23T05:05:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.825+0000 7fcf881beb80 1 Processor -- start 2023-09-23T05:05:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a7c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a79db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a79c4e30 con 0x55c6a7a7cbd0 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7961710 con 0x55c6a7a71d20 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a79613b0 con 0x55c6a7a71400 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf881beb80 10 client.0 _ll_register_callbacks cb 0x55c6a77be800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf71d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a7c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf71d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a7c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48238/0 (socket says 172.21.15.5:48238) 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.826+0000 7fcf71d9a700 1 -- 172.21.15.5:0/3275348513 learned_addr learned my addr 172.21.15.5:0/3275348513 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 -- 172.21.15.5:0/3275348513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 msgr2=0x55c6a7a7c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a7c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 -- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 msgr2=0x55c6a7a79db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a79db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.827+0000 7fcf7259b700 1 -- 172.21.15.5:0/3275348513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6800a9d0 con 0x55c6a7a71d20 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.828+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a797c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ad70 tx=0x7fcf68004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab9fc7fa5fa4322 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.828+0000 7fcf55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.828+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68005450 con 0x55c6a7a71d20 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.828+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fcf68003af0 con 0x55c6a7a71d20 2023-09-23T05:05:21.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.828+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a79bde10 con 0x55c6a7a71d20 2023-09-23T05:05:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.829+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6800fec0 con 0x55c6a7a71d20 2023-09-23T05:05:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.829+0000 7fcf881beb80 10 client.27782 fetch_fsmap learned FSMap version 79 2023-09-23T05:05:21.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.829+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55c6a77efde0 con 0x55c6a7a71d20 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6a79bde10 con 0x55c6a7a71d20 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf68020040 con 0x55c6a7a71d20 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 10 client.27782 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 10 client.27782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 20 client.27782 populate_metadata read hostname 'smithi005' 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7a8c5b0 0x55c6a7a8e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 4 client.27782 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:05:21.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.830+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a79bdb30 con 0x55c6a7a8c5b0 2023-09-23T05:05:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.832+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7a8c5b0 0x55c6a7a8e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.834+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7a8c5b0 0x55c6a7a8e9d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0d9caac91e3b9d server_cookie=719477f4ce4ec892 in_seq=0 out_seq=0 2023-09-23T05:05:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.834+0000 7fcf55ffb700 10 client.27782 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.835+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3275348513 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6a79bdb30 con 0x55c6a7a8c5b0 2023-09-23T05:05:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.835+0000 7fcf55ffb700 10 client.27782 handle_command_reply: tid=0 2023-09-23T05:05:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.835+0000 7fcf881beb80 1 client.27782 shutdown 2023-09-23T05:05:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 msgr2=0x55c6a7a797c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a797c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ad70 tx=0x7fcf68004100 comp rx=0 tx=0).stop 2023-09-23T05:05:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 shutdown_connections 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7a8c5b0 0x55c6a7a8e9d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a79db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a7c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf55ffb700 0 client.27782 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3275348513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a797c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 >> 172.21.15.5:0/3275348513 conn(0x55c6a79e6260 msgr2=0x55c6a7a6d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 shutdown_connections 2023-09-23T05:05:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.836+0000 7fcf881beb80 1 -- 172.21.15.5:0/3275348513 wait complete. 2023-09-23T05:05:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.837+0000 7fcf881beb80 20 client.27782 trim_cache size 0 max 16384 2023-09-23T05:05:21.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.866+0000 7fcf881beb80 1 Processor -- start 2023-09-23T05:05:21.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.866+0000 7fcf881beb80 1 -- start start 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7845920 con 0x55c6a7a7cbd0 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7a6e5b0 con 0x55c6a7a71d20 2023-09-23T05:05:21.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf881beb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7a69db0 con 0x55c6a7a71400 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.867+0000 7fcf7259b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:59458/0 (socket says 172.21.15.5:59458) 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 -- 172.21.15.5:0/905261862 learned_addr learned my addr 172.21.15.5:0/905261862 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 -- 172.21.15.5:0/905261862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 msgr2=0x55c6a7a83750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a83750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 -- 172.21.15.5:0/905261862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 msgr2=0x55c6a7a6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf7259b700 1 -- 172.21.15.5:0/905261862 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a77d8c60 con 0x55c6a7a71d20 2023-09-23T05:05:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.868+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:05:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.869+0000 7fcf7259b700 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf68004e50 tx=0x7fcf6800aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b328688b79eb2c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.869+0000 7fcf617fa700 1 -- 172.21.15.5:0/905261862 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800fec0 con 0x55c6a7a71d20 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.869+0000 7fcf617fa700 1 -- 172.21.15.5:0/905261862 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6800a850 con 0x55c6a7a71d20 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.869+0000 7fcf617fa700 1 -- 172.21.15.5:0/905261862 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68022ce0 con 0x55c6a7a71d20 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.870+0000 7fcf881beb80 1 -- 172.21.15.5:0/905261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 msgr2=0x55c6a7a7c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.870+0000 7fcf881beb80 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf68004e50 tx=0x7fcf6800aca0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 -- 172.21.15.5:0/905261862 shutdown_connections 2023-09-23T05:05:21.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a71400 0x55c6a7a83750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 --2- 172.21.15.5:0/905261862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a7c0e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 -- 172.21.15.5:0/905261862 >> 172.21.15.5:0/905261862 conn(0x55c6a79e6260 msgr2=0x55c6a7a74af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 -- 172.21.15.5:0/905261862 shutdown_connections 2023-09-23T05:05:21.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.871+0000 7fcf881beb80 1 -- 172.21.15.5:0/905261862 wait complete. 2023-09-23T05:05:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.872+0000 7fcf881beb80 1 -- start start 2023-09-23T05:05:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.872+0000 7fcf881beb80 1 Processor -- start 2023-09-23T05:05:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a81c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7a83d40 con 0x55c6a7a71400 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7a82170 con 0x55c6a7a71d20 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a7a822e0 con 0x55c6a7a7cbd0 2023-09-23T05:05:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf881beb80 10 client.0 _ll_register_callbacks cb 0x7fcf6800abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:05:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf71d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf71d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50890/0 (socket says 172.21.15.5:50890) 2023-09-23T05:05:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.875+0000 7fcf71d9a700 1 -- 172.21.15.5:0/3426797993 learned_addr learned my addr 172.21.15.5:0/3426797993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.876+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a81c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.877+0000 7fcf71d9a700 1 -- 172.21.15.5:0/3426797993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a7cbd0 msgr2=0x55c6a7a81c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.877+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a81c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.877+0000 7fcf71d9a700 1 -- 172.21.15.5:0/3426797993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 msgr2=0x55c6a7a816f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.877+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a816f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.878+0000 7fcf71d9a700 1 -- 172.21.15.5:0/3426797993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6a77efde0 con 0x55c6a7a71400 2023-09-23T05:05:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.879+0000 7fcf71d9a700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0098e0 tx=0x7fcf5c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c431a40a1d4de2a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.879+0000 7fcf55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:05:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.881+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00bb10 con 0x55c6a7a71400 2023-09-23T05:05:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.881+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fcf5c024980 con 0x55c6a7a71400 2023-09-23T05:05:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.883+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a7810f70 con 0x55c6a7a71400 2023-09-23T05:05:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.886+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5c024b40 con 0x55c6a7a71400 2023-09-23T05:05:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.886+0000 7fcf881beb80 10 client.17745 fetch_fsmap learned FSMap version 79 2023-09-23T05:05:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.886+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55c6a7816250 con 0x55c6a7a71400 2023-09-23T05:05:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.887+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf5c035030 con 0x55c6a7a71400 2023-09-23T05:05:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf5c0099c0 con 0x55c6a7a71400 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 10 client.17745 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 20 client.17745 populate_metadata read hostname 'smithi005' 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7ae1770 0x55c6a7ae3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 4 client.17745 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:05:21.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.888+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6a7845f60 con 0x55c6a7ae1770 2023-09-23T05:05:21.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.889+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7ae1770 0x55c6a7ae3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:21.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.890+0000 7fcf7259b700 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7ae1770 0x55c6a7ae3b90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb0663a1956322 server_cookie=af32d064255629ce in_seq=0 out_seq=0 2023-09-23T05:05:21.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.890+0000 7fcf55ffb700 10 client.17745 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:21.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.893+0000 7fcf55ffb700 1 -- 172.21.15.5:0/3426797993 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6a7845f60 con 0x55c6a7ae1770 2023-09-23T05:05:21.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.893+0000 7fcf55ffb700 10 client.17745 handle_command_reply: tid=0 2023-09-23T05:05:21.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.893+0000 7fcf881beb80 1 client.17745 shutdown 2023-09-23T05:05:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 msgr2=0x55c6a7a83430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0098e0 tx=0x7fcf5c0058e0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 shutdown_connections 2023-09-23T05:05:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55c6a7ae1770 0x55c6a7ae3b90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55c6a7a7cbd0 0x55c6a7a81c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6a7a71400 0x55c6a7a83430 secure :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0098e0 tx=0x7fcf5c0058e0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 --2- 172.21.15.5:0/3426797993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c6a7a71d20 0x55c6a7a816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.894+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 >> 172.21.15.5:0/3426797993 conn(0x55c6a79e6260 msgr2=0x55c6a7a6d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.895+0000 7fcf55ffb700 0 client.17745 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.895+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 shutdown_connections 2023-09-23T05:05:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.896+0000 7fcf881beb80 1 -- 172.21.15.5:0/3426797993 wait complete. 2023-09-23T05:05:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.898+0000 7fcf881beb80 20 client.17745 trim_cache size 0 max 16384 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:05:21.899 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.898+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 msgr2=0x7fcf580a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.898+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 0x7fcf580a3d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf64095d20 tx=0x7fcf64004be0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.898+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 msgr2=0x7fcf6c07d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.898+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400b970 tx=0x7fcf6400bc80 comp rx=0 tx=0).stop 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 shutdown_connections 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcf580a1840 0x7fcf580a3d00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcf6c10d490 0x7fcf6c07d480 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf6c107e20 0x7fcf6c083420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 --2- 172.21.15.5:0/3024921220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf6c1087e0 0x7fcf6c083960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.899+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 >> 172.21.15.5:0/3024921220 conn(0x7fcf6c0725f0 msgr2=0x7fcf6c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.900+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 shutdown_connections 2023-09-23T05:05:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:21.900+0000 7fcf747ff700 1 -- 172.21.15.5:0/3024921220 wait complete. 2023-09-23T05:05:21.939 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:05:22.600 INFO:tasks.workunit.client.0.smithi005.stdout: 10 186638 26.58 MB/sec execute 237 sec latency 656.010 ms 2023-09-23T05:05:23.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:23 smithi118 ceph-mon[134297]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 556 op/s 2023-09-23T05:05:23.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 186730 26.49 MB/sec execute 238 sec latency 1384.844 ms 2023-09-23T05:05:23.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:23 smithi005 ceph-mon[97638]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 556 op/s 2023-09-23T05:05:23.807 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:23 smithi069 ceph-mon[133931]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 556 op/s 2023-09-23T05:05:24.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 186982 26.45 MB/sec execute 239 sec latency 1110.709 ms 2023-09-23T05:05:25.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:25 smithi069 ceph-mon[133931]: pgmap v601: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 525 op/s 2023-09-23T05:05:25.601 INFO:tasks.workunit.client.0.smithi005.stdout: 10 187222 26.39 MB/sec execute 240 sec latency 155.299 ms 2023-09-23T05:05:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:25 smithi005 ceph-mon[97638]: pgmap v601: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 525 op/s 2023-09-23T05:05:25.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:25 smithi118 ceph-mon[134297]: pgmap v601: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 525 op/s 2023-09-23T05:05:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:05:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:05:26.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:26 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:05:26.603 INFO:tasks.workunit.client.0.smithi005.stdout: 10 187692 26.40 MB/sec execute 241 sec latency 105.600 ms 2023-09-23T05:05:26.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:05:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:05:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:26 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:05:26.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:05:26.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:05:26.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:26 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:05:27.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:27 smithi069 ceph-mon[133931]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.9 MiB/s wr, 436 op/s 2023-09-23T05:05:27.603 INFO:tasks.workunit.client.0.smithi005.stdout: 10 188139 26.35 MB/sec execute 242 sec latency 92.368 ms 2023-09-23T05:05:27.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:27 smithi005 ceph-mon[97638]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.9 MiB/s wr, 436 op/s 2023-09-23T05:05:27.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:27 smithi118 ceph-mon[134297]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.9 MiB/s wr, 436 op/s 2023-09-23T05:05:28.605 INFO:tasks.workunit.client.0.smithi005.stdout: 10 188338 26.28 MB/sec execute 243 sec latency 618.333 ms 2023-09-23T05:05:29.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 188603 26.25 MB/sec execute 244 sec latency 174.354 ms 2023-09-23T05:05:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:30 smithi005 ceph-mon[97638]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-23T05:05:30.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:30 smithi118 ceph-mon[134297]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-23T05:05:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:30 smithi069 ceph-mon[133931]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 615 op/s 2023-09-23T05:05:30.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 189053 26.23 MB/sec execute 245 sec latency 161.808 ms 2023-09-23T05:05:31.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 189771 26.28 MB/sec execute 246 sec latency 108.250 ms 2023-09-23T05:05:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:31 smithi069 ceph-mon[133931]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 589 op/s 2023-09-23T05:05:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:31 smithi005 ceph-mon[97638]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 589 op/s 2023-09-23T05:05:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:31 smithi118 ceph-mon[134297]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 589 op/s 2023-09-23T05:05:32.610 INFO:tasks.workunit.client.0.smithi005.stdout: 10 190422 26.37 MB/sec execute 247 sec latency 96.613 ms 2023-09-23T05:05:33.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 190881 26.31 MB/sec execute 248 sec latency 97.205 ms 2023-09-23T05:05:34.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:34 smithi005 ceph-mon[97638]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 635 op/s 2023-09-23T05:05:34.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:34 smithi118 ceph-mon[134297]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 635 op/s 2023-09-23T05:05:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:34 smithi069 ceph-mon[133931]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 635 op/s 2023-09-23T05:05:34.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 191349 26.30 MB/sec execute 249 sec latency 122.817 ms 2023-09-23T05:05:35.608 INFO:tasks.workunit.client.0.smithi005.stdout: 10 191953 26.35 MB/sec execute 250 sec latency 98.250 ms 2023-09-23T05:05:36.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:36 smithi005 ceph-mon[97638]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 595 op/s 2023-09-23T05:05:36.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:36 smithi118 ceph-mon[134297]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 595 op/s 2023-09-23T05:05:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:36 smithi069 ceph-mon[133931]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 595 op/s 2023-09-23T05:05:36.609 INFO:tasks.workunit.client.0.smithi005.stdout: 10 192434 26.32 MB/sec execute 251 sec latency 109.727 ms 2023-09-23T05:05:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:37 smithi069 ceph-mon[133931]: pgmap v607: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.2 MiB/s wr, 502 op/s 2023-09-23T05:05:37.609 INFO:tasks.workunit.client.0.smithi005.stdout: 10 193004 26.33 MB/sec execute 252 sec latency 106.539 ms 2023-09-23T05:05:37.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:37 smithi005 ceph-mon[97638]: pgmap v607: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.2 MiB/s wr, 502 op/s 2023-09-23T05:05:37.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:37 smithi118 ceph-mon[134297]: pgmap v607: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.2 MiB/s wr, 502 op/s 2023-09-23T05:05:38.612 INFO:tasks.workunit.client.0.smithi005.stdout: 10 193654 26.40 MB/sec execute 253 sec latency 67.421 ms 2023-09-23T05:05:39.612 INFO:tasks.workunit.client.0.smithi005.stdout: 10 194138 26.42 MB/sec execute 254 sec latency 95.008 ms 2023-09-23T05:05:40.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:40 smithi005 ceph-mon[97638]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-09-23T05:05:40.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:40 smithi118 ceph-mon[134297]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-09-23T05:05:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:40 smithi069 ceph-mon[133931]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 655 op/s 2023-09-23T05:05:40.613 INFO:tasks.workunit.client.0.smithi005.stdout: 10 194594 26.38 MB/sec execute 255 sec latency 93.728 ms 2023-09-23T05:05:41.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 195177 26.41 MB/sec execute 256 sec latency 129.018 ms 2023-09-23T05:05:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:41 smithi069 ceph-mon[133931]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 549 op/s 2023-09-23T05:05:42.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:41 smithi005 ceph-mon[97638]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 549 op/s 2023-09-23T05:05:42.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:41 smithi118 ceph-mon[134297]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 549 op/s 2023-09-23T05:05:42.612 INFO:tasks.workunit.client.0.smithi005.stdout: 10 195495 26.38 MB/sec execute 257 sec latency 339.329 ms 2023-09-23T05:05:43.612 INFO:tasks.workunit.client.0.smithi005.stdout: 10 196036 26.35 MB/sec execute 258 sec latency 121.865 ms 2023-09-23T05:05:44.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:44 smithi005 ceph-mon[97638]: pgmap v610: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 532 op/s 2023-09-23T05:05:44.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:44 smithi118 ceph-mon[134297]: pgmap v610: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 532 op/s 2023-09-23T05:05:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:44 smithi069 ceph-mon[133931]: pgmap v610: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 532 op/s 2023-09-23T05:05:44.612 INFO:tasks.workunit.client.0.smithi005.stdout: 10 196759 26.40 MB/sec execute 259 sec latency 82.282 ms 2023-09-23T05:05:45.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 197371 26.47 MB/sec execute 260 sec latency 109.722 ms 2023-09-23T05:05:46.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:46 smithi005 ceph-mon[97638]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 481 op/s 2023-09-23T05:05:46.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:46 smithi118 ceph-mon[134297]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 481 op/s 2023-09-23T05:05:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:46 smithi069 ceph-mon[133931]: pgmap v611: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 481 op/s 2023-09-23T05:05:46.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 197863 26.45 MB/sec execute 261 sec latency 259.393 ms 2023-09-23T05:05:47.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 198237 26.42 MB/sec execute 262 sec latency 103.722 ms 2023-09-23T05:05:48.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:48 smithi005 ceph-mon[97638]: pgmap v612: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 390 op/s 2023-09-23T05:05:48.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:48 smithi118 ceph-mon[134297]: pgmap v612: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 390 op/s 2023-09-23T05:05:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:48 smithi069 ceph-mon[133931]: pgmap v612: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 390 op/s 2023-09-23T05:05:48.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 198835 26.47 MB/sec execute 263 sec latency 73.425 ms 2023-09-23T05:05:49.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 199408 26.45 MB/sec execute 264 sec latency 78.390 ms 2023-09-23T05:05:50.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:50 smithi005 ceph-mon[97638]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.6 MiB/s wr, 585 op/s 2023-09-23T05:05:50.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:50 smithi118 ceph-mon[134297]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.6 MiB/s wr, 585 op/s 2023-09-23T05:05:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:50 smithi069 ceph-mon[133931]: pgmap v613: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.6 MiB/s wr, 585 op/s 2023-09-23T05:05:50.614 INFO:tasks.workunit.client.0.smithi005.stdout: 10 200087 26.50 MB/sec execute 265 sec latency 79.189 ms 2023-09-23T05:05:51.615 INFO:tasks.workunit.client.0.smithi005.stdout: 10 200739 26.56 MB/sec execute 266 sec latency 75.115 ms 2023-09-23T05:05:51.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:05:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:51 smithi069 ceph-mon[133931]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:05:52.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.274+0000 7fd82edff700 1 -- 172.21.15.5:0/4047339405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075c70 msgr2=0x7fd8281117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.274+0000 7fd82edff700 1 --2- 172.21.15.5:0/4047339405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075c70 0x7fd8281117c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd814009990 tx=0x7fd814009ca0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.280+0000 7fd82edff700 1 -- 172.21.15.5:0/4047339405 shutdown_connections 2023-09-23T05:05:52.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.280+0000 7fd82edff700 1 --2- 172.21.15.5:0/4047339405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.280+0000 7fd82edff700 1 --2- 172.21.15.5:0/4047339405 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075350 0x7fd828075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.280+0000 7fd82edff700 1 --2- 172.21.15.5:0/4047339405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075c70 0x7fd8281117c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.280+0000 7fd82edff700 1 -- 172.21.15.5:0/4047339405 >> 172.21.15.5:0/4047339405 conn(0x7fd8280feb50 msgr2=0x7fd828100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.281+0000 7fd82edff700 1 -- 172.21.15.5:0/4047339405 shutdown_connections 2023-09-23T05:05:52.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.281+0000 7fd82edff700 1 -- 172.21.15.5:0/4047339405 wait complete. 2023-09-23T05:05:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.282+0000 7fd82edff700 1 Processor -- start 2023-09-23T05:05:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.282+0000 7fd82edff700 1 -- start start 2023-09-23T05:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 0x7fd82819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281a2570 con 0x7fd828111d00 2023-09-23T05:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281a26e0 con 0x7fd828075350 2023-09-23T05:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.283+0000 7fd82edff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281a28a0 con 0x7fd828075c70 2023-09-23T05:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.284+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.284+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:57736/0 (socket says 172.21.15.5:57736) 2023-09-23T05:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.284+0000 7fd82cb9b700 1 -- 172.21.15.5:0/3448275140 learned_addr learned my addr 172.21.15.5:0/3448275140 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.284+0000 7fd827fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 0x7fd82819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.284+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 -- 172.21.15.5:0/3448275140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 msgr2=0x7fd82819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 -- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 msgr2=0x7fd8281a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 -- 172.21.15.5:0/3448275140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8140095f0 con 0x7fd828075c70 2023-09-23T05:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:05:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.285+0000 7fd827fff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 0x7fd82819dad0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd814000c00 tx=0x7fd814005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.286+0000 7fd825ffb700 1 -- 172.21.15.5:0/3448275140 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81401d070 con 0x7fd828075c70 2023-09-23T05:05:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.286+0000 7fd825ffb700 1 -- 172.21.15.5:0/3448275140 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd814021410 con 0x7fd828075c70 2023-09-23T05:05:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.286+0000 7fd825ffb700 1 -- 172.21.15.5:0/3448275140 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814017650 con 0x7fd828075c70 2023-09-23T05:05:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.286+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8281a2b20 con 0x7fd828075c70 2023-09-23T05:05:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.286+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281a2da0 con 0x7fd828075c70 2023-09-23T05:05:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.292+0000 7fd84266bb80 1 Processor -- start 2023-09-23T05:05:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.294+0000 7fd84266bb80 1 -- start start 2023-09-23T05:05:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb5f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cac8460 con 0x55623cb6abd0 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cab9c20 con 0x55623cb5f400 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd84266bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cab7540 con 0x55623cb5fd20 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:34240/0 (socket says 172.21.15.5:34240) 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.298+0000 7fd82cb9b700 1 -- 172.21.15.5:0/1075978914 learned_addr learned my addr 172.21.15.5:0/1075978914 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82d39c700 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb5f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 -- 172.21.15.5:0/1075978914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 msgr2=0x55623cb5f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb5f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 -- 172.21.15.5:0/1075978914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 msgr2=0x55623cb6cfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T05:05:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 -- 172.21.15.5:0/1075978914 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55623cb158d0 con 0x55623cb5fd20 2023-09-23T05:05:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.299+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0094b0 tx=0x7fd81c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3125f0d51f355248 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.300+0000 7fd825ffb700 1 -- 172.21.15.5:0/3448275140 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd814021580 con 0x7fd828075c70 2023-09-23T05:05:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.300+0000 7fd812ffd700 1 -- 172.21.15.5:0/1075978914 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c003c00 con 0x55623cb5fd20 2023-09-23T05:05:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.300+0000 7fd812ffd700 1 -- 172.21.15.5:0/1075978914 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c00f040 con 0x55623cb5fd20 2023-09-23T05:05:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.300+0000 7fd812ffd700 1 -- 172.21.15.5:0/1075978914 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c004450 con 0x55623cb5fd20 2023-09-23T05:05:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 -- 172.21.15.5:0/1075978914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 msgr2=0x55623cb6a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0094b0 tx=0x7fd81c009920 comp rx=0 tx=0).stop 2023-09-23T05:05:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 -- 172.21.15.5:0/1075978914 shutdown_connections 2023-09-23T05:05:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb5f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 --2- 172.21.15.5:0/1075978914 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6a690 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.301+0000 7fd84266bb80 1 -- 172.21.15.5:0/1075978914 >> 172.21.15.5:0/1075978914 conn(0x55623cad4260 msgr2=0x55623cb5b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.302+0000 7fd84266bb80 1 -- 172.21.15.5:0/1075978914 shutdown_connections 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.302+0000 7fd84266bb80 1 -- 172.21.15.5:0/1075978914 wait complete. 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.303+0000 7fd84266bb80 1 -- start start 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.303+0000 7fd84266bb80 1 Processor -- start 2023-09-23T05:05:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 0x55623cb679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb67f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 0x55623cb68440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cab2e30 con 0x55623cb5f400 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca4f710 con 0x55623cb6abd0 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623ca4f3b0 con 0x55623cb5fd20 2023-09-23T05:05:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd825ffb700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 0x7fd8180a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd84266bb80 10 client.0 _ll_register_callbacks cb 0x55623c8ac800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd82d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 0x55623cb679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd825ffb700 1 -- 172.21.15.5:0/3448275140 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd8140ce260 con 0x7fd828075c70 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd82d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 0x55623cb679c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34976/0 (socket says 172.21.15.5:34976) 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.304+0000 7fd82d39c700 1 -- 172.21.15.5:0/3551835126 learned_addr learned my addr 172.21.15.5:0/3551835126 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 0x55623cb68440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 0x7fd8180a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 -- 172.21.15.5:0/3551835126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 msgr2=0x55623cb67f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb67f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 -- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 msgr2=0x55623cb679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 0x55623cb679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.305+0000 7fd827fff700 1 -- 172.21.15.5:0/3551835126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c009190 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd827fff700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 0x55623cb68440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd8140cb520 tx=0x7fd81409ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1caa299cdc7bbd70 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd807fff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 0x7fd8180a3c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd82000b640 tx=0x7fd82000b390 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81409b230 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd81409b4d0 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55623cb53250 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8140cb9b0 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd84266bb80 10 client.27797 fetch_fsmap learned FSMap version 79 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.306+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55623c8ddde0 con 0x55623cb6abd0 2023-09-23T05:05:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8140b2030 con 0x55623cb6abd0 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8140b3240 con 0x55623cb6abd0 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 10 client.27797 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 10 client.27797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 20 client.27797 populate_metadata read hostname 'smithi005' 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cb7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 4 client.27797 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55623caabe10 con 0x55623cb7a370 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.307+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cb7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.308+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cb7c790 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7515e8a00e60de98 server_cookie=24d20bb120b39d3b in_seq=0 out_seq=0 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.308+0000 7fd807fff700 10 client.27797 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.308+0000 7fd807fff700 1 -- 172.21.15.5:0/3551835126 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55623caabe10 con 0x55623cb7a370 2023-09-23T05:05:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.308+0000 7fd807fff700 10 client.27797 handle_command_reply: tid=0 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 client.27797 shutdown 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 msgr2=0x55623cb68440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 0x55623cb68440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd8140cb520 tx=0x7fd81409ac50 comp rx=0 tx=0).stop 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 shutdown_connections 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cb7c790 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb6abd0 0x55623cb68440 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd807fff700 0 client.27797 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb5f400 0x55623cb679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3551835126 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb67f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 >> 172.21.15.5:0/3551835126 conn(0x55623cad4260 msgr2=0x55623cb5b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 shutdown_connections 2023-09-23T05:05:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 1 -- 172.21.15.5:0/3551835126 wait complete. 2023-09-23T05:05:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.309+0000 7fd84266bb80 20 client.27797 trim_cache size 0 max 16384 2023-09-23T05:05:52.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.329+0000 7fd84266bb80 1 Processor -- start 2023-09-23T05:05:52.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.329+0000 7fd84266bb80 1 -- start start 2023-09-23T05:05:52.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5f400 0x55623cb61ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5fd20 0x55623cb64c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623c933920 con 0x55623cb6abd0 2023-09-23T05:05:52.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cb5c450 con 0x55623cb5fd20 2023-09-23T05:05:52.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd84266bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cb5c630 con 0x55623cb5f400 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd827fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5fd20 0x55623cb64c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34984/0 (socket says 172.21.15.5:34984) 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 -- 172.21.15.5:0/2644055973 learned_addr learned my addr 172.21.15.5:0/2644055973 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 -- 172.21.15.5:0/2644055973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5f400 msgr2=0x55623cb61ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5f400 0x55623cb61ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 -- 172.21.15.5:0/2644055973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5fd20 msgr2=0x55623cb64c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5fd20 0x55623cb64c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.330+0000 7fd82d39c700 1 -- 172.21.15.5:0/2644055973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55623cb57480 con 0x55623cb6abd0 2023-09-23T05:05:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.331+0000 7fd82d39c700 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd820023400 tx=0x7fd82000bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df77ace2d62401e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.331+0000 7fd8137fe700 1 -- 172.21.15.5:0/2644055973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000fd30 con 0x55623cb6abd0 2023-09-23T05:05:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.331+0000 7fd8137fe700 1 -- 172.21.15.5:0/2644055973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82000fe90 con 0x55623cb6abd0 2023-09-23T05:05:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.331+0000 7fd8137fe700 1 -- 172.21.15.5:0/2644055973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820005280 con 0x55623cb6abd0 2023-09-23T05:05:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- 172.21.15.5:0/2644055973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 msgr2=0x55623cb65170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd820023400 tx=0x7fd82000bba0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- 172.21.15.5:0/2644055973 shutdown_connections 2023-09-23T05:05:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb65170 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5f400 0x55623cb61ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 --2- 172.21.15.5:0/2644055973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5fd20 0x55623cb64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- 172.21.15.5:0/2644055973 >> 172.21.15.5:0/2644055973 conn(0x55623cad4260 msgr2=0x55623cb613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- 172.21.15.5:0/2644055973 shutdown_connections 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- 172.21.15.5:0/2644055973 wait complete. 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.332+0000 7fd84266bb80 1 -- start start 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 Processor -- start 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb71630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cb70470 con 0x55623cb6abd0 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cb705b0 con 0x55623cb5f400 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623cb70770 con 0x55623cb5fd20 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd84266bb80 10 client.0 _ll_register_callbacks cb 0x55623c8ac800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd82d39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd82cb9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb71630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:57776/0 (socket says 172.21.15.5:57776) 2023-09-23T05:05:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.333+0000 7fd82cb9b700 1 -- 172.21.15.5:0/3965833522 learned_addr learned my addr 172.21.15.5:0/3965833522 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:05:52.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 -- 172.21.15.5:0/3965833522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 msgr2=0x55623cb6f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 -- 172.21.15.5:0/3965833522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 msgr2=0x55623cb71630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb71630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 -- 172.21.15.5:0/3965833522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8200230e0 con 0x55623cb6abd0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd82d39c700 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6fea0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd82000bd30 tx=0x7fd82000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9df6233c9d66f01 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd807fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820004220 con 0x55623cb6abd0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd820004380 con 0x55623cb6abd0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.334+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd81c00f040 con 0x55623cb6abd0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd820004540 con 0x55623cb6abd0 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 10 client.17760 fetch_fsmap learned FSMap version 79 2023-09-23T05:05:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x55623c8c6c60 con 0x55623cb6abd0 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81c00f040 con 0x55623cb6abd0 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd82003f070 con 0x55623cb6abd0 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 10 client.17760 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 20 client.17760 populate_metadata read hostname 'smithi005' 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cbd1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 4 client.17760 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:05:52.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.335+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55623caabb30 con 0x55623cb7a370 2023-09-23T05:05:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:51 smithi005 ceph-mon[97638]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:05:52.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:51 smithi118 ceph-mon[134297]: pgmap v614: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.8 MiB/s wr, 483 op/s 2023-09-23T05:05:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.581+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cbd1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:05:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.582+0000 7fd82cb9b700 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cbd1620 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74aa2ffb4fc1511d server_cookie=cdf7e0a6619f8764 in_seq=0 out_seq=0 2023-09-23T05:05:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.582+0000 7fd807fff700 10 client.17760 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:52.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.584+0000 7fd807fff700 1 -- 172.21.15.5:0/3965833522 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55623caabb30 con 0x55623cb7a370 2023-09-23T05:05:52.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.584+0000 7fd807fff700 10 client.17760 handle_command_reply: tid=0 2023-09-23T05:05:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.587+0000 7fd84266bb80 1 client.17760 shutdown 2023-09-23T05:05:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 msgr2=0x55623cb6fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6fea0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd82000bd30 tx=0x7fd82000fd60 comp rx=0 tx=0).stop 2023-09-23T05:05:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 shutdown_connections 2023-09-23T05:05:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55623cb7a370 0x55623cbd1620 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55623cb6abd0 0x55623cb6fea0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55623cb5f400 0x55623cb71630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 --2- 172.21.15.5:0/3965833522 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55623cb5fd20 0x55623cb6f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 >> 172.21.15.5:0/3965833522 conn(0x55623cad4260 msgr2=0x55623cb681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.590+0000 7fd807fff700 0 client.17760 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.591+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 shutdown_connections 2023-09-23T05:05:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.591+0000 7fd84266bb80 1 -- 172.21.15.5:0/3965833522 wait complete. 2023-09-23T05:05:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.592+0000 7fd84266bb80 20 client.17760 trim_cache size 0 max 16384 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:05:52.593 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:05:52.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:05:52.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:05:52.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:05:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.602+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 msgr2=0x7fd8180a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.602+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 0x7fd8180a3c70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd82000b640 tx=0x7fd82000b390 comp rx=0 tx=0).stop 2023-09-23T05:05:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.602+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 msgr2=0x7fd82819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:05:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.602+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 0x7fd82819dad0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd814000c00 tx=0x7fd814005740 comp rx=0 tx=0).stop 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 shutdown_connections 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd8180a17b0 0x7fd8180a3c70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd828111d00 0x7fd8281a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd828075350 0x7fd82819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 --2- 172.21.15.5:0/3448275140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd828075c70 0x7fd82819dad0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:05:52.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.606+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 >> 172.21.15.5:0/3448275140 conn(0x7fd8280feb50 msgr2=0x7fd828100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:05:52.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.612+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 shutdown_connections 2023-09-23T05:05:52.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:05:52.612+0000 7fd82edff700 1 -- 172.21.15.5:0/3448275140 wait complete. 2023-09-23T05:05:52.615 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201156 26.52 MB/sec execute 267 sec latency 288.937 ms 2023-09-23T05:05:52.647 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:05:53.617 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201337 26.45 MB/sec execute 268 sec latency 454.990 ms 2023-09-23T05:05:54.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:54 smithi118 ceph-mon[134297]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 519 op/s 2023-09-23T05:05:54.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:54 smithi005 ceph-mon[97638]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 519 op/s 2023-09-23T05:05:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:54 smithi069 ceph-mon[133931]: pgmap v615: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.3 MiB/s wr, 519 op/s 2023-09-23T05:05:54.623 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201356 26.36 MB/sec execute 269 sec latency 1381.378 ms 2023-09-23T05:05:55.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:55 smithi069 ceph-mon[133931]: pgmap v616: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.7 MiB/s wr, 537 op/s 2023-09-23T05:05:55.623 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201446 26.28 MB/sec execute 270 sec latency 1837.869 ms 2023-09-23T05:05:55.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:55 smithi005 ceph-mon[97638]: pgmap v616: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.7 MiB/s wr, 537 op/s 2023-09-23T05:05:55.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:55 smithi118 ceph-mon[134297]: pgmap v616: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.7 MiB/s wr, 537 op/s 2023-09-23T05:05:56.623 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201604 26.21 MB/sec execute 271 sec latency 190.728 ms 2023-09-23T05:05:57.626 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201758 26.15 MB/sec execute 272 sec latency 356.201 ms 2023-09-23T05:05:58.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:58 smithi005 ceph-mon[97638]: pgmap v617: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 497 op/s 2023-09-23T05:05:58.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:58 smithi118 ceph-mon[134297]: pgmap v617: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 497 op/s 2023-09-23T05:05:58.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:58 smithi069 ceph-mon[133931]: pgmap v617: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.6 MiB/s wr, 497 op/s 2023-09-23T05:05:58.626 INFO:tasks.workunit.client.0.smithi005.stdout: 10 201996 26.10 MB/sec execute 273 sec latency 368.500 ms 2023-09-23T05:05:59.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:05:59 smithi069 ceph-mon[133931]: pgmap v618: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 646 op/s 2023-09-23T05:05:59.627 INFO:tasks.workunit.client.0.smithi005.stdout: 10 202119 26.04 MB/sec execute 274 sec latency 636.240 ms 2023-09-23T05:05:59.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:05:59 smithi005 ceph-mon[97638]: pgmap v618: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 646 op/s 2023-09-23T05:05:59.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:05:59 smithi118 ceph-mon[134297]: pgmap v618: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 646 op/s 2023-09-23T05:06:00.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 202377 25.97 MB/sec execute 275 sec latency 246.893 ms 2023-09-23T05:06:01.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 202692 25.90 MB/sec execute 276 sec latency 326.121 ms 2023-09-23T05:06:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:01 smithi069 ceph-mon[133931]: pgmap v619: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 597 op/s 2023-09-23T05:06:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:01 smithi005 ceph-mon[97638]: pgmap v619: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 597 op/s 2023-09-23T05:06:02.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:01 smithi118 ceph-mon[134297]: pgmap v619: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 597 op/s 2023-09-23T05:06:02.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 202859 25.84 MB/sec execute 277 sec latency 396.567 ms 2023-09-23T05:06:03.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 202942 25.77 MB/sec execute 278 sec latency 423.523 ms 2023-09-23T05:06:04.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:04 smithi005 ceph-mon[97638]: pgmap v620: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 690 op/s 2023-09-23T05:06:04.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:04 smithi118 ceph-mon[134297]: pgmap v620: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 690 op/s 2023-09-23T05:06:04.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:04 smithi069 ceph-mon[133931]: pgmap v620: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 690 op/s 2023-09-23T05:06:04.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 203009 25.70 MB/sec execute 279 sec latency 401.276 ms 2023-09-23T05:06:05.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 203100 25.64 MB/sec execute 280 sec latency 392.158 ms 2023-09-23T05:06:06.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:06 smithi005 ceph-mon[97638]: pgmap v621: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 649 op/s 2023-09-23T05:06:06.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:06 smithi118 ceph-mon[134297]: pgmap v621: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 649 op/s 2023-09-23T05:06:06.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:06 smithi069 ceph-mon[133931]: pgmap v621: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 649 op/s 2023-09-23T05:06:06.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 203268 25.57 MB/sec execute 281 sec latency 633.829 ms 2023-09-23T05:06:07.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:07 smithi005 ceph-mon[97638]: osdmap e117: 12 total, 12 up, 12 in 2023-09-23T05:06:07.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:07 smithi118 ceph-mon[134297]: osdmap e117: 12 total, 12 up, 12 in 2023-09-23T05:06:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:07 smithi069 ceph-mon[133931]: osdmap e117: 12 total, 12 up, 12 in 2023-09-23T05:06:07.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 203868 25.63 MB/sec execute 282 sec latency 765.878 ms 2023-09-23T05:06:08.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:08 smithi005 ceph-mon[97638]: pgmap v623: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 631 op/s 2023-09-23T05:06:08.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:08 smithi118 ceph-mon[134297]: pgmap v623: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 631 op/s 2023-09-23T05:06:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:08 smithi069 ceph-mon[133931]: pgmap v623: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 13 MiB/s wr, 631 op/s 2023-09-23T05:06:08.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 204521 25.70 MB/sec execute 283 sec latency 85.027 ms 2023-09-23T05:06:08.879 INFO:tasks.workunit.client.0.smithi005.stdout:[139892] write failed on handle 10221 (Resource temporarily unavailable) 2023-09-23T05:06:08.880 INFO:tasks.workunit.client.0.smithi005.stdout:Child failed with status 1 2023-09-23T05:06:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:06:08.886 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-09-23T05:06:08.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-09-23T05:06:10.141 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-09-23T05:06:11.863 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=be869081a06548ce91c468e53c6dbbe4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-09-23T05:06:11.866 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-09-23T05:06:11.878 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-09-23T05:06:11.878 INFO:teuthology.task.exec:Executing custom commands... 2023-09-23T05:06:11.878 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T05:06:11.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-09-23T05:06:11.892 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:10 smithi069 ceph-mon[133931]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.1 MiB/s wr, 615 op/s 2023-09-23T05:06:11.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:10 smithi118 ceph-mon[134297]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.1 MiB/s wr, 615 op/s 2023-09-23T05:06:11.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:10 smithi005 ceph-mon[97638]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 8.1 MiB/s wr, 615 op/s 2023-09-23T05:06:11.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-09-23T05:06:12.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-23T05:06:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:11 smithi069 ceph-mon[133931]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 518 op/s 2023-09-23T05:06:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.277+0000 7f76fa9ff700 1 -- 172.21.15.5:0/4056029746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4073ae0 msgr2=0x7f76f4111890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.277+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/4056029746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4073ae0 0x7f76f4111890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f76e80098f0 tx=0x7f76e8009c00 comp rx=0 tx=0).stop 2023-09-23T05:06:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 -- 172.21.15.5:0/4056029746 shutdown_connections 2023-09-23T05:06:12.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/4056029746 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 0x7f76f41141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/4056029746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f40731c0 0x7f76f40735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/4056029746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4073ae0 0x7f76f4111890 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 -- 172.21.15.5:0/4056029746 >> 172.21.15.5:0/4056029746 conn(0x7f76f40fc8d0 msgr2=0x7f76f40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 -- 172.21.15.5:0/4056029746 shutdown_connections 2023-09-23T05:06:12.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.278+0000 7f76fa9ff700 1 -- 172.21.15.5:0/4056029746 wait complete. 2023-09-23T05:06:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 Processor -- start 2023-09-23T05:06:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 -- start start 2023-09-23T05:06:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f4073ae0 0x7f76f419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 0x7f76f41a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a2600 con 0x7f76f40731c0 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a2770 con 0x7f76f4073ae0 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.279+0000 7f76fa9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a2930 con 0x7f76f4111dd0 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f4073ae0 0x7f76f419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40024/0 (socket says 172.21.15.5:40024) 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 -- 172.21.15.5:0/2129670097 learned_addr learned my addr 172.21.15.5:0/2129670097 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f8f9c700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 0x7f76f41a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 -- 172.21.15.5:0/2129670097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 msgr2=0x7f76f41a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 0x7f76f41a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 -- 172.21.15.5:0/2129670097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f4073ae0 msgr2=0x7f76f419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f4073ae0 0x7f76f419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 -- 172.21.15.5:0/2129670097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e8009520 con 0x7f76f40731c0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f3fff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f76e400e9c0 tx=0x7f76e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.280+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4004a30 con 0x7f76f40731c0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.281+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76e400cc80 con 0x7f76f40731c0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.281+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f41a2c10 con 0x7f76f40731c0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.281+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40054e0 con 0x7f76f40731c0 2023-09-23T05:06:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.281+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41aa450 con 0x7f76f40731c0 2023-09-23T05:06:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.282+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f41042d0 con 0x7f76f40731c0 2023-09-23T05:06:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.284+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f76e4004b90 con 0x7f76f40731c0 2023-09-23T05:06:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.285+0000 7f76f17fa700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 0x7f76dc0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.285+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f76e4014070 con 0x7f76f40731c0 2023-09-23T05:06:12.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.285+0000 7f76f37fe700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 0x7f76dc0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.286+0000 7f76f37fe700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 0x7f76dc0a3c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005e90 tx=0x7f76e8019040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.290+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f76e4096620 con 0x7f76f40731c0 2023-09-23T05:06:12.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:11 smithi005 ceph-mon[97638]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 518 op/s 2023-09-23T05:06:12.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:11 smithi118 ceph-mon[134297]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 518 op/s 2023-09-23T05:06:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.532+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f76f41173a0 con 0x7f76dc0a1790 2023-09-23T05:06:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.545+0000 7f76f17fa700 1 -- 172.21.15.5:0/2129670097 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f76f41173a0 con 0x7f76dc0a1790 2023-09-23T05:06:12.546 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-09-23T00:00:00", "created": "2023-09-23T04:46:17", "first": "2023-09-23T04:47:00", "last": "2023-09-23T05:06:00", "last_pruned": "2023-09-23T05:06:00", "created_count": 20, "pruned_count": 13, "active": true} 2023-09-23T05:06:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 msgr2=0x7f76dc0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 0x7f76dc0a3c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f76e8005e90 tx=0x7f76e8019040 comp rx=0 tx=0).stop 2023-09-23T05:06:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 msgr2=0x7f76f419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f76e400e9c0 tx=0x7f76e400ecd0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 shutdown_connections 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76dc0a1790 0x7f76dc0a3c50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76f4111dd0 0x7f76f41a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f40731c0 0x7f76f419d620 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 --2- 172.21.15.5:0/2129670097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76f4073ae0 0x7f76f419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 >> 172.21.15.5:0/2129670097 conn(0x7f76f40fc8d0 msgr2=0x7f76f40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 shutdown_connections 2023-09-23T05:06:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.548+0000 7f76fa9ff700 1 -- 172.21.15.5:0/2129670097 wait complete. 2023-09-23T05:06:12.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-23T05:06:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 -- 172.21.15.5:0/4121386278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f810b3a0 msgr2=0x7f99f810d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 --2- 172.21.15.5:0/4121386278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f810b3a0 0x7f99f810d860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f99f000b0a0 tx=0x7f99f000b3b0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 -- 172.21.15.5:0/4121386278 shutdown_connections 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 --2- 172.21.15.5:0/4121386278 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f810b3a0 0x7f99f810d860 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 --2- 172.21.15.5:0/4121386278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 0x7f99f8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 --2- 172.21.15.5:0/4121386278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f81028a0 0x7f99f810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 -- 172.21.15.5:0/4121386278 >> 172.21.15.5:0/4121386278 conn(0x7f99f80fb7c0 msgr2=0x7f99f80fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 -- 172.21.15.5:0/4121386278 shutdown_connections 2023-09-23T05:06:12.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.770+0000 7f9a007ff700 1 -- 172.21.15.5:0/4121386278 wait complete. 2023-09-23T05:06:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.771+0000 7f9a007ff700 1 Processor -- start 2023-09-23T05:06:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.771+0000 7f9a007ff700 1 -- start start 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 0x7f99f8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 0x7f99f819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f819bff0 con 0x7f99f810b3a0 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f819c160 con 0x7f99f8101f80 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f9a007ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f819c320 con 0x7f99f81028a0 2023-09-23T05:06:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fdd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fdd9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37084/0 (socket says 172.21.15.5:37084) 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fdd9a700 1 -- 172.21.15.5:0/808193786 learned_addr learned my addr 172.21.15.5:0/808193786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fed9c700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 0x7f99f819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fe59b700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 0x7f99f8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fed9c700 1 -- 172.21.15.5:0/808193786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 msgr2=0x7f99f8197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fed9c700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fed9c700 1 -- 172.21.15.5:0/808193786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 msgr2=0x7f99f8197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.772+0000 7f99fed9c700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 0x7f99f8197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99fed9c700 1 -- 172.21.15.5:0/808193786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0009d00 con 0x7f99f810b3a0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99fdd9a700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99fed9c700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 0x7f99f819b9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009fd0 tx=0x7f99f0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f000e040 con 0x7f99f810b3a0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f819c5a0 con 0x7f99f810b3a0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99f001d470 con 0x7f99f810b3a0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00052a0 con 0x7f99f810b3a0 2023-09-23T05:06:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.773+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f81a3e40 con 0x7f99f810b3a0 2023-09-23T05:06:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.774+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f804eb20 con 0x7f99f810b3a0 2023-09-23T05:06:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.776+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f99f001d5e0 con 0x7f99f810b3a0 2023-09-23T05:06:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.777+0000 7f99ef7fe700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 0x7f99e40a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.778+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9681+0+0 (secure 0 0 0) 0x7f99f00cea10 con 0x7f99f810b3a0 2023-09-23T05:06:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.778+0000 7f99fe59b700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 0x7f99e40a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.778+0000 7f99fe59b700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 0x7f99e40a3c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c60 tx=0x7f99f4009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:12.781+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f99f0097860 con 0x7f99f810b3a0 2023-09-23T05:06:13.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.021+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 --> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f99f8110f70 con 0x7f99e40a1790 2023-09-23T05:06:13.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.034+0000 7f99ef7fe700 1 -- 172.21.15.5:0/808193786 <== mgr.15594 v2:172.21.15.5:6800/2064728089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f99f8110f70 con 0x7f99e40a1790 2023-09-23T05:06:13.036 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2023-09-23T05:06:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 msgr2=0x7f99e40a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 0x7f99e40a3c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009c60 tx=0x7f99f4009260 comp rx=0 tx=0).stop 2023-09-23T05:06:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 msgr2=0x7f99f819b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 0x7f99f819b9b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009fd0 tx=0x7f99f0007930 comp rx=0 tx=0).stop 2023-09-23T05:06:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 shutdown_connections 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f99e40a1790 0x7f99e40a3c50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f810b3a0 0x7f99f819b9b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f8101f80 0x7f99f8197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 --2- 172.21.15.5:0/808193786 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f99f81028a0 0x7f99f8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 >> 172.21.15.5:0/808193786 conn(0x7f99f80fb7c0 msgr2=0x7f99f810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 shutdown_connections 2023-09-23T05:06:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:13.037+0000 7f9a007ff700 1 -- 172.21.15.5:0/808193786 wait complete. 2023-09-23T05:06:13.057 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-23T05:06:13.067 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-23T05:06:14.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:14 smithi005 ceph-mon[97638]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:14 smithi005 ceph-mon[97638]: from='client.17772 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:14 smithi005 ceph-mon[97638]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 385 op/s 2023-09-23T05:06:14.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:14 smithi118 ceph-mon[134297]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:14 smithi118 ceph-mon[134297]: from='client.17772 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:14 smithi118 ceph-mon[134297]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 385 op/s 2023-09-23T05:06:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:14 smithi069 ceph-mon[133931]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:14 smithi069 ceph-mon[133931]: from='client.17772 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-23T05:06:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:14 smithi069 ceph-mon[133931]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 385 op/s 2023-09-23T05:06:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:16 smithi005 ceph-mon[97638]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2023-09-23T05:06:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:16 smithi005 ceph-mon[97638]: osdmap e118: 12 total, 12 up, 12 in 2023-09-23T05:06:16.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:16 smithi118 ceph-mon[134297]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2023-09-23T05:06:16.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:16 smithi118 ceph-mon[134297]: osdmap e118: 12 total, 12 up, 12 in 2023-09-23T05:06:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:16 smithi069 ceph-mon[133931]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 310 op/s 2023-09-23T05:06:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:16 smithi069 ceph-mon[133931]: osdmap e118: 12 total, 12 up, 12 in 2023-09-23T05:06:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:18 smithi005 ceph-mon[97638]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2023-09-23T05:06:18.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:18 smithi118 ceph-mon[134297]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2023-09-23T05:06:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:18 smithi069 ceph-mon[133931]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2023-09-23T05:06:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:20 smithi005 ceph-mon[97638]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T05:06:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:20 smithi118 ceph-mon[134297]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T05:06:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:20 smithi069 ceph-mon[133931]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 138 op/s 2023-09-23T05:06:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:21 smithi069 ceph-mon[133931]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 KiB/s wr, 66 op/s 2023-09-23T05:06:22.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:21 smithi005 ceph-mon[97638]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 KiB/s wr, 66 op/s 2023-09-23T05:06:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:21 smithi118 ceph-mon[134297]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 885 KiB/s wr, 66 op/s 2023-09-23T05:06:22.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:06:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 -- 172.21.15.5:0/620494212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9434069a50 msgr2=0x7f9434105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 --2- 172.21.15.5:0/620494212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9434069a50 0x7f9434105ae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424009ca0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 -- 172.21.15.5:0/620494212 shutdown_connections 2023-09-23T05:06:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 --2- 172.21.15.5:0/620494212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 0x7f943410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 --2- 172.21.15.5:0/620494212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94340690a0 0x7f9434069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 --2- 172.21.15.5:0/620494212 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f9434069a50 0x7f9434105ae0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 -- 172.21.15.5:0/620494212 >> 172.21.15.5:0/620494212 conn(0x7f9434077ca0 msgr2=0x7f94340780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 -- 172.21.15.5:0/620494212 shutdown_connections 2023-09-23T05:06:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.866+0000 7f943b7ff700 1 -- 172.21.15.5:0/620494212 wait complete. 2023-09-23T05:06:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.867+0000 7f943b7ff700 1 Processor -- start 2023-09-23T05:06:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 -- start start 2023-09-23T05:06:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 0x7f9434199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 0x7f943419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419e670 con 0x7f9434106210 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419e7e0 con 0x7f943419dbf0 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943b7ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419e9a0 con 0x7f94340690a0 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943959b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943959b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37738/0 (socket says 172.21.15.5:37738) 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f943959b700 1 -- 172.21.15.5:0/3531179145 learned_addr learned my addr 172.21.15.5:0/3531179145 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f9439d9c700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 0x7f943419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.868+0000 7f9438d9a700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 0x7f9434199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 -- 172.21.15.5:0/3531179145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 msgr2=0x7f943419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 0x7f943419e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 -- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 msgr2=0x7f9434199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 0x7f9434199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 -- 172.21.15.5:0/3531179145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240095f0 con 0x7f94340690a0 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f9439d9c700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 0x7f943419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943959b700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9a0 tx=0x7f943000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f942a7fc700 1 -- 172.21.15.5:0/3531179145 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430009790 con 0x7f94340690a0 2023-09-23T05:06:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f942a7fc700 1 -- 172.21.15.5:0/3531179145 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9430010430 con 0x7f94340690a0 2023-09-23T05:06:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94341a5f00 con 0x7f94340690a0 2023-09-23T05:06:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.869+0000 7f942a7fc700 1 -- 172.21.15.5:0/3531179145 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430018680 con 0x7f94340690a0 2023-09-23T05:06:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.870+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94341a6380 con 0x7f94340690a0 2023-09-23T05:06:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.873+0000 7f942a7fc700 1 -- 172.21.15.5:0/3531179145 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f94300105a0 con 0x7f94340690a0 2023-09-23T05:06:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 Processor -- start 2023-09-23T05:06:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 -- start start 2023-09-23T05:06:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afba15460 con 0x558afbaac400 2023-09-23T05:06:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afba06c20 con 0x558afbab7bd0 2023-09-23T05:06:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.874+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afba04540 con 0x558afbaacd20 2023-09-23T05:06:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9439d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f942a7fc700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 0x7f94200a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9439d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41362/0 (socket says 172.21.15.5:41362) 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9439d9c700 1 -- 172.21.15.5:0/3559176553 learned_addr learned my addr 172.21.15.5:0/3559176553 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f942a7fc700 1 -- 172.21.15.5:0/3531179145 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9430014070 con 0x7f94340690a0 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9439d9c700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 0x7f94200a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 -- 172.21.15.5:0/3559176553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 msgr2=0x558afbab9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 -- 172.21.15.5:0/3559176553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 msgr2=0x558afbaac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 -- 172.21.15.5:0/3559176553 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558afba628d0 con 0x558afbaacd20 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f9438d9a700 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.875+0000 7f943959b700 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab7690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94300cabf0 tx=0x7f9430010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a051838211319d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f941f7fe700 1 -- 172.21.15.5:0/3559176553 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943009e560 con 0x558afbaacd20 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f941f7fe700 1 -- 172.21.15.5:0/3559176553 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943009ab10 con 0x558afbaacd20 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f941f7fe700 1 -- 172.21.15.5:0/3559176553 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430099900 con 0x558afbaacd20 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3559176553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 msgr2=0x558afbab7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f9439d9c700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 0x7f94200a3bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f942c0043a0 tx=0x7f942c004b40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab7690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94300cabf0 tx=0x7f9430010c80 comp rx=0 tx=0).stop 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3559176553 shutdown_connections 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3559176553 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab7690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3559176553 >> 172.21.15.5:0/3559176553 conn(0x558afba21260 msgr2=0x558afbaa8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.876+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3559176553 shutdown_connections 2023-09-23T05:06:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.877+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3559176553 wait complete. 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.877+0000 7f944f0ceb80 1 -- start start 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.877+0000 7f944f0ceb80 1 Processor -- start 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbab7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afb9ffe30 con 0x558afbaac400 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afb99c710 con 0x558afbab7bd0 2023-09-23T05:06:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afb99c3b0 con 0x558afbaacd20 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f944f0ceb80 10 client.0 _ll_register_callbacks cb 0x558afb7f9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37752/0 (socket says 172.21.15.5:37752) 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 -- 172.21.15.5:0/2171769308 learned_addr learned my addr 172.21.15.5:0/2171769308 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f943959b700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbab7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9439d9c700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 -- 172.21.15.5:0/2171769308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 msgr2=0x558afbab5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 -- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 msgr2=0x558afbab7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbab7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f9438d9a700 1 -- 172.21.15.5:0/2171769308 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943410e600 con 0x558afbaacd20 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.878+0000 7f943959b700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbab7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f9438d9a700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7bd0750b7777ec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f941cff9700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:06:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942401d070 con 0x558afbaacd20 2023-09-23T05:06:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558afbaa0250 con 0x558afbaacd20 2023-09-23T05:06:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f9424004500 con 0x558afbaacd20 2023-09-23T05:06:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94240046c0 con 0x558afbaacd20 2023-09-23T05:06:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f944f0ceb80 10 client.27862 fetch_fsmap learned FSMap version 79 2023-09-23T05:06:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x7f94341a6380 con 0x558afbaacd20 2023-09-23T05:06:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.879+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558afbaa0250 con 0x558afbaacd20 2023-09-23T05:06:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f942400f910 con 0x558afbaacd20 2023-09-23T05:06:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 10 client.27862 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:06:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 10 client.27862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:06:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 20 client.27862 populate_metadata read hostname 'smithi005' 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbac7640 0x558afbac9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 4 client.27862 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558afb9f8e10 con 0x558afbac7640 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f943959b700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbac7640 0x558afbac9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f943959b700 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbac7640 0x558afbac9a60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28e68835ca7f2cc server_cookie=e84aa25fd6ad8e83 in_seq=0 out_seq=0 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.880+0000 7f941cff9700 10 client.27862 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f941cff9700 1 -- 172.21.15.5:0/2171769308 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558afb9f8e10 con 0x558afbac7640 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f941cff9700 10 client.27862 handle_command_reply: tid=0 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 client.27862 shutdown 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 msgr2=0x558afbab4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424005ad0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 shutdown_connections 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbac7640 0x558afbac9a60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbab7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f941cff9700 0 client.27862 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2171769308 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab4910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.881+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 >> 172.21.15.5:0/2171769308 conn(0x558afba21260 msgr2=0x558afbaa8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.882+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 shutdown_connections 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.882+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2171769308 wait complete. 2023-09-23T05:06:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.882+0000 7f944f0ceb80 20 client.27862 trim_cache size 0 max 16384 2023-09-23T05:06:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.899+0000 7f944f0ceb80 1 Processor -- start 2023-09-23T05:06:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 -- start start 2023-09-23T05:06:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaaf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afb880920 con 0x558afbaac400 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afbaa93d0 con 0x558afbab7bd0 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afbab0760 con 0x558afbaacd20 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f943959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaaf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f9439d9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:37768/0 (socket says 172.21.15.5:37768) 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.900+0000 7f9438d9a700 1 -- 172.21.15.5:0/3039520656 learned_addr learned my addr 172.21.15.5:0/3039520656 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 -- 172.21.15.5:0/3039520656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 msgr2=0x558afbab71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 -- 172.21.15.5:0/3039520656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 msgr2=0x558afbab1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 -- 172.21.15.5:0/3039520656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558afb82ade0 con 0x558afbaac400 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f9438d9a700 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f9439d9c700 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f943959b700 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaaf2a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f943009a5d0 tx=0x7f943009a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f067ad848bdb427 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f941ffff700 1 -- 172.21.15.5:0/3039520656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94300a2820 con 0x558afbaac400 2023-09-23T05:06:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f941ffff700 1 -- 172.21.15.5:0/3039520656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94300987b0 con 0x558afbaac400 2023-09-23T05:06:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.901+0000 7f941ffff700 1 -- 172.21.15.5:0/3039520656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943009a8d0 con 0x558afbaac400 2023-09-23T05:06:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3039520656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 msgr2=0x558afbaaf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaaf2a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f943009a5d0 tx=0x7f943009a530 comp rx=0 tx=0).stop 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3039520656 shutdown_connections 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbab7bd0 0x558afbab1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbaaf2a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/3039520656 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbaacd20 0x558afbab71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3039520656 >> 172.21.15.5:0/3039520656 conn(0x558afba21260 msgr2=0x558afbac3910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3039520656 shutdown_connections 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- 172.21.15.5:0/3039520656 wait complete. 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 -- start start 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.902+0000 7f944f0ceb80 1 Processor -- start 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbaacd20 0x558afbabc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 0x558afbabcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afbabd380 con 0x558afbaac400 2023-09-23T05:06:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afbabd4c0 con 0x558afbaacd20 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afbabd680 con 0x558afbab7bd0 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f944f0ceb80 10 client.0 _ll_register_callbacks cb 0x7f9424005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f943959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f9439d9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 0x558afbabcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f943959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41402/0 (socket says 172.21.15.5:41402) 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f9438d9a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbaacd20 0x558afbabc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.903+0000 7f943959b700 1 -- 172.21.15.5:0/2231848697 learned_addr learned my addr 172.21.15.5:0/2231848697 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 -- 172.21.15.5:0/2231848697 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 msgr2=0x558afbabcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 0x558afbabcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 -- 172.21.15.5:0/2231848697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbaacd20 msgr2=0x558afbabc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbaacd20 0x558afbabc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 -- 172.21.15.5:0/2231848697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94341a6380 con 0x558afbaac400 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f9439d9c700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 0x558afbabcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f943959b700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9430009230 tx=0x7f9430099fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b8d4828455dab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f941cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943009a870 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f94300b8740 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f943009ab10 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f943009e380 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f944f0ceb80 10 client.17787 fetch_fsmap learned FSMap version 79 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.904+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x558afba628d0 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943009ab10 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 79) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94300cd430 con 0x558afbaac400 2023-09-23T05:06:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 10 client.17787 fetch_fsmap finished waiting for FSMap version 79 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 20 client.17787 populate_metadata read hostname 'smithi005' 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbb1c460 0x558afbb1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 4 client.17787 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558afb9f8b30 con 0x558afbb1c460 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.905+0000 7f9438d9a700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbb1c460 0x558afbb1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:22.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f941cff9700 10 client.17787 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f9438d9a700 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbb1c460 0x558afbb1e850 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f2b4390a63a8d4 server_cookie=3217d1cb2b3e3c5c in_seq=0 out_seq=0 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f941cff9700 1 -- 172.21.15.5:0/2231848697 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558afb9f8b30 con 0x558afbb1c460 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f941cff9700 10 client.17787 handle_command_reply: tid=0 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 client.17787 shutdown 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 msgr2=0x558afbabe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9430009230 tx=0x7f9430099fe0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 shutdown_connections 2023-09-23T05:06:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558afbb1c460 0x558afbb1e850 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558afbab7bd0 0x558afbabcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558afbaac400 0x558afbabe5c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 --2- 172.21.15.5:0/2231848697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558afbaacd20 0x558afbabc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 >> 172.21.15.5:0/2231848697 conn(0x558afba21260 msgr2=0x558afbab2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f941cff9700 0 client.17787 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 shutdown_connections 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 1 -- 172.21.15.5:0/2231848697 wait complete. 2023-09-23T05:06:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.906+0000 7f944f0ceb80 20 client.17787 trim_cache size 0 max 16384 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:06:22.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:06:22.918 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:06:22.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:06:22.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:06:22.918 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 msgr2=0x7f94200a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 0x7f94200a3bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f942c0043a0 tx=0x7f942c004b40 comp rx=0 tx=0).stop 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 msgr2=0x7f9434199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9a0 tx=0x7f943000ecb0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 shutdown_connections 2023-09-23T05:06:22.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f94200a1710 0x7f94200a3bd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f943419dbf0 0x7f943419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f94340690a0 0x7f9434199250 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 --2- 172.21.15.5:0/3531179145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9434106210 0x7f9434199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 >> 172.21.15.5:0/3531179145 conn(0x7f9434077ca0 msgr2=0x7f943410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 shutdown_connections 2023-09-23T05:06:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:22.907+0000 7f943b7ff700 1 -- 172.21.15.5:0/3531179145 wait complete. 2023-09-23T05:06:22.926 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:06:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:24 smithi005 ceph-mon[97638]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 392 KiB/s wr, 36 op/s 2023-09-23T05:06:24.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:24 smithi118 ceph-mon[134297]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 392 KiB/s wr, 36 op/s 2023-09-23T05:06:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:24 smithi069 ceph-mon[133931]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 392 KiB/s wr, 36 op/s 2023-09-23T05:06:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:26 smithi005 ceph-mon[97638]: pgmap v633: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 437 KiB/s wr, 36 op/s 2023-09-23T05:06:26.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:26 smithi118 ceph-mon[134297]: pgmap v633: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 437 KiB/s wr, 36 op/s 2023-09-23T05:06:26.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:26 smithi069 ceph-mon[133931]: pgmap v633: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 437 KiB/s wr, 36 op/s 2023-09-23T05:06:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:28 smithi005 ceph-mon[97638]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 401 KiB/s wr, 29 op/s 2023-09-23T05:06:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:28 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:06:28.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:28 smithi118 ceph-mon[134297]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 401 KiB/s wr, 29 op/s 2023-09-23T05:06:28.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:28 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:06:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:28 smithi069 ceph-mon[133931]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 401 KiB/s wr, 29 op/s 2023-09-23T05:06:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:28 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:06:29.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:06:29.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:06:29.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:06:29.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:06:29.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:06:29.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:06:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:06:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:06:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:06:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:30 smithi005 ceph-mon[97638]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 472 KiB/s wr, 33 op/s 2023-09-23T05:06:30.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:30 smithi118 ceph-mon[134297]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 472 KiB/s wr, 33 op/s 2023-09-23T05:06:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:30 smithi069 ceph-mon[133931]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 472 KiB/s wr, 33 op/s 2023-09-23T05:06:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:31 smithi069 ceph-mon[133931]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 693 KiB/s wr, 35 op/s 2023-09-23T05:06:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:31 smithi005 ceph-mon[97638]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 693 KiB/s wr, 35 op/s 2023-09-23T05:06:32.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:31 smithi118 ceph-mon[134297]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 693 KiB/s wr, 35 op/s 2023-09-23T05:06:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:34 smithi005 ceph-mon[97638]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-23T05:06:34.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:34 smithi118 ceph-mon[134297]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-23T05:06:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:34 smithi069 ceph-mon[133931]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-23T05:06:36.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:36 smithi005 ceph-mon[97638]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.2 MiB/s wr, 92 op/s 2023-09-23T05:06:36.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:36 smithi118 ceph-mon[134297]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.2 MiB/s wr, 92 op/s 2023-09-23T05:06:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:36 smithi069 ceph-mon[133931]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.2 MiB/s wr, 92 op/s 2023-09-23T05:06:38.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:38 smithi005 ceph-mon[97638]: pgmap v639: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.4 MiB/s wr, 93 op/s 2023-09-23T05:06:38.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:38 smithi118 ceph-mon[134297]: pgmap v639: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.4 MiB/s wr, 93 op/s 2023-09-23T05:06:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:38 smithi069 ceph-mon[133931]: pgmap v639: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.4 MiB/s wr, 93 op/s 2023-09-23T05:06:40.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:40 smithi005 ceph-mon[97638]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.3 MiB/s wr, 97 op/s 2023-09-23T05:06:40.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:40 smithi118 ceph-mon[134297]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.3 MiB/s wr, 97 op/s 2023-09-23T05:06:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:40 smithi069 ceph-mon[133931]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.3 MiB/s wr, 97 op/s 2023-09-23T05:06:42.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:42 smithi005 ceph-mon[97638]: pgmap v641: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.3 MiB/s wr, 95 op/s 2023-09-23T05:06:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:42 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:42.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:42 smithi118 ceph-mon[134297]: pgmap v641: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.3 MiB/s wr, 95 op/s 2023-09-23T05:06:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:42 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:42 smithi069 ceph-mon[133931]: pgmap v641: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.3 MiB/s wr, 95 op/s 2023-09-23T05:06:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:42 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:44.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:44 smithi005 ceph-mon[97638]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2023-09-23T05:06:44.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:44 smithi118 ceph-mon[134297]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2023-09-23T05:06:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:44 smithi069 ceph-mon[133931]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 3.0 MiB/s wr, 88 op/s 2023-09-23T05:06:46.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:46 smithi005 ceph-mon[97638]: pgmap v643: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 46 op/s 2023-09-23T05:06:46.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:46 smithi118 ceph-mon[134297]: pgmap v643: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 46 op/s 2023-09-23T05:06:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:46 smithi069 ceph-mon[133931]: pgmap v643: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 46 op/s 2023-09-23T05:06:47.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:47 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:47.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:47 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:47.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:47 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:06:48.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:48 smithi005 ceph-mon[97638]: pgmap v644: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 395 KiB/s wr, 22 op/s 2023-09-23T05:06:48.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:48 smithi118 ceph-mon[134297]: pgmap v644: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 395 KiB/s wr, 22 op/s 2023-09-23T05:06:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:48 smithi069 ceph-mon[133931]: pgmap v644: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 395 KiB/s wr, 22 op/s 2023-09-23T05:06:50.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:50 smithi005 ceph-mon[97638]: pgmap v645: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 19 op/s 2023-09-23T05:06:50.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:50 smithi118 ceph-mon[134297]: pgmap v645: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 19 op/s 2023-09-23T05:06:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:50 smithi069 ceph-mon[133931]: pgmap v645: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 19 op/s 2023-09-23T05:06:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:51 smithi069 ceph-mon[133931]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-23T05:06:52.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:51 smithi005 ceph-mon[97638]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-23T05:06:52.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:51 smithi118 ceph-mon[134297]: pgmap v646: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-23T05:06:52.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:06:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.166+0000 7f4fee9ff700 1 -- 172.21.15.5:0/60598737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe8101a80 msgr2=0x7f4fe8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.166+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/60598737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe8101a80 0x7f4fe8105ac0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009990 tx=0x7f4fdc009ca0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 -- 172.21.15.5:0/60598737 shutdown_connections 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/60598737 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 0x7f4fe810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/60598737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe81010d0 0x7f4fe81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/60598737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe8101a80 0x7f4fe8105ac0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 -- 172.21.15.5:0/60598737 >> 172.21.15.5:0/60598737 conn(0x7f4fe80fc990 msgr2=0x7f4fe80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 -- 172.21.15.5:0/60598737 shutdown_connections 2023-09-23T05:06:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.167+0000 7f4fee9ff700 1 -- 172.21.15.5:0/60598737 wait complete. 2023-09-23T05:06:53.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 Processor -- start 2023-09-23T05:06:53.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 -- start start 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 0x7f4fe8073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe8073ba0 0x7f4fe81a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81a5830 con 0x7f4fe81010d0 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81a59a0 con 0x7f4fe8073ba0 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fee9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81a5b60 con 0x7f4fe81061f0 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 0x7f4fe8073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47178/0 (socket says 172.21.15.5:47178) 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.168+0000 7f4fe7fff700 1 -- 172.21.15.5:0/944660369 learned_addr learned my addr 172.21.15.5:0/944660369 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe8073ba0 0x7f4fe81a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 -- 172.21.15.5:0/944660369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 msgr2=0x7f4fe8073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 0x7f4fe8073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 -- 172.21.15.5:0/944660369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe8073ba0 msgr2=0x7f4fe81a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe8073ba0 0x7f4fe81a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 -- 172.21.15.5:0/944660369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0095f0 con 0x7f4fe81010d0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800d950 tx=0x7f4fd800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe57fa700 1 -- 172.21.15.5:0/944660369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800f890 con 0x7f4fe81010d0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe57fa700 1 -- 172.21.15.5:0/944660369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd8010460 con 0x7f4fe81010d0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fe57fa700 1 -- 172.21.15.5:0/944660369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800e610 con 0x7f4fe81010d0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.169+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe81a5e40 con 0x7f4fe81010d0 2023-09-23T05:06:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.170+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe81a62c0 con 0x7f4fe81010d0 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.173+0000 7f4fe57fa700 1 -- 172.21.15.5:0/944660369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f4fd80108d0 con 0x7f4fe81010d0 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.174+0000 7f50022dcb80 1 Processor -- start 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.174+0000 7f4fe57fa700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 0x7f4fd00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.174+0000 7f50022dcb80 1 -- start start 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.174+0000 7f4fe57fa700 1 -- 172.21.15.5:0/944660369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f4fd80cd4c0 con 0x7f4fe81010d0 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 0x7f4fd00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac823c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac819a460 con 0x560ac8231d20 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac818bc20 con 0x560ac823cbd0 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f50022dcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8189540 con 0x560ac8231400 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 -- 172.21.15.5:0/2700902626 learned_addr learned my addr 172.21.15.5:0/2700902626 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac823c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 -- 172.21.15.5:0/2700902626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 msgr2=0x560ac823efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 -- 172.21.15.5:0/2700902626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 msgr2=0x560ac823c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac823c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe7fff700 1 -- 172.21.15.5:0/2700902626 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ac81e78d0 con 0x560ac8231400 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.175+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac823c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 0x7f4fd00a3c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000f540 tx=0x7f4fe0009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80a1e00 tx=0x7f4fd809a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b011c8e5ff4ab1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f4fce7fc700 1 -- 172.21.15.5:0/2700902626 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80a2330 con 0x560ac8231400 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f4fce7fc700 1 -- 172.21.15.5:0/2700902626 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd80cd170 con 0x560ac8231400 2023-09-23T05:06:53.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f4fce7fc700 1 -- 172.21.15.5:0/2700902626 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd809a200 con 0x560ac8231400 2023-09-23T05:06:53.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f50022dcb80 1 -- 172.21.15.5:0/2700902626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 msgr2=0x560ac82317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.176+0000 7f50022dcb80 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80a1e00 tx=0x7f4fd809a720 comp rx=0 tx=0).stop 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 -- 172.21.15.5:0/2700902626 shutdown_connections 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82317e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 --2- 172.21.15.5:0/2700902626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac823c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 -- 172.21.15.5:0/2700902626 >> 172.21.15.5:0/2700902626 conn(0x560ac81a6260 msgr2=0x560ac822d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 -- 172.21.15.5:0/2700902626 shutdown_connections 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 -- 172.21.15.5:0/2700902626 wait complete. 2023-09-23T05:06:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 -- start start 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.177+0000 7f50022dcb80 1 Processor -- start 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8239e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8184e30 con 0x560ac8231d20 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8121710 con 0x560ac823cbd0 2023-09-23T05:06:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac81213b0 con 0x560ac8231400 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f50022dcb80 10 client.0 _ll_register_callbacks cb 0x560ac7f7e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57702/0 (socket says 172.21.15.5:57702) 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f4fe7fff700 1 -- 172.21.15.5:0/3060015343 learned_addr learned my addr 172.21.15.5:0/3060015343 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8239e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.178+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 -- 172.21.15.5:0/3060015343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 msgr2=0x560ac823a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 -- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 msgr2=0x560ac8239e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8239e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 -- 172.21.15.5:0/3060015343 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe81a62c0 con 0x560ac8231400 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8239e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b2f0 tx=0x7f4fd80cd2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee24dd1ab3d03f92 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:06:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80addf0 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f4fd80b8270 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ac8225250 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fd8099490 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f50022dcb80 10 client.27880 fetch_fsmap learned FSMap version 81 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x560ac7fafde0 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.179+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ac8225250 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4fd80a1ea0 con 0x560ac8231400 2023-09-23T05:06:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 10 client.27880 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 10 client.27880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 20 client.27880 populate_metadata read hostname 'smithi005' 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac824c030 0x560ac824e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 4 client.27880 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ac817de10 con 0x560ac824c030 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.180+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac824c030 0x560ac824e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac824c030 0x560ac824e450 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94117787bd3bd7a server_cookie=ddcf76c3c7c214ab in_seq=0 out_seq=0 2023-09-23T05:06:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f4fc77fe700 10 client.27880 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3060015343 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ac817de10 con 0x560ac824c030 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f4fc77fe700 10 client.27880 handle_command_reply: tid=0 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 client.27880 shutdown 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 msgr2=0x560ac82398e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b2f0 tx=0x7f4fd80cd2e0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 shutdown_connections 2023-09-23T05:06:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac824c030 0x560ac824e450 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac823cbd0 0x560ac823a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac8231400 0x560ac82398e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3060015343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8239e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 >> 172.21.15.5:0/3060015343 conn(0x560ac81a6260 msgr2=0x560ac822d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.181+0000 7f4fc77fe700 0 client.27880 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.182+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 shutdown_connections 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.182+0000 7f50022dcb80 1 -- 172.21.15.5:0/3060015343 wait complete. 2023-09-23T05:06:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.182+0000 7f50022dcb80 20 client.27880 trim_cache size 0 max 16384 2023-09-23T05:06:53.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 Processor -- start 2023-09-23T05:06:53.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 -- start start 2023-09-23T05:06:53.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac823a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8236b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8005920 con 0x560ac8231d20 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8232620 con 0x560ac8231400 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f50022dcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8227860 con 0x560ac823cbd0 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f4fecf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8236b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f4fecf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57706/0 (socket says 172.21.15.5:57706) 2023-09-23T05:06:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f4fecf9c700 1 -- 172.21.15.5:0/280589614 learned_addr learned my addr 172.21.15.5:0/280589614 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.200+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac823a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 -- 172.21.15.5:0/280589614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 msgr2=0x560ac823a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac823a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 -- 172.21.15.5:0/280589614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 msgr2=0x560ac8236b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8236b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 -- 172.21.15.5:0/280589614 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ac7f98c60 con 0x560ac823cbd0 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fecf9c700 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0019590 tx=0x7f4fe000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88574e44164bc30d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fceffd700 1 -- 172.21.15.5:0/280589614 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0028070 con 0x560ac823cbd0 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fceffd700 1 -- 172.21.15.5:0/280589614 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe000f040 con 0x560ac823cbd0 2023-09-23T05:06:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.201+0000 7f4fceffd700 1 -- 172.21.15.5:0/280589614 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000e040 con 0x560ac823cbd0 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 -- 172.21.15.5:0/280589614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 msgr2=0x560ac82370d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0019590 tx=0x7f4fe000b040 comp rx=0 tx=0).stop 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 -- 172.21.15.5:0/280589614 shutdown_connections 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac82370d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac823a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 --2- 172.21.15.5:0/280589614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac8236b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 -- 172.21.15.5:0/280589614 >> 172.21.15.5:0/280589614 conn(0x560ac81a6260 msgr2=0x560ac8238940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 -- 172.21.15.5:0/280589614 shutdown_connections 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.202+0000 7f50022dcb80 1 -- 172.21.15.5:0/280589614 wait complete. 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 -- start start 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 Processor -- start 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac82433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac8241be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8242120 con 0x560ac8231d20 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8242290 con 0x560ac8231400 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac8242450 con 0x560ac823cbd0 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.203+0000 7f50022dcb80 10 client.0 _ll_register_callbacks cb 0x7f4fd8099990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fecf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac8241be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac82433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47216/0 (socket says 172.21.15.5:47216) 2023-09-23T05:06:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fecf9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac8241be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57722/0 (socket says 172.21.15.5:57722) 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 -- 172.21.15.5:0/3010369961 learned_addr learned my addr 172.21.15.5:0/3010369961 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 -- 172.21.15.5:0/3010369961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 msgr2=0x560ac8241be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac8241be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 -- 172.21.15.5:0/3010369961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 msgr2=0x560ac82433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac82433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 -- 172.21.15.5:0/3010369961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe000fde0 con 0x560ac8231d20 2023-09-23T05:06:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac82433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fe77fe700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8068e70 tx=0x7f4fdc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937cac68ee45f42e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.204+0000 7f4fc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc01d070 con 0x560ac8231d20 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fd80cd170 con 0x560ac8231d20 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f4fdc0280b0 con 0x560ac8231d20 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fdc028270 con 0x560ac8231d20 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 10 client.17805 fetch_fsmap learned FSMap version 81 2023-09-23T05:06:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x560ac7fd6250 con 0x560ac8231d20 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd80cd170 con 0x560ac8231d20 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f4fdc00f580 con 0x560ac8231d20 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 10 client.17805 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 20 client.17805 populate_metadata read hostname 'smithi005' 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac82a11f0 0x560ac82a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 4 client.17805 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:06:53.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.205+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ac817db30 con 0x560ac82a11f0 2023-09-23T05:06:53.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac82a11f0 0x560ac82a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f4fe7fff700 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac82a11f0 0x560ac82a3610 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f63f5dbb9b50772 server_cookie=19f39f9aab18d014 in_seq=0 out_seq=0 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f4fc77fe700 10 client.17805 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f4fc77fe700 1 -- 172.21.15.5:0/3010369961 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560ac817db30 con 0x560ac82a11f0 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f4fc77fe700 10 client.17805 handle_command_reply: tid=0 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f50022dcb80 1 client.17805 shutdown 2023-09-23T05:06:53.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 msgr2=0x560ac82416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.206+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8068e70 tx=0x7f4fdc005fd0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 shutdown_connections 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x560ac82a11f0 0x560ac82a3610 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x560ac823cbd0 0x560ac8241be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ac8231400 0x560ac82433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f4fc77fe700 0 client.17805 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 --2- 172.21.15.5:0/3010369961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ac8231d20 0x560ac82416a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 >> 172.21.15.5:0/3010369961 conn(0x560ac81a6260 msgr2=0x560ac823b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 shutdown_connections 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 1 -- 172.21.15.5:0/3010369961 wait complete. 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f50022dcb80 20 client.17805 trim_cache size 0 max 16384 2023-09-23T05:06:53.216 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:06:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:06:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:06:53.218 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 msgr2=0x7f4fd00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.207+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 0x7f4fd00a3c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000f540 tx=0x7f4fe0009920 comp rx=0 tx=0).stop 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 msgr2=0x7f4fe8073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800d950 tx=0x7f4fd800dc60 comp rx=0 tx=0).stop 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 shutdown_connections 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f4fd00a1790 0x7f4fd00a3c50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4fe8073ba0 0x7f4fe81a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fe81010d0 0x7f4fe8073120 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 --2- 172.21.15.5:0/944660369 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4fe81061f0 0x7f4fe8073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:06:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 >> 172.21.15.5:0/944660369 conn(0x7f4fe80fc990 msgr2=0x7f4fe80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:06:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 shutdown_connections 2023-09-23T05:06:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:06:53.208+0000 7f4fee9ff700 1 -- 172.21.15.5:0/944660369 wait complete. 2023-09-23T05:06:53.226 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:06:54.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:54 smithi005 ceph-mon[97638]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-23T05:06:54.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:54 smithi118 ceph-mon[134297]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-23T05:06:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:54 smithi069 ceph-mon[133931]: pgmap v647: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-23T05:06:56.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:56 smithi005 ceph-mon[97638]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-23T05:06:56.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:56 smithi118 ceph-mon[134297]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-23T05:06:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:56 smithi069 ceph-mon[133931]: pgmap v648: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-23T05:06:58.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:06:58 smithi005 ceph-mon[97638]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-23T05:06:58.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:06:58 smithi118 ceph-mon[134297]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-23T05:06:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:06:58 smithi069 ceph-mon[133931]: pgmap v649: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-23T05:07:00.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:00 smithi005 ceph-mon[97638]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-23T05:07:00.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:00 smithi118 ceph-mon[134297]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-23T05:07:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:00 smithi069 ceph-mon[133931]: pgmap v650: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-23T05:07:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:01 smithi069 ceph-mon[133931]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-23T05:07:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:01 smithi005 ceph-mon[97638]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-23T05:07:02.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:01 smithi118 ceph-mon[134297]: pgmap v651: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-23T05:07:04.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:04 smithi005 ceph-mon[97638]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 146 KiB/s wr, 20 op/s 2023-09-23T05:07:04.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:04 smithi118 ceph-mon[134297]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 146 KiB/s wr, 20 op/s 2023-09-23T05:07:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:04 smithi069 ceph-mon[133931]: pgmap v652: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 146 KiB/s wr, 20 op/s 2023-09-23T05:07:06.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:06 smithi005 ceph-mon[97638]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:06.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:06 smithi118 ceph-mon[134297]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:06.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:06 smithi069 ceph-mon[133931]: pgmap v653: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:07.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:07 smithi005 ceph-mon[97638]: osdmap e119: 12 total, 12 up, 12 in 2023-09-23T05:07:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:07 smithi118 ceph-mon[134297]: osdmap e119: 12 total, 12 up, 12 in 2023-09-23T05:07:07.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:07 smithi069 ceph-mon[133931]: osdmap e119: 12 total, 12 up, 12 in 2023-09-23T05:07:08.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:08 smithi005 ceph-mon[97638]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 172 KiB/s wr, 37 op/s 2023-09-23T05:07:08.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:08 smithi118 ceph-mon[134297]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 172 KiB/s wr, 37 op/s 2023-09-23T05:07:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:08 smithi069 ceph-mon[133931]: pgmap v655: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 172 KiB/s wr, 37 op/s 2023-09-23T05:07:10.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:10 smithi005 ceph-mon[97638]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 159 KiB/s wr, 36 op/s 2023-09-23T05:07:10.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:10 smithi118 ceph-mon[134297]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 159 KiB/s wr, 36 op/s 2023-09-23T05:07:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:10 smithi069 ceph-mon[133931]: pgmap v656: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 159 KiB/s wr, 36 op/s 2023-09-23T05:07:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:11 smithi069 ceph-mon[133931]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 161 KiB/s wr, 35 op/s 2023-09-23T05:07:12.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:11 smithi005 ceph-mon[97638]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 161 KiB/s wr, 35 op/s 2023-09-23T05:07:12.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:11 smithi118 ceph-mon[134297]: pgmap v657: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 161 KiB/s wr, 35 op/s 2023-09-23T05:07:14.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:14 smithi005 ceph-mon[97638]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 221 KiB/s wr, 39 op/s 2023-09-23T05:07:14.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:14 smithi118 ceph-mon[134297]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 221 KiB/s wr, 39 op/s 2023-09-23T05:07:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:14 smithi069 ceph-mon[133931]: pgmap v658: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 221 KiB/s wr, 39 op/s 2023-09-23T05:07:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:16 smithi005 ceph-mon[97638]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 235 KiB/s wr, 38 op/s 2023-09-23T05:07:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:16 smithi005 ceph-mon[97638]: osdmap e120: 12 total, 12 up, 12 in 2023-09-23T05:07:16.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:16 smithi118 ceph-mon[134297]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 235 KiB/s wr, 38 op/s 2023-09-23T05:07:16.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:16 smithi118 ceph-mon[134297]: osdmap e120: 12 total, 12 up, 12 in 2023-09-23T05:07:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:16 smithi069 ceph-mon[133931]: pgmap v659: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 235 KiB/s wr, 38 op/s 2023-09-23T05:07:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:16 smithi069 ceph-mon[133931]: osdmap e120: 12 total, 12 up, 12 in 2023-09-23T05:07:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:18 smithi005 ceph-mon[97638]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-23T05:07:18.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:18 smithi118 ceph-mon[134297]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-23T05:07:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:18 smithi069 ceph-mon[133931]: pgmap v661: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-23T05:07:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:20 smithi005 ceph-mon[97638]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 222 KiB/s wr, 24 op/s 2023-09-23T05:07:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:20 smithi118 ceph-mon[134297]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 222 KiB/s wr, 24 op/s 2023-09-23T05:07:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:20 smithi069 ceph-mon[133931]: pgmap v662: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 222 KiB/s wr, 24 op/s 2023-09-23T05:07:22.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:21 smithi069 ceph-mon[133931]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 24 op/s 2023-09-23T05:07:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:21 smithi005 ceph-mon[97638]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 24 op/s 2023-09-23T05:07:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:21 smithi118 ceph-mon[134297]: pgmap v663: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 24 op/s 2023-09-23T05:07:23.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:07:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 -- 172.21.15.5:0/1131161382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc10b4b0 msgr2=0x7f19fc10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 --2- 172.21.15.5:0/1131161382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc10b4b0 0x7f19fc10d920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b0a0 tx=0x7f19f400b3b0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 -- 172.21.15.5:0/1131161382 shutdown_connections 2023-09-23T05:07:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 --2- 172.21.15.5:0/1131161382 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc10b4b0 0x7f19fc10d920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 --2- 172.21.15.5:0/1131161382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 --2- 172.21.15.5:0/1131161382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc102a00 0x7f19fc10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 -- 172.21.15.5:0/1131161382 >> 172.21.15.5:0/1131161382 conn(0x7f19fc076f30 msgr2=0x7f19fc077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.444+0000 7f1a025ff700 1 -- 172.21.15.5:0/1131161382 shutdown_connections 2023-09-23T05:07:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.445+0000 7f1a025ff700 1 -- 172.21.15.5:0/1131161382 wait complete. 2023-09-23T05:07:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.446+0000 7f1a025ff700 1 Processor -- start 2023-09-23T05:07:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.446+0000 7f1a025ff700 1 -- start start 2023-09-23T05:07:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.446+0000 7f1a025ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.446+0000 7f1a025ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f1a025ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc10b4b0 0x7f19fc19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f1a025ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e290 con 0x7f19fc1020e0 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f1a025ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e400 con 0x7f19fc10b4b0 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f1a025ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e5c0 con 0x7f19fc102a00 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57700/0 (socket says 172.21.15.5:57700) 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 -- 172.21.15.5:0/3651240093 learned_addr learned my addr 172.21.15.5:0/3651240093 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc10b4b0 0x7f19fc19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 -- 172.21.15.5:0/3651240093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc10b4b0 msgr2=0x7f19fc19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc10b4b0 0x7f19fc19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 -- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 msgr2=0x7f19fc1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19f3fff700 1 -- 172.21.15.5:0/3651240093 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009d00 con 0x7f19fc102a00 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.447+0000 7f19fbfff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19e400e9a0 tx=0x7f19e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f19f9ffb700 1 -- 172.21.15.5:0/3651240093 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4009790 con 0x7f19fc102a00 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f19f9ffb700 1 -- 172.21.15.5:0/3651240093 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e4010430 con 0x7f19fc102a00 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f19f9ffb700 1 -- 172.21.15.5:0/3651240093 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4018680 con 0x7f19fc102a00 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc19e8a0 con 0x7f19fc102a00 2023-09-23T05:07:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.448+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc1a60e0 con 0x7f19fc102a00 2023-09-23T05:07:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.451+0000 7f19f9ffb700 1 -- 172.21.15.5:0/3651240093 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f19e40105a0 con 0x7f19fc102a00 2023-09-23T05:07:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.452+0000 7f1a16000b80 1 Processor -- start 2023-09-23T05:07:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.452+0000 7f1a16000b80 1 -- start start 2023-09-23T05:07:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351dffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55733513b460 con 0x5573351d2400 2023-09-23T05:07:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55733512cc20 con 0x5573351ddbd0 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a16000b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55733512a540 con 0x5573351d2d20 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19f9ffb700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 0x7f19e80a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57710/0 (socket says 172.21.15.5:57710) 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 -- 172.21.15.5:0/2572887855 learned_addr learned my addr 172.21.15.5:0/2572887855 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19f9ffb700 1 -- 172.21.15.5:0/3651240093 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f19e4014070 con 0x7f19fc102a00 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351dffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 0x7f19e80a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 -- 172.21.15.5:0/2572887855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 msgr2=0x5573351dffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351dffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 -- 172.21.15.5:0/2572887855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 msgr2=0x5573351d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.453+0000 7f19fbfff700 1 -- 172.21.15.5:0/2572887855 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573351888d0 con 0x5573351d2d20 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:07:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0115f0 tx=0x7f19ec011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42a858556a0af16 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 0x7f19e80a3c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b780 tx=0x7f19f4006230 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f19f27fc700 1 -- 172.21.15.5:0/2572887855 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00f680 con 0x5573351d2d20 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f19f27fc700 1 -- 172.21.15.5:0/2572887855 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19ec00fca0 con 0x5573351d2d20 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.454+0000 7f19f27fc700 1 -- 172.21.15.5:0/2572887855 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec019cb0 con 0x5573351d2d20 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- 172.21.15.5:0/2572887855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 msgr2=0x5573351dd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0115f0 tx=0x7f19ec011900 comp rx=0 tx=0).stop 2023-09-23T05:07:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- 172.21.15.5:0/2572887855 shutdown_connections 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351dffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2572887855 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2d20 0x5573351dd690 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- 172.21.15.5:0/2572887855 >> 172.21.15.5:0/2572887855 conn(0x557335147260 msgr2=0x5573351ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- 172.21.15.5:0/2572887855 shutdown_connections 2023-09-23T05:07:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- 172.21.15.5:0/2572887855 wait complete. 2023-09-23T05:07:23.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.455+0000 7f1a16000b80 1 -- start start 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 Processor -- start 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351daf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335125e30 con 0x5573351d2d20 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573350c2710 con 0x5573351ddbd0 2023-09-23T05:07:23.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573350c23b0 con 0x5573351d2400 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f1a16000b80 10 client.0 _ll_register_callbacks cb 0x557334f1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.456+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57724/0 (socket says 172.21.15.5:57724) 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 -- 172.21.15.5:0/3184865123 learned_addr learned my addr 172.21.15.5:0/3184865123 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 -- 172.21.15.5:0/3184865123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 msgr2=0x5573351daf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351daf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 -- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 msgr2=0x5573351da810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351da810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 -- 172.21.15.5:0/3184865123 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec009dd0 con 0x5573351d2400 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19fbfff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f19ec000c00 tx=0x7f19ec005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499b5d9047cb0db9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.457+0000 7f19db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec004220 con 0x5573351d2400 2023-09-23T05:07:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573351c6250 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f19ec004380 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19ec00fe90 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 10 client.27898 fetch_fsmap learned FSMap version 81 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x557334f50de0 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573351c6250 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f19ec0039e0 con 0x5573351d2400 2023-09-23T05:07:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 10 client.27898 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 10 client.27898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 20 client.27898 populate_metadata read hostname 'smithi005' 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573351ed5a0 0x5573351ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 4 client.27898 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.458+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55733511ee10 con 0x5573351ed5a0 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.459+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573351ed5a0 0x5573351ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.459+0000 7f19f3fff700 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573351ed5a0 0x5573351ef9c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=856d864d9d644c1 server_cookie=2a8339dc708b3e25 in_seq=0 out_seq=0 2023-09-23T05:07:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.459+0000 7f19db7fe700 10 client.27898 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f19db7fe700 1 -- 172.21.15.5:0/3184865123 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55733511ee10 con 0x5573351ed5a0 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f19db7fe700 10 client.27898 handle_command_reply: tid=0 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 client.27898 shutdown 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 msgr2=0x5573351dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f19ec000c00 tx=0x7f19ec005ce0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 shutdown_connections 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573351ed5a0 0x5573351ef9c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351daf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351dd600 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 --2- 172.21.15.5:0/3184865123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f19db7fe700 0 client.27898 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 >> 172.21.15.5:0/3184865123 conn(0x557335147260 msgr2=0x5573351ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 shutdown_connections 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 1 -- 172.21.15.5:0/3184865123 wait complete. 2023-09-23T05:07:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.460+0000 7f1a16000b80 20 client.27898 trim_cache size 0 max 16384 2023-09-23T05:07:23.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 Processor -- start 2023-09-23T05:07:23.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 -- start start 2023-09-23T05:07:23.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351d2d20 0x5573351dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 0x5573351d7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557334fa6920 con 0x5573351d2400 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573351d66c0 con 0x5573351d2d20 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.478+0000 7f1a16000b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573351d3360 con 0x5573351ddbd0 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f1a00b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 0x5573351d7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f1a00b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 0x5573351d7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57730/0 (socket says 172.21.15.5:57730) 2023-09-23T05:07:23.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59200/0 (socket says 172.21.15.5:59200) 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f1a00b9c700 1 -- 172.21.15.5:0/2459562772 learned_addr learned my addr 172.21.15.5:0/2459562772 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351d2d20 0x5573351dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 -- 172.21.15.5:0/2459562772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 msgr2=0x5573351d7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 0x5573351d7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 -- 172.21.15.5:0/2459562772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351d2d20 msgr2=0x5573351dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351d2d20 0x5573351dd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 -- 172.21.15.5:0/2459562772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557334f39c60 con 0x5573351d2400 2023-09-23T05:07:23.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f19ec011510 tx=0x7f19ec011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8995b9f76376514 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.479+0000 7f19f2ffd700 1 -- 172.21.15.5:0/2459562772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec005d50 con 0x5573351d2400 2023-09-23T05:07:23.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f19f2ffd700 1 -- 172.21.15.5:0/2459562772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19ec025520 con 0x5573351d2400 2023-09-23T05:07:23.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f19f2ffd700 1 -- 172.21.15.5:0/2459562772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00f7f0 con 0x5573351d2400 2023-09-23T05:07:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 -- 172.21.15.5:0/2459562772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 msgr2=0x5573351da7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f19ec011510 tx=0x7f19ec011900 comp rx=0 tx=0).stop 2023-09-23T05:07:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 -- 172.21.15.5:0/2459562772 shutdown_connections 2023-09-23T05:07:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351ddbd0 0x5573351d7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2400 0x5573351da7c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2459562772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351d2d20 0x5573351dd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 -- 172.21.15.5:0/2459562772 >> 172.21.15.5:0/2459562772 conn(0x557335147260 msgr2=0x5573351da0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 -- 172.21.15.5:0/2459562772 shutdown_connections 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.480+0000 7f1a16000b80 1 -- 172.21.15.5:0/2459562772 wait complete. 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 -- start start 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 Processor -- start 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573351e3340 con 0x5573351d2d20 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573351e3480 con 0x5573351ddbd0 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573351e3640 con 0x5573351d2400 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.481+0000 7f1a16000b80 10 client.0 _ll_register_callbacks cb 0x7f19ec011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57742/0 (socket says 172.21.15.5:57742) 2023-09-23T05:07:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 -- 172.21.15.5:0/2563518840 learned_addr learned my addr 172.21.15.5:0/2563518840 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 -- 172.21.15.5:0/2563518840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 msgr2=0x5573351e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 -- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 msgr2=0x5573351e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 -- 172.21.15.5:0/2563518840 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557334f50de0 con 0x5573351d2400 2023-09-23T05:07:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f1a00b9c700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19fbfff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f19ec003df0 tx=0x7f19ec011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3777d6ec5b9fe20f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec003b80 con 0x5573351d2400 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.482+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19ec00fca0 con 0x5573351d2400 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f19ec000e40 con 0x5573351d2400 2023-09-23T05:07:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19ec005d50 con 0x5573351d2400 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 10 client.27904 fetch_fsmap learned FSMap version 81 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x557334f77250 con 0x5573351d2400 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19ec00fca0 con 0x5573351d2400 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f19ec0257d0 con 0x5573351d2400 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 10 client.27904 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 10 client.27904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 20 client.27904 populate_metadata read hostname 'smithi005' 2023-09-23T05:07:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573352424b0 0x5573352448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 4 client.27904 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55733511eb30 con 0x5573352424b0 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.483+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573352424b0 0x5573352448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f19f3fff700 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573352424b0 0x5573352448a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b32147dfcf6c3f8 server_cookie=250c72e69465d36e in_seq=0 out_seq=0 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f19db7fe700 10 client.27904 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f19db7fe700 1 -- 172.21.15.5:0/2563518840 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55733511eb30 con 0x5573352424b0 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f19db7fe700 10 client.27904 handle_command_reply: tid=0 2023-09-23T05:07:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 client.27904 shutdown 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 msgr2=0x5573351e45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f19ec003df0 tx=0x7f19ec011ea0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 shutdown_connections 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5573352424b0 0x5573352448a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573351ddbd0 0x5573351e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5573351d2400 0x5573351e45c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f19db7fe700 0 client.27904 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 --2- 172.21.15.5:0/2563518840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573351d2d20 0x5573351e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.484+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 >> 172.21.15.5:0/2563518840 conn(0x557335147260 msgr2=0x5573351d4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 shutdown_connections 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a16000b80 1 -- 172.21.15.5:0/2563518840 wait complete. 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a16000b80 20 client.27904 trim_cache size 0 max 16384 2023-09-23T05:07:23.495 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:07:23.496 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:07:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 msgr2=0x7f19e80a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 0x7f19e80a3c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b780 tx=0x7f19f4006230 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 msgr2=0x7f19fc1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f19e400e9a0 tx=0x7f19e400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 shutdown_connections 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f19e80a1790 0x7f19e80a3c50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19fc10b4b0 0x7f19fc19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc1020e0 0x7f19fc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 --2- 172.21.15.5:0/3651240093 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f19fc102a00 0x7f19fc1997f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 >> 172.21.15.5:0/3651240093 conn(0x7f19fc076f30 msgr2=0x7f19fc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 shutdown_connections 2023-09-23T05:07:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:23.485+0000 7f1a025ff700 1 -- 172.21.15.5:0/3651240093 wait complete. 2023-09-23T05:07:23.504 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:07:24.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:24 smithi005 ceph-mon[97638]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:24.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:24 smithi118 ceph-mon[134297]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:24 smithi069 ceph-mon[133931]: pgmap v664: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 127 KiB/s wr, 19 op/s 2023-09-23T05:07:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:26 smithi005 ceph-mon[97638]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-23T05:07:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:26 smithi118 ceph-mon[134297]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-23T05:07:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:26 smithi069 ceph-mon[133931]: pgmap v665: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-23T05:07:28.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:28 smithi005 ceph-mon[97638]: pgmap v666: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 62 KiB/s wr, 14 op/s 2023-09-23T05:07:28.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:28 smithi118 ceph-mon[134297]: pgmap v666: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 62 KiB/s wr, 14 op/s 2023-09-23T05:07:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:28 smithi069 ceph-mon[133931]: pgmap v666: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 62 KiB/s wr, 14 op/s 2023-09-23T05:07:29.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:29 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:07:29.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:29 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:07:29.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:29 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:07:30.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:30 smithi005 ceph-mon[97638]: pgmap v667: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-23T05:07:30.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:30 smithi118 ceph-mon[134297]: pgmap v667: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-23T05:07:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:30 smithi069 ceph-mon[133931]: pgmap v667: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-23T05:07:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: pgmap v668: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-23T05:07:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:31 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: pgmap v668: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-23T05:07:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:31 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: pgmap v668: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:31 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T05:07:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T05:07:33.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:33.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:32 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:33.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:33.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:32 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi069 to 196.1M 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi069 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:33.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:33.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:32 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:34.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T05:07:34.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T05:07:34.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: pgmap v669: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:07:34.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:33 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: pgmap v669: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:07:34.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:07:34.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:33 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:34.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi118 to 196.2M 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi118 to 205743820: error parsing value: Value '205743820' is below minimum 939524096 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: pgmap v669: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-23T05:07:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:07:34.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:07:34.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:33 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:07:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:34 smithi069 ceph-mon[133931]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T05:07:35.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:34 smithi069 ceph-mon[133931]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T05:07:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:34 smithi005 ceph-mon[97638]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T05:07:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:34 smithi005 ceph-mon[97638]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T05:07:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:34 smithi118 ceph-mon[134297]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-09-23T05:07:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:34 smithi118 ceph-mon[134297]: Unable to set osd_memory_target on smithi005 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-23T05:07:36.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:35 smithi005 ceph-mon[97638]: pgmap v670: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-23T05:07:36.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:35 smithi118 ceph-mon[134297]: pgmap v670: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-23T05:07:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:35 smithi069 ceph-mon[133931]: pgmap v670: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-23T05:07:38.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:38 smithi005 ceph-mon[97638]: pgmap v671: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-23T05:07:38.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:38 smithi118 ceph-mon[134297]: pgmap v671: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-23T05:07:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:38 smithi069 ceph-mon[133931]: pgmap v671: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-23T05:07:40.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:40 smithi005 ceph-mon[97638]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-23T05:07:40.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:40 smithi118 ceph-mon[134297]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-23T05:07:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:40 smithi069 ceph-mon[133931]: pgmap v672: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-23T05:07:42.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:41 smithi069 ceph-mon[133931]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-23T05:07:42.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:41 smithi005 ceph-mon[97638]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-23T05:07:42.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:41 smithi118 ceph-mon[134297]: pgmap v673: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-23T05:07:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:44 smithi005 ceph-mon[97638]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-23T05:07:44.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:44 smithi118 ceph-mon[134297]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-23T05:07:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:44 smithi069 ceph-mon[133931]: pgmap v674: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-23T05:07:46.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:46 smithi005 ceph-mon[97638]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-09-23T05:07:46.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:46 smithi118 ceph-mon[134297]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-09-23T05:07:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:46 smithi069 ceph-mon[133931]: pgmap v675: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2023-09-23T05:07:48.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:48 smithi005 ceph-mon[97638]: pgmap v676: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-23T05:07:48.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:48 smithi118 ceph-mon[134297]: pgmap v676: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-23T05:07:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:48 smithi069 ceph-mon[133931]: pgmap v676: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-23T05:07:50.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:50 smithi005 ceph-mon[97638]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-23T05:07:50.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:50 smithi118 ceph-mon[134297]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-23T05:07:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:50 smithi069 ceph-mon[133931]: pgmap v677: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-23T05:07:52.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:51 smithi069 ceph-mon[133931]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2023-09-23T05:07:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:51 smithi005 ceph-mon[97638]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2023-09-23T05:07:52.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:51 smithi118 ceph-mon[134297]: pgmap v678: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2023-09-23T05:07:53.505 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:07:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 -- 172.21.15.5:0/3205059208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 msgr2=0x7fd92c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 --2- 172.21.15.5:0/3205059208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1014b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd928009990 tx=0x7fd928009ca0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 -- 172.21.15.5:0/3205059208 shutdown_connections 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 --2- 172.21.15.5:0/3205059208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 --2- 172.21.15.5:0/3205059208 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1014b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 --2- 172.21.15.5:0/3205059208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c101a80 0x7fd92c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 -- 172.21.15.5:0/3205059208 >> 172.21.15.5:0/3205059208 conn(0x7fd92c0fc990 msgr2=0x7fd92c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 -- 172.21.15.5:0/3205059208 shutdown_connections 2023-09-23T05:07:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.724+0000 7fd933fff700 1 -- 172.21.15.5:0/3205059208 wait complete. 2023-09-23T05:07:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 Processor -- start 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 -- start start 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c19dbc0 0x7fd92c19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19e640 con 0x7fd92c1061f0 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19e7b0 con 0x7fd92c19dbc0 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.725+0000 7fd933fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19e970 con 0x7fd92c1010d0 2023-09-23T05:07:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 -- 172.21.15.5:0/4206801924 learned_addr learned my addr 172.21.15.5:0/4206801924 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd93159a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd93259c700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c19dbc0 0x7fd92c19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 -- 172.21.15.5:0/4206801924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c19dbc0 msgr2=0x7fd92c19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c19dbc0 0x7fd92c19e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 -- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 msgr2=0x7fd92c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 -- 172.21.15.5:0/4206801924 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd924009c40 con 0x7fd92c1010d0 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd93159a700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.726+0000 7fd931d9b700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd928004730 tx=0x7fd928004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.727+0000 7fd922ffd700 1 -- 172.21.15.5:0/4206801924 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92801d070 con 0x7fd92c1010d0 2023-09-23T05:07:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.727+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280095f0 con 0x7fd92c1010d0 2023-09-23T05:07:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.727+0000 7fd922ffd700 1 -- 172.21.15.5:0/4206801924 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd928005a00 con 0x7fd92c1010d0 2023-09-23T05:07:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.727+0000 7fd922ffd700 1 -- 172.21.15.5:0/4206801924 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928005340 con 0x7fd92c1010d0 2023-09-23T05:07:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.727+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c1a6100 con 0x7fd92c1010d0 2023-09-23T05:07:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.730+0000 7fd922ffd700 1 -- 172.21.15.5:0/4206801924 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd9280049d0 con 0x7fd92c1010d0 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.731+0000 7fd947866b80 1 Processor -- start 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.731+0000 7fd947866b80 1 -- start start 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.731+0000 7fd922ffd700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 0x7fd9180a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd922ffd700 1 -- 172.21.15.5:0/4206801924 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd9280ce7d0 con 0x7fd92c1010d0 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5e4c460 con 0x55ffc5eeebd0 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5e3dc20 con 0x55ffc5ee3d20 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd947866b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5e3b540 con 0x55ffc5ee3400 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd93259c700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 0x7fd9180a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57470/0 (socket says 172.21.15.5:57470) 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 -- 172.21.15.5:0/198554560 learned_addr learned my addr 172.21.15.5:0/198554560 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd93159a700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 -- 172.21.15.5:0/198554560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 msgr2=0x55ffc5eee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 -- 172.21.15.5:0/198554560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 msgr2=0x55ffc5ef0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd93259c700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd931d9b700 1 -- 172.21.15.5:0/198554560 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc5e998d0 con 0x55ffc5ee3400 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd93159a700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:07:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.732+0000 7fd93259c700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd931d9b700 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd928097db0 tx=0x7fd92809afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d815bb141506bdeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd93259c700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 0x7fd9180a3bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd924009fd0 tx=0x7fd924009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd917fff700 1 -- 172.21.15.5:0/198554560 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92809b340 con 0x55ffc5ee3400 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd917fff700 1 -- 172.21.15.5:0/198554560 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd92809f490 con 0x55ffc5ee3400 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd917fff700 1 -- 172.21.15.5:0/198554560 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92809b5a0 con 0x55ffc5ee3400 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd947866b80 1 -- 172.21.15.5:0/198554560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 msgr2=0x55ffc5ee37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd947866b80 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd928097db0 tx=0x7fd92809afb0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.733+0000 7fd947866b80 1 -- 172.21.15.5:0/198554560 shutdown_connections 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3400 0x55ffc5ee37e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 --2- 172.21.15.5:0/198554560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 -- 172.21.15.5:0/198554560 >> 172.21.15.5:0/198554560 conn(0x55ffc5e58260 msgr2=0x55ffc5edf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 -- 172.21.15.5:0/198554560 shutdown_connections 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 -- 172.21.15.5:0/198554560 wait complete. 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 -- start start 2023-09-23T05:07:53.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.734+0000 7fd947866b80 1 Processor -- start 2023-09-23T05:07:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5eeefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eefa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5e36e30 con 0x55ffc5eeebd0 2023-09-23T05:07:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5dd3710 con 0x55ffc5ee3400 2023-09-23T05:07:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5dd33b0 con 0x55ffc5ee3d20 2023-09-23T05:07:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd947866b80 10 client.0 _ll_register_callbacks cb 0x55ffc5c30800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:07:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd93159a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd93159a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57476/0 (socket says 172.21.15.5:57476) 2023-09-23T05:07:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd93159a700 1 -- 172.21.15.5:0/3229958111 learned_addr learned my addr 172.21.15.5:0/3229958111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd93259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eefa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.735+0000 7fd931d9b700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5eeefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 -- 172.21.15.5:0/3229958111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 msgr2=0x55ffc5eeefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5eeefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 -- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 msgr2=0x55ffc5eefa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eefa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 -- 172.21.15.5:0/3229958111 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c1a6100 con 0x55ffc5ee3d20 2023-09-23T05:07:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd93159a700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd91c013d30 tx=0x7fd91c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980a5fef4219b13b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd9157fa700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:07:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c011e20 con 0x55ffc5ee3d20 2023-09-23T05:07:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffc5ed7250 con 0x55ffc5ee3d20 2023-09-23T05:07:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd91c004830 con 0x55ffc5ee3d20 2023-09-23T05:07:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd91c0049f0 con 0x55ffc5ee3d20 2023-09-23T05:07:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd947866b80 10 client.27916 fetch_fsmap learned FSMap version 81 2023-09-23T05:07:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.736+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55ffc5c61de0 con 0x55ffc5ee3d20 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffc5ed7250 con 0x55ffc5ee3d20 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd91c00f580 con 0x55ffc5ee3d20 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 10 client.27916 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 10 client.27916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 20 client.27916 populate_metadata read hostname 'smithi005' 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5efe260 0x55ffc5f00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 4 client.27916 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffc5e2fe10 con 0x55ffc5efe260 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.737+0000 7fd931d9b700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5efe260 0x55ffc5f00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.738+0000 7fd931d9b700 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5efe260 0x55ffc5f00680 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e033c8ca4ebb2080 server_cookie=c68256f5142fddee in_seq=0 out_seq=0 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.738+0000 7fd9157fa700 10 client.27916 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.738+0000 7fd9157fa700 1 -- 172.21.15.5:0/3229958111 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ffc5e2fe10 con 0x55ffc5efe260 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.738+0000 7fd9157fa700 10 client.27916 handle_command_reply: tid=0 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.738+0000 7fd947866b80 1 client.27916 shutdown 2023-09-23T05:07:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 msgr2=0x55ffc5eef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd91c013d30 tx=0x7fd91c0116c0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 shutdown_connections 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5efe260 0x55ffc5f00680 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eefa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5eeefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd9157fa700 0 client.27916 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 --2- 172.21.15.5:0/3229958111 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5eef4f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 >> 172.21.15.5:0/3229958111 conn(0x55ffc5e58260 msgr2=0x55ffc5edf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 shutdown_connections 2023-09-23T05:07:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 1 -- 172.21.15.5:0/3229958111 wait complete. 2023-09-23T05:07:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.739+0000 7fd947866b80 20 client.27916 trim_cache size 0 max 16384 2023-09-23T05:07:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.756+0000 7fd947866b80 1 Processor -- start 2023-09-23T05:07:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.756+0000 7fd947866b80 1 -- start start 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ee8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eeafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5cb7920 con 0x55ffc5eeebd0 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5ed9890 con 0x55ffc5ee3400 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd947866b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5eee860 con 0x55ffc5ee3d20 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eeafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ee8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eeafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48458/0 (socket says 172.21.15.5:48458) 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93259c700 1 -- 172.21.15.5:0/3785992334 learned_addr learned my addr 172.21.15.5:0/3785992334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd931d9b700 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 -- 172.21.15.5:0/3785992334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 msgr2=0x55ffc5ef9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 -- 172.21.15.5:0/3785992334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 msgr2=0x55ffc5eeafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eeafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.757+0000 7fd93159a700 1 -- 172.21.15.5:0/3785992334 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc5c4ac60 con 0x55ffc5ee3d20 2023-09-23T05:07:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd93159a700 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ee8310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd91c013d00 tx=0x7fd91c009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9540c4fb498807d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd920ff9700 1 -- 172.21.15.5:0/3785992334 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00f3c0 con 0x55ffc5ee3d20 2023-09-23T05:07:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd920ff9700 1 -- 172.21.15.5:0/3785992334 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c014e90 con 0x55ffc5ee3d20 2023-09-23T05:07:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd920ff9700 1 -- 172.21.15.5:0/3785992334 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c005640 con 0x55ffc5ee3d20 2023-09-23T05:07:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd947866b80 1 -- 172.21.15.5:0/3785992334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 msgr2=0x55ffc5ee8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.758+0000 7fd947866b80 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ee8310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd91c013d00 tx=0x7fd91c009f40 comp rx=0 tx=0).stop 2023-09-23T05:07:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 -- 172.21.15.5:0/3785992334 shutdown_connections 2023-09-23T05:07:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5eeebd0 0x55ffc5eeafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 --2- 172.21.15.5:0/3785992334 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ee8310 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 -- 172.21.15.5:0/3785992334 >> 172.21.15.5:0/3785992334 conn(0x55ffc5e58260 msgr2=0x55ffc5eef840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 -- 172.21.15.5:0/3785992334 shutdown_connections 2023-09-23T05:07:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 -- 172.21.15.5:0/3785992334 wait complete. 2023-09-23T05:07:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 -- start start 2023-09-23T05:07:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.759+0000 7fd947866b80 1 Processor -- start 2023-09-23T05:07:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ef2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5ef2e60 con 0x55ffc5ee3400 2023-09-23T05:07:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5ef2fd0 con 0x55ffc5eeebd0 2023-09-23T05:07:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc5f520a0 con 0x55ffc5ee3d20 2023-09-23T05:07:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd947866b80 10 client.0 _ll_register_callbacks cb 0x7fd91c013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:07:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd93159a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ef2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48470/0 (socket says 172.21.15.5:48470) 2023-09-23T05:07:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 -- 172.21.15.5:0/724935866 learned_addr learned my addr 172.21.15.5:0/724935866 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:07:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd93259c700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 -- 172.21.15.5:0/724935866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 msgr2=0x55ffc5ef2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ef2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 -- 172.21.15.5:0/724935866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5eeebd0 msgr2=0x55ffc5ef2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 -- 172.21.15.5:0/724935866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc5c61de0 con 0x55ffc5ee3400 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd931d9b700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd928005000 tx=0x7fd928004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191fd0e30b16e60a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.760+0000 7fd9157fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:07:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92809f120 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd92809a830 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd92809f490 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd92809f280 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 10 client.17838 fetch_fsmap learned FSMap version 81 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55ffc5c88250 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd92809f490 con 0x55ffc5ee3400 2023-09-23T05:07:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd9280b3bd0 con 0x55ffc5ee3400 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 20 client.17838 populate_metadata read hostname 'smithi005' 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.761+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5f532c0 0x55ffc5f556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.762+0000 7fd947866b80 4 client.17838 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.762+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ffc5e2fb30 con 0x55ffc5f532c0 2023-09-23T05:07:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.762+0000 7fd93159a700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5f532c0 0x55ffc5f556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.762+0000 7fd93159a700 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5f532c0 0x55ffc5f556e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65033671d7cb812a server_cookie=59a04d8bf0067100 in_seq=0 out_seq=0 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.762+0000 7fd9157fa700 10 client.17838 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd9157fa700 1 -- 172.21.15.5:0/724935866 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ffc5e2fb30 con 0x55ffc5f532c0 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd9157fa700 10 client.17838 handle_command_reply: tid=0 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 client.17838 shutdown 2023-09-23T05:07:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 msgr2=0x55ffc5ef3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd928005000 tx=0x7fd928004d70 comp rx=0 tx=0).stop 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 shutdown_connections 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55ffc5f532c0 0x55ffc5f556e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc5eeebd0 0x55ffc5ef2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffc5ee3400 0x55ffc5ef3f50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd9157fa700 0 client.17838 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:07:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 --2- 172.21.15.5:0/724935866 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55ffc5ee3d20 0x55ffc5ef2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 >> 172.21.15.5:0/724935866 conn(0x55ffc5e58260 msgr2=0x55ffc5ede9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 shutdown_connections 2023-09-23T05:07:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 1 -- 172.21.15.5:0/724935866 wait complete. 2023-09-23T05:07:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.763+0000 7fd947866b80 20 client.17838 trim_cache size 0 max 16384 2023-09-23T05:07:53.779 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:07:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:07:53.781 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:07:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 msgr2=0x7fd9180a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 0x7fd9180a3bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd924009fd0 tx=0x7fd924009920 comp rx=0 tx=0).stop 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 msgr2=0x7fd92c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd928004730 tx=0x7fd928004760 comp rx=0 tx=0).stop 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 shutdown_connections 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd9180a1710 0x7fd9180a3bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd92c19dbc0 0x7fd92c19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd92c1010d0 0x7fd92c1992c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 --2- 172.21.15.5:0/4206801924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92c1061f0 0x7fd92c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:07:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 >> 172.21.15.5:0/4206801924 conn(0x7fd92c0fc990 msgr2=0x7fd92c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:07:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 shutdown_connections 2023-09-23T05:07:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:07:53.764+0000 7fd933fff700 1 -- 172.21.15.5:0/4206801924 wait complete. 2023-09-23T05:07:53.784 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:07:54.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:54 smithi005 ceph-mon[97638]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-23T05:07:54.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:54 smithi118 ceph-mon[134297]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-23T05:07:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:54 smithi069 ceph-mon[133931]: pgmap v679: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-23T05:07:56.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:56 smithi005 ceph-mon[97638]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-23T05:07:56.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:56 smithi118 ceph-mon[134297]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-23T05:07:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:56 smithi069 ceph-mon[133931]: pgmap v680: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-23T05:07:58.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:07:58 smithi005 ceph-mon[97638]: pgmap v681: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-23T05:07:58.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:07:58 smithi118 ceph-mon[134297]: pgmap v681: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-23T05:07:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:07:58 smithi069 ceph-mon[133931]: pgmap v681: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-23T05:08:00.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:00 smithi005 ceph-mon[97638]: pgmap v682: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-23T05:08:00.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:00 smithi118 ceph-mon[134297]: pgmap v682: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-23T05:08:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:00 smithi069 ceph-mon[133931]: pgmap v682: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-23T05:08:02.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:01 smithi069 ceph-mon[133931]: pgmap v683: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-23T05:08:02.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:01 smithi005 ceph-mon[97638]: pgmap v683: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-23T05:08:02.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:01 smithi118 ceph-mon[134297]: pgmap v683: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-23T05:08:04.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:04 smithi005 ceph-mon[97638]: pgmap v684: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-23T05:08:04.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:04 smithi118 ceph-mon[134297]: pgmap v684: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-23T05:08:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:04 smithi069 ceph-mon[133931]: pgmap v684: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-23T05:08:06.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:06 smithi005 ceph-mon[97638]: pgmap v685: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-23T05:08:06.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:06 smithi118 ceph-mon[134297]: pgmap v685: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-23T05:08:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:06 smithi069 ceph-mon[133931]: pgmap v685: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-23T05:08:07.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:07 smithi005 ceph-mon[97638]: osdmap e121: 12 total, 12 up, 12 in 2023-09-23T05:08:07.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:07 smithi118 ceph-mon[134297]: osdmap e121: 12 total, 12 up, 12 in 2023-09-23T05:08:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:07 smithi069 ceph-mon[133931]: osdmap e121: 12 total, 12 up, 12 in 2023-09-23T05:08:08.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:08 smithi005 ceph-mon[97638]: pgmap v687: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:08.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:08 smithi118 ceph-mon[134297]: pgmap v687: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:08 smithi069 ceph-mon[133931]: pgmap v687: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:10.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:10 smithi118 ceph-mon[134297]: pgmap v688: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:10 smithi069 ceph-mon[133931]: pgmap v688: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:10.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:10 smithi005 ceph-mon[97638]: pgmap v688: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:12.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:11 smithi069 ceph-mon[133931]: pgmap v689: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:11 smithi005 ceph-mon[97638]: pgmap v689: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:12.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:11 smithi118 ceph-mon[134297]: pgmap v689: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:14.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:14 smithi005 ceph-mon[97638]: pgmap v690: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:14.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:14 smithi118 ceph-mon[134297]: pgmap v690: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:14 smithi069 ceph-mon[133931]: pgmap v690: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-23T05:08:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:16 smithi005 ceph-mon[97638]: pgmap v691: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 25 op/s 2023-09-23T05:08:16.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:16 smithi005 ceph-mon[97638]: osdmap e122: 12 total, 12 up, 12 in 2023-09-23T05:08:16.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:16 smithi118 ceph-mon[134297]: pgmap v691: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 25 op/s 2023-09-23T05:08:16.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:16 smithi118 ceph-mon[134297]: osdmap e122: 12 total, 12 up, 12 in 2023-09-23T05:08:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:16 smithi069 ceph-mon[133931]: pgmap v691: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 25 op/s 2023-09-23T05:08:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:16 smithi069 ceph-mon[133931]: osdmap e122: 12 total, 12 up, 12 in 2023-09-23T05:08:18.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:18 smithi005 ceph-mon[97638]: pgmap v693: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:18.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:18 smithi118 ceph-mon[134297]: pgmap v693: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:18 smithi069 ceph-mon[133931]: pgmap v693: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:20.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:20 smithi005 ceph-mon[97638]: pgmap v694: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:20.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:20 smithi118 ceph-mon[134297]: pgmap v694: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:20 smithi069 ceph-mon[133931]: pgmap v694: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:08:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:21 smithi005 ceph-mon[97638]: pgmap v695: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:22.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:21 smithi118 ceph-mon[134297]: pgmap v695: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:21 smithi069 ceph-mon[133931]: pgmap v695: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:23.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:08:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 -- 172.21.15.5:0/383381538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270111e70 msgr2=0x7f3270114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 --2- 172.21.15.5:0/383381538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270111e70 0x7f3270114260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3268009f90 tx=0x7f326800b330 comp rx=0 tx=0).stop 2023-09-23T05:08:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 -- 172.21.15.5:0/383381538 shutdown_connections 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 --2- 172.21.15.5:0/383381538 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270111e70 0x7f3270114260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 --2- 172.21.15.5:0/383381538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32700731c0 0x7f32700735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 --2- 172.21.15.5:0/383381538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3270073ae0 0x7f3270111930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 -- 172.21.15.5:0/383381538 >> 172.21.15.5:0/383381538 conn(0x7f32700fc990 msgr2=0x7f32700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 -- 172.21.15.5:0/383381538 shutdown_connections 2023-09-23T05:08:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.009+0000 7f3274fff700 1 -- 172.21.15.5:0/383381538 wait complete. 2023-09-23T05:08:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.010+0000 7f3274fff700 1 Processor -- start 2023-09-23T05:08:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.010+0000 7f3274fff700 1 -- start start 2023-09-23T05:08:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32700731c0 0x7f327019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 0x7f32701a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701a15f0 con 0x7f3270111e70 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701a1760 con 0x7f32700731c0 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f3274fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32701a1920 con 0x7f3270073ae0 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:38926/0 (socket says 172.21.15.5:38926) 2023-09-23T05:08:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 0x7f32701a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 -- 172.21.15.5:0/3696352335 learned_addr learned my addr 172.21.15.5:0/3696352335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326e59c700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32700731c0 0x7f327019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 -- 172.21.15.5:0/3696352335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32700731c0 msgr2=0x7f327019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32700731c0 0x7f327019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 -- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 msgr2=0x7f32701a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 0x7f32701a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326dd9b700 1 -- 172.21.15.5:0/3696352335 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600095f0 con 0x7f3270073ae0 2023-09-23T05:08:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.011+0000 7f326ed9d700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 0x7f32701a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9a0 tx=0x7f326400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f325f7fe700 1 -- 172.21.15.5:0/3696352335 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264009790 con 0x7f3270073ae0 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f325f7fe700 1 -- 172.21.15.5:0/3696352335 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3264010430 con 0x7f3270073ae0 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268009c40 con 0x7f3270073ae0 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f325f7fe700 1 -- 172.21.15.5:0/3696352335 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264018680 con 0x7f3270073ae0 2023-09-23T05:08:24.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.012+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32701aa380 con 0x7f3270073ae0 2023-09-23T05:08:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.017+0000 7f325f7fe700 1 -- 172.21.15.5:0/3696352335 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f32640105a0 con 0x7f3270073ae0 2023-09-23T05:08:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.018+0000 7f32889c3b80 1 Processor -- start 2023-09-23T05:08:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.018+0000 7f32889c3b80 1 -- start start 2023-09-23T05:08:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.018+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b816207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.018+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b8162b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.018+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f32889c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81589460 con 0x558b81620400 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f32889c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8157ac20 con 0x558b81620d20 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f32889c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81578540 con 0x558b8162bbd0 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f325f7fe700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 0x7f32580a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:38938/0 (socket says 172.21.15.5:38938) 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 -- 172.21.15.5:0/1095246756 learned_addr learned my addr 172.21.15.5:0/1095246756 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326ed9d700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b816207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326e59c700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b8162b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326ed9d700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 0x7f32580a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 -- 172.21.15.5:0/1095246756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 msgr2=0x558b8162b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b8162b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 -- 172.21.15.5:0/1095246756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 msgr2=0x558b816207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b816207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 -- 172.21.15.5:0/1095246756 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b815d68d0 con 0x558b8162bbd0 2023-09-23T05:08:24.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326e59c700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b8162b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:08:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f325f7fe700 1 -- 172.21.15.5:0/3696352335 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3264014070 con 0x7f3270073ae0 2023-09-23T05:08:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.019+0000 7f326dd9b700 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3264079950 tx=0x7f3264007c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93ce407b44ccd83 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f325cff9700 1 -- 172.21.15.5:0/1095246756 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264079080 con 0x558b8162bbd0 2023-09-23T05:08:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f325cff9700 1 -- 172.21.15.5:0/1095246756 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f326409aad0 con 0x558b8162bbd0 2023-09-23T05:08:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f325cff9700 1 -- 172.21.15.5:0/1095246756 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264007a80 con 0x558b8162bbd0 2023-09-23T05:08:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 -- 172.21.15.5:0/1095246756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 msgr2=0x558b8162dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f326ed9d700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 0x7f32580a3bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32680060b0 tx=0x7f3268009700 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3264079950 tx=0x7f3264007c70 comp rx=0 tx=0).stop 2023-09-23T05:08:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 -- 172.21.15.5:0/1095246756 shutdown_connections 2023-09-23T05:08:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b8162dfc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b816207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 --2- 172.21.15.5:0/1095246756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b8162b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.020+0000 7f32889c3b80 1 -- 172.21.15.5:0/1095246756 >> 172.21.15.5:0/1095246756 conn(0x558b81595260 msgr2=0x558b8161c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.021+0000 7f32889c3b80 1 -- 172.21.15.5:0/1095246756 shutdown_connections 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.021+0000 7f32889c3b80 1 -- 172.21.15.5:0/1095246756 wait complete. 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.021+0000 7f32889c3b80 1 -- start start 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 Processor -- start 2023-09-23T05:08:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620d20 0x558b81628870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b8162bbd0 0x558b81628f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81573e30 con 0x558b81620d20 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81510710 con 0x558b8162bbd0 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.022+0000 7f32889c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b815103b0 con 0x558b81620400 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:38952/0 (socket says 172.21.15.5:38952) 2023-09-23T05:08:24.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 -- 172.21.15.5:0/846949770 learned_addr learned my addr 172.21.15.5:0/846949770 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f32889c3b80 10 client.0 _ll_register_callbacks cb 0x558b8136d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326dd9b700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620d20 0x558b81628870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326ed9d700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b8162bbd0 0x558b81628f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 -- 172.21.15.5:0/846949770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b8162bbd0 msgr2=0x558b81628f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b8162bbd0 0x558b81628f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 -- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620d20 msgr2=0x558b81628870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620d20 0x558b81628870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 -- 172.21.15.5:0/846949770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32701aa380 con 0x558b81620400 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f326e59c700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3260012840 tx=0x7f3260012b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa8ed4a86430a37 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f3255ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.023+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260017070 con 0x558b81620400 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b81614250 con 0x558b81620400 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3260004030 con 0x558b81620400 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32600041f0 con 0x558b81620400 2023-09-23T05:08:24.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 10 client.27934 fetch_fsmap learned FSMap version 81 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x558b8139ede0 con 0x558b81620400 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b81614250 con 0x558b81620400 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f326000f640 con 0x558b81620400 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 10 client.27934 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 10 client.27934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 20 client.27934 populate_metadata read hostname 'smithi005' 2023-09-23T05:08:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8163b530 0x558b8163d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 4 client.27934 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.024+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b8156ce10 con 0x558b8163b530 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.025+0000 7f326dd9b700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8163b530 0x558b8163d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.025+0000 7f326dd9b700 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8163b530 0x558b8163d9f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629b9eb7a33574ef server_cookie=e137fbb84001a457 in_seq=0 out_seq=0 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.025+0000 7f3255ffb700 10 client.27934 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f3255ffb700 1 -- 172.21.15.5:0/846949770 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558b8156ce10 con 0x558b8163b530 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f3255ffb700 10 client.27934 handle_command_reply: tid=0 2023-09-23T05:08:24.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 client.27934 shutdown 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 msgr2=0x558b8162b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3260012840 tx=0x7f3260012b50 comp rx=0 tx=0).stop 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 shutdown_connections 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8163b530 0x558b8163d9f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b8162bbd0 0x558b81628f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620400 0x558b8162b5f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 --2- 172.21.15.5:0/846949770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620d20 0x558b81628870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 >> 172.21.15.5:0/846949770 conn(0x558b81595260 msgr2=0x558b8161c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f3255ffb700 0 client.27934 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 shutdown_connections 2023-09-23T05:08:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.026+0000 7f32889c3b80 1 -- 172.21.15.5:0/846949770 wait complete. 2023-09-23T05:08:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.027+0000 7f32889c3b80 20 client.27934 trim_cache size 0 max 16384 2023-09-23T05:08:24.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 Processor -- start 2023-09-23T05:08:24.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 -- start start 2023-09-23T05:08:24.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620400 0x558b81628f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 0x558b816378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b813f4920 con 0x558b8162bbd0 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81613d90 con 0x558b81620400 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f32889c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b81632c00 con 0x558b81620d20 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:38964/0 (socket says 172.21.15.5:38964) 2023-09-23T05:08:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f326dd9b700 1 -- 172.21.15.5:0/2414281958 learned_addr learned my addr 172.21.15.5:0/2414281958 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.044+0000 7f326ed9d700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 0x558b816378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326e59c700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620400 0x558b81628f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 -- 172.21.15.5:0/2414281958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620400 msgr2=0x558b81628f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620400 0x558b81628f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 -- 172.21.15.5:0/2414281958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 msgr2=0x558b816378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 0x558b816378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 -- 172.21.15.5:0/2414281958 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b81387c60 con 0x558b81620d20 2023-09-23T05:08:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326ed9d700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 0x558b816378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:08:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f326dd9b700 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32640968d0 tx=0x7f3264099e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b1b7dc2c767989 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f325d7fa700 1 -- 172.21.15.5:0/2414281958 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326409a3d0 con 0x558b81620d20 2023-09-23T05:08:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f325d7fa700 1 -- 172.21.15.5:0/2414281958 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32640a2760 con 0x558b81620d20 2023-09-23T05:08:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.045+0000 7f325d7fa700 1 -- 172.21.15.5:0/2414281958 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326409e150 con 0x558b81620d20 2023-09-23T05:08:24.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- 172.21.15.5:0/2414281958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 msgr2=0x558b816326c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32640968d0 tx=0x7f3264099e60 comp rx=0 tx=0).stop 2023-09-23T05:08:24.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- 172.21.15.5:0/2414281958 shutdown_connections 2023-09-23T05:08:24.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b8162bbd0 0x558b816378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620400 0x558b81628f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 --2- 172.21.15.5:0/2414281958 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b81620d20 0x558b816326c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- 172.21.15.5:0/2414281958 >> 172.21.15.5:0/2414281958 conn(0x558b81595260 msgr2=0x558b8161af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- 172.21.15.5:0/2414281958 shutdown_connections 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- 172.21.15.5:0/2414281958 wait complete. 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 -- start start 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.046+0000 7f32889c3b80 1 Processor -- start 2023-09-23T05:08:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b816838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8168e540 con 0x558b81620400 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8168e6b0 con 0x558b81620d20 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b8168e870 con 0x558b8162bbd0 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f32889c3b80 10 client.0 _ll_register_callbacks cb 0x7f3260009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:08:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326dd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b816838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57602/0 (socket says 172.21.15.5:57602) 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326e59c700 1 -- 172.21.15.5:0/3124606800 learned_addr learned my addr 172.21.15.5:0/3124606800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.047+0000 7f326ed9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:38966/0 (socket says 172.21.15.5:38966) 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 -- 172.21.15.5:0/3124606800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 msgr2=0x558b81683e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 -- 172.21.15.5:0/3124606800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 msgr2=0x558b816838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b816838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 -- 172.21.15.5:0/3124606800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b815d68d0 con 0x558b81620400 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326ed9d700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b816838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f326e59c700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3260012560 tx=0x7f326000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af10b9de29f514ef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f3255ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260017040 con 0x558b81620400 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f326409aad0 con 0x558b81620400 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.048+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3260005da0 con 0x558b81620400 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32600052c0 con 0x558b81620400 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 10 client.17856 fetch_fsmap learned FSMap version 81 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x558b813c5250 con 0x558b81620400 2023-09-23T05:08:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f326409aad0 con 0x558b81620400 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 81) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f326001c460 con 0x558b81620400 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 10 client.17856 fetch_fsmap finished waiting for FSMap version 81 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 20 client.17856 populate_metadata read hostname 'smithi005' 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8168ffe0 0x558b81692400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 4 client.17856 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b8156cb30 con 0x558b8168ffe0 2023-09-23T05:08:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.049+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8168ffe0 0x558b81692400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f326dd9b700 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8168ffe0 0x558b81692400 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402f3ed43acfedb7 server_cookie=b43b60967b99b511 in_seq=0 out_seq=0 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f3255ffb700 10 client.17856 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f3255ffb700 1 -- 172.21.15.5:0/3124606800 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558b8156cb30 con 0x558b8168ffe0 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f3255ffb700 10 client.17856 handle_command_reply: tid=0 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 client.17856 shutdown 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 msgr2=0x558b81683390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3260012560 tx=0x7f326000f3c0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 shutdown_connections 2023-09-23T05:08:24.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x558b8168ffe0 0x558b81692400 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x558b8162bbd0 0x558b81683e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b81620400 0x558b81683390 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 --2- 172.21.15.5:0/3124606800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b81620d20 0x558b816838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f3255ffb700 0 client.17856 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 >> 172.21.15.5:0/3124606800 conn(0x558b81595260 msgr2=0x558b81623c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.050+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 shutdown_connections 2023-09-23T05:08:24.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f32889c3b80 1 -- 172.21.15.5:0/3124606800 wait complete. 2023-09-23T05:08:24.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f32889c3b80 20 client.17856 trim_cache size 0 max 16384 2023-09-23T05:08:24.063 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:08:24.063 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:08:24.063 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:08:24.064 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:08:24.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 msgr2=0x7f32580a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 0x7f32580a3bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32680060b0 tx=0x7f3268009700 comp rx=0 tx=0).stop 2023-09-23T05:08:24.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 msgr2=0x7f327019dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f326400e9a0 tx=0x7f326400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 shutdown_connections 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f32580a1710 0x7f32580a3bd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3270111e70 0x7f32701a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32700731c0 0x7f327019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 --2- 172.21.15.5:0/3696352335 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3270073ae0 0x7f327019dc40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:24.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 >> 172.21.15.5:0/3696352335 conn(0x7f32700fc990 msgr2=0x7f3270114060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 shutdown_connections 2023-09-23T05:08:24.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:24.051+0000 7f3274fff700 1 -- 172.21.15.5:0/3696352335 wait complete. 2023-09-23T05:08:24.070 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:08:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:24 smithi005 ceph-mon[97638]: pgmap v696: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:24.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:24 smithi118 ceph-mon[134297]: pgmap v696: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:24 smithi069 ceph-mon[133931]: pgmap v696: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:26.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:26 smithi005 ceph-mon[97638]: pgmap v697: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:26.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:26 smithi118 ceph-mon[134297]: pgmap v697: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:26 smithi069 ceph-mon[133931]: pgmap v697: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:08:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:28 smithi005 ceph-mon[97638]: pgmap v698: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:08:28.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:28 smithi118 ceph-mon[134297]: pgmap v698: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:08:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:28 smithi069 ceph-mon[133931]: pgmap v698: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:08:30.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:30 smithi005 ceph-mon[97638]: pgmap v699: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:30.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:30 smithi118 ceph-mon[134297]: pgmap v699: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:30 smithi069 ceph-mon[133931]: pgmap v699: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:31 smithi005 ceph-mon[97638]: pgmap v700: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:32.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:31 smithi118 ceph-mon[134297]: pgmap v700: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:32 smithi069 ceph-mon[133931]: pgmap v700: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:34 smithi005 ceph-mon[97638]: pgmap v701: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:08:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:34.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:34 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:34.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:34 smithi118 ceph-mon[134297]: pgmap v701: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:08:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:34.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:34 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:34 smithi069 ceph-mon[133931]: pgmap v701: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:08:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:34 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:35.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:35 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:08:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:35 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:08:35.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:35 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:35 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:08:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:35 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:08:35.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:35 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:35 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:08:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:35 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:08:35.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:35 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:08:36.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:36 smithi005 ceph-mon[97638]: pgmap v702: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:36.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:36 smithi118 ceph-mon[134297]: pgmap v702: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:36 smithi069 ceph-mon[133931]: pgmap v702: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:37.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:37 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:08:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:37 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:08:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:37 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:08:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:38 smithi069 ceph-mon[133931]: pgmap v703: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:38.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:38 smithi005 ceph-mon[97638]: pgmap v703: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:38.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:38 smithi118 ceph-mon[134297]: pgmap v703: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:40 smithi069 ceph-mon[133931]: pgmap v704: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:40.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:40 smithi005 ceph-mon[97638]: pgmap v704: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:40 smithi118 ceph-mon[134297]: pgmap v704: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:42 smithi005 ceph-mon[97638]: pgmap v705: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:42.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:42 smithi118 ceph-mon[134297]: pgmap v705: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:42 smithi069 ceph-mon[133931]: pgmap v705: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:44 smithi005 ceph-mon[97638]: pgmap v706: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:44.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:44 smithi118 ceph-mon[134297]: pgmap v706: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:44 smithi069 ceph-mon[133931]: pgmap v706: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:46.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:46 smithi118 ceph-mon[134297]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:46 smithi069 ceph-mon[133931]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:46.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:46 smithi005 ceph-mon[97638]: pgmap v707: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:48.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:48 smithi118 ceph-mon[134297]: pgmap v708: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:48 smithi069 ceph-mon[133931]: pgmap v708: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:48 smithi005 ceph-mon[97638]: pgmap v708: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:50 smithi069 ceph-mon[133931]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:50.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:50 smithi005 ceph-mon[97638]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:50 smithi118 ceph-mon[134297]: pgmap v709: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:52 smithi005 ceph-mon[97638]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:52.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:52 smithi118 ceph-mon[134297]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:52 smithi069 ceph-mon[133931]: pgmap v710: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:08:54.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:08:54.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.288+0000 7fd2d4fff700 1 -- 172.21.15.5:0/1136355240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 msgr2=0x7fd2d0073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/1136355240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0073630 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009990 tx=0x7fd2c0009ca0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 -- 172.21.15.5:0/1136355240 shutdown_connections 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/1136355240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d010d6a0 0x7fd2d010fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/1136355240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0073630 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/1136355240 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d0073b70 0x7fd2d010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 -- 172.21.15.5:0/1136355240 >> 172.21.15.5:0/1136355240 conn(0x7fd2d00fc990 msgr2=0x7fd2d00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 -- 172.21.15.5:0/1136355240 shutdown_connections 2023-09-23T05:08:54.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.289+0000 7fd2d4fff700 1 -- 172.21.15.5:0/1136355240 wait complete. 2023-09-23T05:08:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 Processor -- start 2023-09-23T05:08:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 -- start start 2023-09-23T05:08:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 0x7fd2d019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e150 con 0x7fd2d0073250 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e2c0 con 0x7fd2d0073b70 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.290+0000 7fd2d4fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019e480 con 0x7fd2d010d6a0 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ced9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 0x7fd2d019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51946/0 (socket says 172.21.15.5:51946) 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:35696/0 (socket says 172.21.15.5:35696) 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ce59c700 1 -- 172.21.15.5:0/3730186970 learned_addr learned my addr 172.21.15.5:0/3730186970 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3730186970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 msgr2=0x7fd2d019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 0x7fd2d019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 msgr2=0x7fd2d0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3730186970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00095f0 con 0x7fd2d0073b70 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 0x7fd2d019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.291+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800e9f0 tx=0x7fd2b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.292+0000 7fd2c77fe700 1 -- 172.21.15.5:0/3730186970 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800c9c0 con 0x7fd2d0073b70 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.292+0000 7fd2c77fe700 1 -- 172.21.15.5:0/3730186970 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8010430 con 0x7fd2d0073b70 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.292+0000 7fd2c77fe700 1 -- 172.21.15.5:0/3730186970 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8018680 con 0x7fd2d0073b70 2023-09-23T05:08:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.292+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d019e760 con 0x7fd2d0073b70 2023-09-23T05:08:54.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.292+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d01a5fa0 con 0x7fd2d0073b70 2023-09-23T05:08:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.295+0000 7fd2c77fe700 1 -- 172.21.15.5:0/3730186970 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd2b8010860 con 0x7fd2d0073b70 2023-09-23T05:08:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2e88f7b80 1 Processor -- start 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2e88f7b80 1 -- start start 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2c77fe700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 0x7fd2bc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2c77fe700 1 -- 172.21.15.5:0/3730186970 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd2b8014070 con 0x7fd2d0073b70 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877400 0x557b168777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.296+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16882bd0 0x557b16884fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b167e0460 con 0x557b16877400 2023-09-23T05:08:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b167d1c20 con 0x557b16882bd0 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b167cf540 con 0x557b16877d20 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 0x7fd2bc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877400 0x557b168777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54066/0 (socket says 172.21.15.5:54066) 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 -- 172.21.15.5:0/531426777 learned_addr learned my addr 172.21.15.5:0/531426777 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16882bd0 0x557b16884fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 -- 172.21.15.5:0/531426777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16882bd0 msgr2=0x557b16884fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16882bd0 0x557b16884fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 -- 172.21.15.5:0/531426777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877400 msgr2=0x557b168777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877400 0x557b168777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 -- 172.21.15.5:0/531426777 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b1682d8d0 con 0x557b16877d20 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.297+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8096e40 tx=0x7fd2b809a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7376803e83e2cc3f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 0x7fd2bc0a3bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009ec0 tx=0x7fd2c8009920 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2c4ff9700 1 -- 172.21.15.5:0/531426777 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b809a3d0 con 0x557b16877d20 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2c4ff9700 1 -- 172.21.15.5:0/531426777 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b809e520 con 0x557b16877d20 2023-09-23T05:08:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2c4ff9700 1 -- 172.21.15.5:0/531426777 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b809a630 con 0x557b16877d20 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/531426777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 msgr2=0x557b16882690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8096e40 tx=0x7fd2b809a2b0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/531426777 shutdown_connections 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16882bd0 0x557b16884fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877400 0x557b168777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/531426777 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882690 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/531426777 >> 172.21.15.5:0/531426777 conn(0x557b167ec260 msgr2=0x557b16873790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/531426777 shutdown_connections 2023-09-23T05:08:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.298+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/531426777 wait complete. 2023-09-23T05:08:54.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.299+0000 7fd2e88f7b80 1 -- start start 2023-09-23T05:08:54.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.299+0000 7fd2e88f7b80 1 Processor -- start 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 0x557b1687f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 0x557b1687ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b167cae30 con 0x557b16877d20 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b16767710 con 0x557b16877400 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b167673b0 con 0x557b16882bd0 2023-09-23T05:08:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2e88f7b80 10 client.0 _ll_register_callbacks cb 0x557b165c4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:35708/0 (socket says 172.21.15.5:35708) 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ce59c700 1 -- 172.21.15.5:0/3798717804 learned_addr learned my addr 172.21.15.5:0/3798717804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 0x557b1687ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 0x557b1687f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3798717804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 msgr2=0x557b1687ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 0x557b1687ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3798717804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 msgr2=0x557b16882620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2c7fff700 1 -- 172.21.15.5:0/3798717804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d01a5fa0 con 0x557b16877d20 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.300+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 0x557b1687ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 0x557b1687f830 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd2b809a0d0 tx=0x7fd2b80107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9499a7ba15410dbc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b809a3d0 con 0x557b16877d20 2023-09-23T05:08:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b1686b250 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd2b8099ad0 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b809a630 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2e88f7b80 10 client.17868 fetch_fsmap learned FSMap version 82 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x557b165f5de0 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.301+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b1686b250 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd2b80cd5b0 con 0x557b16877d20 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 10 client.17868 fetch_fsmap finished waiting for FSMap version 82 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:08:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 20 client.17868 populate_metadata read hostname 'smithi005' 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b16892270 0x557b16894690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 4 client.17868 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b167c3e10 con 0x557b16892270 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.302+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b16892270 0x557b16894690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b16892270 0x557b16894690 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad42c233beaf3c4c server_cookie=b1dd59abaee171b5 in_seq=0 out_seq=0 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2b5ffb700 10 client.17868 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/3798717804 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b167c3e10 con 0x557b16892270 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2b5ffb700 10 client.17868 handle_command_reply: tid=0 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2e88f7b80 1 client.17868 shutdown 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 msgr2=0x557b1687f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.303+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 0x557b1687f830 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd2b809a0d0 tx=0x7fd2b80107d0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 shutdown_connections 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b16892270 0x557b16894690 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16882bd0 0x557b1687ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2b5ffb700 0 client.17868 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b16882620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/3798717804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16877d20 0x557b1687f830 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 >> 172.21.15.5:0/3798717804 conn(0x557b167ec260 msgr2=0x557b16873320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 shutdown_connections 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/3798717804 wait complete. 2023-09-23T05:08:54.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.304+0000 7fd2e88f7b80 20 client.17868 trim_cache size 0 max 16384 2023-09-23T05:08:54.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.321+0000 7fd2e88f7b80 1 Processor -- start 2023-09-23T05:08:54.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.321+0000 7fd2e88f7b80 1 -- start start 2023-09-23T05:08:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b1688e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b1687cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1664b920 con 0x557b16882bd0 2023-09-23T05:08:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b16878170 con 0x557b16877400 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b16894d20 con 0x557b16877d20 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54090/0 (socket says 172.21.15.5:54090) 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 -- 172.21.15.5:0/326772318 learned_addr learned my addr 172.21.15.5:0/326772318 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b1687cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b1688e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 -- 172.21.15.5:0/326772318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 msgr2=0x557b1688e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b1688e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 -- 172.21.15.5:0/326772318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 msgr2=0x557b1687cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b1687cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2c7fff700 1 -- 172.21.15.5:0/326772318 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b165dec60 con 0x557b16877d20 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.322+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b1688e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8096e40 tx=0x7fd2b809e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79dcfcebcf4fa577 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2c57fa700 1 -- 172.21.15.5:0/326772318 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8099d30 con 0x557b16877d20 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2c57fa700 1 -- 172.21.15.5:0/326772318 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b809a3d0 con 0x557b16877d20 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2c57fa700 1 -- 172.21.15.5:0/326772318 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b809a890 con 0x557b16877d20 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/326772318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 msgr2=0x557b16882150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8096e40 tx=0x7fd2b809e430 comp rx=0 tx=0).stop 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/326772318 shutdown_connections 2023-09-23T05:08:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b1687cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877400 0x557b1688e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/326772318 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877d20 0x557b16882150 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.323+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/326772318 >> 172.21.15.5:0/326772318 conn(0x557b167ec260 msgr2=0x557b16871f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.324+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/326772318 shutdown_connections 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.324+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/326772318 wait complete. 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.324+0000 7fd2e88f7b80 1 -- start start 2023-09-23T05:08:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.324+0000 7fd2e88f7b80 1 Processor -- start 2023-09-23T05:08:54.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877d20 0x557b16887e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b168df570 con 0x557b16882bd0 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b168df6e0 con 0x557b16877d20 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b168df8a0 con 0x557b16877400 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2e88f7b80 10 client.0 _ll_register_callbacks cb 0x7fd2c80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ced9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54104/0 (socket says 172.21.15.5:54104) 2023-09-23T05:08:54.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ced9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 -- 172.21.15.5:0/1774594498 learned_addr learned my addr 172.21.15.5:0/1774594498 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877d20 0x557b16887e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 -- 172.21.15.5:0/1774594498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877d20 msgr2=0x557b16887e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877d20 0x557b16887e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 -- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 msgr2=0x557b168df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ce59c700 1 -- 172.21.15.5:0/1774594498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b165f5de0 con 0x557b16877400 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.325+0000 7fd2ced9d700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2ce59c700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c010 tx=0x7fd2c0005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5755080241aef867 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c001d040 con 0x557b16877400 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b809e520 con 0x557b16877400 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd2c00054c0 con 0x557b16877400 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c0005680 con 0x557b16877400 2023-09-23T05:08:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2e88f7b80 10 client.27958 fetch_fsmap learned FSMap version 82 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) v3 -- 0x557b1661c250 con 0x557b16877400 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.326+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b809e520 con 0x557b16877400 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 82) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd2c0025490 con 0x557b16877400 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 10 client.27958 fetch_fsmap finished waiting for FSMap version 82 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 10 client.27958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 20 client.27958 populate_metadata read hostname 'smithi005' 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b168e6fa0 0x557b168e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 4 client.27958 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:08:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b167c3b30 con 0x557b168e6fa0 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.327+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b168e6fa0 0x557b168e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2c7fff700 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b168e6fa0 0x557b168e93c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824e914adf11703d server_cookie=afbcada7d969ec04 in_seq=0 out_seq=0 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2b5ffb700 10 client.27958 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2b5ffb700 1 -- 172.21.15.5:0/1774594498 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557b167c3b30 con 0x557b168e6fa0 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2b5ffb700 10 client.27958 handle_command_reply: tid=0 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 client.27958 shutdown 2023-09-23T05:08:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 msgr2=0x557b168878f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c010 tx=0x7fd2c0005910 comp rx=0 tx=0).stop 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 shutdown_connections 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x557b168e6fa0 0x557b168e93c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b16882bd0 0x557b168df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x557b16877400 0x557b168878f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 --2- 172.21.15.5:0/1774594498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b16877d20 0x557b16887e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2b5ffb700 0 client.27958 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:08:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 >> 172.21.15.5:0/1774594498 conn(0x557b167ec260 msgr2=0x557b1687ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 shutdown_connections 2023-09-23T05:08:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 1 -- 172.21.15.5:0/1774594498 wait complete. 2023-09-23T05:08:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.328+0000 7fd2e88f7b80 20 client.27958 trim_cache size 0 max 16384 2023-09-23T05:08:54.339 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:08:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:08:54.341 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:08:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 msgr2=0x7fd2bc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 0x7fd2bc0a3bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009ec0 tx=0x7fd2c8009920 comp rx=0 tx=0).stop 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 msgr2=0x7fd2d01996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800e9f0 tx=0x7fd2b800ed00 comp rx=0 tx=0).stop 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 shutdown_connections 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd2bc0a1710 0x7fd2bc0a3bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd2d010d6a0 0x7fd2d019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0073250 0x7fd2d0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 --2- 172.21.15.5:0/3730186970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2d0073b70 0x7fd2d01996b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:08:54.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 >> 172.21.15.5:0/3730186970 conn(0x7fd2d00fc990 msgr2=0x7fd2d0105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:08:54.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 shutdown_connections 2023-09-23T05:08:54.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:08:54.329+0000 7fd2d4fff700 1 -- 172.21.15.5:0/3730186970 wait complete. 2023-09-23T05:08:54.348 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:08:54.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:54 smithi005 ceph-mon[97638]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:54.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:54 smithi118 ceph-mon[134297]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:54 smithi069 ceph-mon[133931]: pgmap v711: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:08:56.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:56 smithi118 ceph-mon[134297]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:56 smithi069 ceph-mon[133931]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:56.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:56 smithi005 ceph-mon[97638]: pgmap v712: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:08:58 smithi069 ceph-mon[133931]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:58.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:08:58 smithi005 ceph-mon[97638]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:08:58.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:08:58 smithi118 ceph-mon[134297]: pgmap v713: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:00 smithi069 ceph-mon[133931]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:00.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:00 smithi005 ceph-mon[97638]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:00.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:00 smithi118 ceph-mon[134297]: pgmap v714: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:02 smithi005 ceph-mon[97638]: pgmap v715: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:02.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:02 smithi118 ceph-mon[134297]: pgmap v715: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:02.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:02 smithi069 ceph-mon[133931]: pgmap v715: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:04.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:04 smithi118 ceph-mon[134297]: pgmap v716: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:09:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:04 smithi069 ceph-mon[133931]: pgmap v716: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:09:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:04 smithi005 ceph-mon[97638]: pgmap v716: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:09:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:06 smithi069 ceph-mon[133931]: pgmap v717: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:09:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:06 smithi005 ceph-mon[97638]: pgmap v717: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:09:06.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:06 smithi118 ceph-mon[134297]: pgmap v717: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:09:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:07 smithi069 ceph-mon[133931]: osdmap e123: 12 total, 12 up, 12 in 2023-09-23T05:09:07.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:07 smithi069 ceph-mon[133931]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:09:07.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:07 smithi005 ceph-mon[97638]: osdmap e123: 12 total, 12 up, 12 in 2023-09-23T05:09:07.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:07 smithi005 ceph-mon[97638]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:09:07.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:07 smithi118 ceph-mon[134297]: osdmap e123: 12 total, 12 up, 12 in 2023-09-23T05:09:07.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:07 smithi118 ceph-mon[134297]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-23T05:09:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:08 smithi069 ceph-mon[133931]: pgmap v719: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:08.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:08 smithi005 ceph-mon[97638]: pgmap v719: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:08.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:08 smithi118 ceph-mon[134297]: pgmap v719: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:10 smithi069 ceph-mon[133931]: pgmap v720: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:10.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:10 smithi005 ceph-mon[97638]: pgmap v720: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:10.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:10 smithi118 ceph-mon[134297]: pgmap v720: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:12.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:12 smithi005 ceph-mon[97638]: pgmap v721: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:12.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:12 smithi118 ceph-mon[134297]: pgmap v721: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:12 smithi069 ceph-mon[133931]: pgmap v721: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:14 smithi069 ceph-mon[133931]: pgmap v722: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:14.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:14 smithi005 ceph-mon[97638]: pgmap v722: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:14 smithi118 ceph-mon[134297]: pgmap v722: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:09:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:16 smithi069 ceph-mon[133931]: pgmap v723: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:09:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:16 smithi069 ceph-mon[133931]: osdmap e124: 12 total, 12 up, 12 in 2023-09-23T05:09:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:16 smithi005 ceph-mon[97638]: pgmap v723: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:09:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:16 smithi005 ceph-mon[97638]: osdmap e124: 12 total, 12 up, 12 in 2023-09-23T05:09:16.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:16 smithi118 ceph-mon[134297]: pgmap v723: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:09:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:16 smithi118 ceph-mon[134297]: osdmap e124: 12 total, 12 up, 12 in 2023-09-23T05:09:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:18 smithi069 ceph-mon[133931]: pgmap v725: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:18.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:18 smithi005 ceph-mon[97638]: pgmap v725: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:18 smithi118 ceph-mon[134297]: pgmap v725: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:20 smithi069 ceph-mon[133931]: pgmap v726: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:20 smithi005 ceph-mon[97638]: pgmap v726: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:20.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:20 smithi118 ceph-mon[134297]: pgmap v726: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:09:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:22 smithi005 ceph-mon[97638]: pgmap v727: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:22 smithi118 ceph-mon[134297]: pgmap v727: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:22 smithi069 ceph-mon[133931]: pgmap v727: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:24.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:09:24.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:24 smithi005 ceph-mon[97638]: pgmap v728: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:24 smithi069 ceph-mon[133931]: pgmap v728: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 -- 172.21.15.5:0/402504993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 msgr2=0x7f75f41036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 --2- 172.21.15.5:0/402504993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f41036c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009990 tx=0x7f75e4009ca0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 -- 172.21.15.5:0/402504993 shutdown_connections 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 --2- 172.21.15.5:0/402504993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4108400 0x7f75f410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 --2- 172.21.15.5:0/402504993 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f41036c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 --2- 172.21.15.5:0/402504993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4103c90 0x7f75f4107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 -- 172.21.15.5:0/402504993 >> 172.21.15.5:0/402504993 conn(0x7f75f40feb60 msgr2=0x7f75f4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 -- 172.21.15.5:0/402504993 shutdown_connections 2023-09-23T05:09:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.568+0000 7f75f8bff700 1 -- 172.21.15.5:0/402504993 wait complete. 2023-09-23T05:09:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.569+0000 7f75f8bff700 1 Processor -- start 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.569+0000 7f75f8bff700 1 -- start start 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4108400 0x7f75f4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f419dbc0 0x7f75f419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f419e640 con 0x7f75f419dbc0 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f419e7b0 con 0x7f75f4108400 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f8bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f419e970 con 0x7f75f41032e0 2023-09-23T05:09:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57024/0 (socket says 172.21.15.5:57024) 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 -- 172.21.15.5:0/3407615811 learned_addr learned my addr 172.21.15.5:0/3407615811 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f419dbc0 0x7f75f419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4108400 0x7f75f4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 -- 172.21.15.5:0/3407615811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4108400 msgr2=0x7f75f4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4108400 0x7f75f4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 -- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f419dbc0 msgr2=0x7f75f419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f419dbc0 0x7f75f419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.570+0000 7f75f259c700 1 -- 172.21.15.5:0/3407615811 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40095f0 con 0x7f75f41032e0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75f259c700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005770 tx=0x7f75e40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75e37fe700 1 -- 172.21.15.5:0/3407615811 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e401d070 con 0x7f75f41032e0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75e37fe700 1 -- 172.21.15.5:0/3407615811 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e4022470 con 0x7f75f41032e0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75e37fe700 1 -- 172.21.15.5:0/3407615811 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4017650 con 0x7f75f41032e0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f41a5e70 con 0x7f75f41032e0 2023-09-23T05:09:24.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.571+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41a61e0 con 0x7f75f41032e0 2023-09-23T05:09:24.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.574+0000 7f75e37fe700 1 -- 172.21.15.5:0/3407615811 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f75e400fb40 con 0x7f75f41032e0 2023-09-23T05:09:24.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.575+0000 7f760c58eb80 1 Processor -- start 2023-09-23T05:09:24.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.575+0000 7f760c58eb80 1 -- start start 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388a1400 0x55a4388a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43880a460 con 0x55a4388acbd0 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4387fbc20 con 0x55a4388a1d20 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f760c58eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4387f9540 con 0x55a4388a1400 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75e37fe700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 0x7f75dc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75e37fe700 1 -- 172.21.15.5:0/3407615811 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f75e40ceb70 con 0x7f75f41032e0 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53474/0 (socket says 172.21.15.5:53474) 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f1d9b700 1 -- 172.21.15.5:0/3825302005 learned_addr learned my addr 172.21.15.5:0/3825302005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388a1400 0x55a4388a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f259c700 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.576+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 0x7f75dc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 -- 172.21.15.5:0/3825302005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388a1400 msgr2=0x55a4388a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388a1400 0x55a4388a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 -- 172.21.15.5:0/3825302005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 msgr2=0x55a4388ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 -- 172.21.15.5:0/3825302005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4388578d0 con 0x55a4388acbd0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009c80 tx=0x7f75e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ef6a3bd894e8c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 0x7f75dc0a3bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009ec0 tx=0x7f75ec00b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75e0ff9700 1 -- 172.21.15.5:0/3825302005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8009750 con 0x55a4388acbd0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75e0ff9700 1 -- 172.21.15.5:0/3825302005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e80044f0 con 0x55a4388acbd0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.577+0000 7f75e0ff9700 1 -- 172.21.15.5:0/3825302005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8003e90 con 0x55a4388acbd0 2023-09-23T05:09:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 -- 172.21.15.5:0/3825302005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 msgr2=0x55a4388aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009c80 tx=0x7f75e8009f90 comp rx=0 tx=0).stop 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 -- 172.21.15.5:0/3825302005 shutdown_connections 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388acbd0 0x55a4388aefc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388a1400 0x55a4388a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 --2- 172.21.15.5:0/3825302005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 -- 172.21.15.5:0/3825302005 >> 172.21.15.5:0/3825302005 conn(0x55a438816260 msgr2=0x55a43889d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 -- 172.21.15.5:0/3825302005 shutdown_connections 2023-09-23T05:09:24.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.578+0000 7f760c58eb80 1 -- 172.21.15.5:0/3825302005 wait complete. 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.579+0000 7f760c58eb80 1 -- start start 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.579+0000 7f760c58eb80 1 Processor -- start 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388ac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4387f4e30 con 0x55a4388a1d20 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438791710 con 0x55a4388a1400 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4387913b0 con 0x55a4388acbd0 2023-09-23T05:09:24.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f760c58eb80 10 client.0 _ll_register_callbacks cb 0x55a4385ee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57050/0 (socket says 172.21.15.5:57050) 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4158578526 learned_addr learned my addr 172.21.15.5:0/4158578526 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53482/0 (socket says 172.21.15.5:53482) 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f259c700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.580+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4158578526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 msgr2=0x55a4388ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 msgr2=0x55a4388a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4158578526 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e80098b0 con 0x55a4388acbd0 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01ae00 tx=0x7f75ec004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96409cf27f50e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec021070 con 0x55a4388acbd0 2023-09-23T05:09:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a438895250 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f75ec008240 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec00e070 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f760c58eb80 10 client.27970 fetch_fsmap learned FSMap version 83 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.581+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55a43861fde0 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a438895250 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75ec007500 con 0x55a4388acbd0 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 10 client.27970 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:09:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 20 client.27970 populate_metadata read hostname 'smithi005' 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a4388bc720 0x55a4388beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 4 client.27970 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4387ede10 con 0x55a4388bc720 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.582+0000 7f75f259c700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a4388bc720 0x55a4388beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.583+0000 7f75f259c700 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a4388bc720 0x55a4388beb40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e466354a52f5168 server_cookie=2982a3c7d9931c0b in_seq=0 out_seq=0 2023-09-23T05:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.583+0000 7f75d9ffb700 10 client.27970 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.583+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4158578526 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4387ede10 con 0x55a4388bc720 2023-09-23T05:09:24.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.583+0000 7f75d9ffb700 10 client.27970 handle_command_reply: tid=0 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.583+0000 7f760c58eb80 1 client.27970 shutdown 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 msgr2=0x55a4388aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01ae00 tx=0x7f75ec004b40 comp rx=0 tx=0).stop 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 shutdown_connections 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a4388bc720 0x55a4388beb40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388aa030 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f75d9ffb700 0 client.27970 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4158578526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 >> 172.21.15.5:0/4158578526 conn(0x55a438816260 msgr2=0x55a43889d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 shutdown_connections 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 1 -- 172.21.15.5:0/4158578526 wait complete. 2023-09-23T05:09:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.584+0000 7f760c58eb80 20 client.27970 trim_cache size 0 max 16384 2023-09-23T05:09:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 Processor -- start 2023-09-23T05:09:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 -- start start 2023-09-23T05:09:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1400 0x55a4388a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438675920 con 0x55a4388a1400 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4388a7e50 con 0x55a4388a1d20 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f760c58eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4388a5740 con 0x55a4388acbd0 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f75f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1400 0x55a4388a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57064/0 (socket says 172.21.15.5:57064) 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.602+0000 7f75f2d9d700 1 -- 172.21.15.5:0/277497989 learned_addr learned my addr 172.21.15.5:0/277497989 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f1d9b700 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 -- 172.21.15.5:0/277497989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 msgr2=0x55a4388b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 -- 172.21.15.5:0/277497989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1400 msgr2=0x55a4388a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1400 0x55a4388a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 -- 172.21.15.5:0/277497989 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a438608c60 con 0x55a4388acbd0 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0078b0 tx=0x7f75ec00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=386b095094b59ff0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75e17fa700 1 -- 172.21.15.5:0/277497989 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec021070 con 0x55a4388acbd0 2023-09-23T05:09:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75e17fa700 1 -- 172.21.15.5:0/277497989 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75ec022040 con 0x55a4388acbd0 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.603+0000 7f75e17fa700 1 -- 172.21.15.5:0/277497989 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec008480 con 0x55a4388acbd0 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- 172.21.15.5:0/277497989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 msgr2=0x55a4388a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0078b0 tx=0x7f75ec00af90 comp rx=0 tx=0).stop 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- 172.21.15.5:0/277497989 shutdown_connections 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388a4c30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1400 0x55a4388a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 --2- 172.21.15.5:0/277497989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- 172.21.15.5:0/277497989 >> 172.21.15.5:0/277497989 conn(0x55a438816260 msgr2=0x55a4388ac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- 172.21.15.5:0/277497989 shutdown_connections 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- 172.21.15.5:0/277497989 wait complete. 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.604+0000 7f760c58eb80 1 -- start start 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 Processor -- start 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4388affb0 con 0x55a4388a1d20 2023-09-23T05:09:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4388b0120 con 0x55a4388a1400 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4388b02e0 con 0x55a4388acbd0 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f760c58eb80 10 client.0 _ll_register_callbacks cb 0x7f75ec008d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f75f259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:57066/0 (socket says 172.21.15.5:57066) 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4126235316 learned_addr learned my addr 172.21.15.5:0/4126235316 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.605+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4126235316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 msgr2=0x55a4388b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 msgr2=0x55a4388b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 -- 172.21.15.5:0/4126235316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a43861fde0 con 0x55a4388acbd0 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f259c700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75f2d9d700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01aeb0 tx=0x7f75ec008470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5abb7a8329e4182e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:09:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00e070 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4388af450 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f75ec01b450 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec021040 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f760c58eb80 10 client.27976 fetch_fsmap learned FSMap version 83 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.606+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55a438646250 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4388af450 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75ec012070 con 0x55a4388acbd0 2023-09-23T05:09:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 10 client.27976 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 10 client.27976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 20 client.27976 populate_metadata read hostname 'smithi005' 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a438911510 0x55a438913900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 4 client.27976 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f75e80044f0 con 0x55a438911510 2023-09-23T05:09:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.607+0000 7f75f259c700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a438911510 0x55a438913900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:24.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f75f259c700 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a438911510 0x55a438913900 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d6c0e9d4cefde3 server_cookie=9978969a91ef0120 in_seq=0 out_seq=0 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f75d9ffb700 10 client.27976 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f75d9ffb700 1 -- 172.21.15.5:0/4126235316 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f75e80044f0 con 0x55a438911510 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f75d9ffb700 10 client.27976 handle_command_reply: tid=0 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 client.27976 shutdown 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 msgr2=0x55a4388af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01aeb0 tx=0x7f75ec008470 comp rx=0 tx=0).stop 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 shutdown_connections 2023-09-23T05:09:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55a438911510 0x55a438913900 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55a4388acbd0 0x55a4388af8a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a4388a1400 0x55a4388b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 --2- 172.21.15.5:0/4126235316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4388a1d20 0x55a4388b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f75d9ffb700 0 client.27976 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 >> 172.21.15.5:0/4126235316 conn(0x55a438816260 msgr2=0x55a43889c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 shutdown_connections 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 1 -- 172.21.15.5:0/4126235316 wait complete. 2023-09-23T05:09:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.608+0000 7f760c58eb80 20 client.27976 trim_cache size 0 max 16384 2023-09-23T05:09:24.619 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:09:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:09:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:09:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:09:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:09:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:09:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:09:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:09:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:09:24.620 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 msgr2=0x7f75dc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 0x7f75dc0a3bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009ec0 tx=0x7f75ec00b040 comp rx=0 tx=0).stop 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 msgr2=0x7f75f4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005770 tx=0x7f75e40057a0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 shutdown_connections 2023-09-23T05:09:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f75dc0a1710 0x7f75dc0a3bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f419dbc0 0x7f75f419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f75f41032e0 0x7f75f4199220 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 --2- 172.21.15.5:0/3407615811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4108400 0x7f75f4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 >> 172.21.15.5:0/3407615811 conn(0x7f75f40feb60 msgr2=0x7f75f40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 shutdown_connections 2023-09-23T05:09:24.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:24.609+0000 7f75f8bff700 1 -- 172.21.15.5:0/3407615811 wait complete. 2023-09-23T05:09:24.627 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:09:24.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:24 smithi118 ceph-mon[134297]: pgmap v728: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:26 smithi069 ceph-mon[133931]: pgmap v729: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:26.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:26 smithi005 ceph-mon[97638]: pgmap v729: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:26 smithi118 ceph-mon[134297]: pgmap v729: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:09:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:28 smithi069 ceph-mon[133931]: pgmap v730: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:09:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:28 smithi005 ceph-mon[97638]: pgmap v730: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:09:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:28 smithi118 ceph-mon[134297]: pgmap v730: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-23T05:09:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:30 smithi069 ceph-mon[133931]: pgmap v731: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:30.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:30 smithi005 ceph-mon[97638]: pgmap v731: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:30.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:30 smithi118 ceph-mon[134297]: pgmap v731: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:32 smithi005 ceph-mon[97638]: pgmap v732: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:32 smithi118 ceph-mon[134297]: pgmap v732: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:32 smithi069 ceph-mon[133931]: pgmap v732: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:34 smithi069 ceph-mon[133931]: pgmap v733: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:34.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:34 smithi005 ceph-mon[97638]: pgmap v733: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:34.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:34 smithi118 ceph-mon[134297]: pgmap v733: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: pgmap v734: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:09:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:36 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: pgmap v734: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:36 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.667 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: pgmap v734: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:36.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:09:36.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:36.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:36 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:37 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:09:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:37 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:09:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:37 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:37.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:37 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:09:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:37 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:09:37.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:37 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:37.693 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:37 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:09:37.694 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:37 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:09:37.694 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:37 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:09:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:38 smithi069 ceph-mon[133931]: pgmap v735: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:38.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:38 smithi005 ceph-mon[97638]: pgmap v735: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:38 smithi118 ceph-mon[134297]: pgmap v735: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:40 smithi069 ceph-mon[133931]: pgmap v736: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:40.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:40 smithi005 ceph-mon[97638]: pgmap v736: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:40.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:40 smithi118 ceph-mon[134297]: pgmap v736: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:42 smithi005 ceph-mon[97638]: pgmap v737: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:42.410 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:42 smithi118 ceph-mon[134297]: pgmap v737: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:42 smithi069 ceph-mon[133931]: pgmap v737: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:44 smithi069 ceph-mon[133931]: pgmap v738: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:44.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:44 smithi005 ceph-mon[97638]: pgmap v738: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:44.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:44 smithi118 ceph-mon[134297]: pgmap v738: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:46 smithi069 ceph-mon[133931]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:46.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:46 smithi005 ceph-mon[97638]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:46.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:46 smithi118 ceph-mon[134297]: pgmap v739: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:48 smithi069 ceph-mon[133931]: pgmap v740: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:48 smithi005 ceph-mon[97638]: pgmap v740: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:48.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:48 smithi118 ceph-mon[134297]: pgmap v740: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:50 smithi069 ceph-mon[133931]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:50 smithi005 ceph-mon[97638]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:50 smithi118 ceph-mon[134297]: pgmap v741: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:52 smithi005 ceph-mon[97638]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:52.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:52 smithi118 ceph-mon[134297]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:52 smithi069 ceph-mon[133931]: pgmap v742: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:09:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:54 smithi069 ceph-mon[133931]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:54.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:09:54.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:54 smithi005 ceph-mon[97638]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:54.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:54 smithi118 ceph-mon[134297]: pgmap v743: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:09:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/305057344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb8101a80 msgr2=0x7f1eb8105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/305057344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb8101a80 0x7f1eb8105ac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009990 tx=0x7f1ea4009ca0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/305057344 shutdown_connections 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/305057344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 0x7f1eb810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/305057344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb81010d0 0x7f1eb81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/305057344 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb8101a80 0x7f1eb8105ac0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/305057344 >> 172.21.15.5:0/305057344 conn(0x7f1eb80fc990 msgr2=0x7f1eb80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/305057344 shutdown_connections 2023-09-23T05:09:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.847+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/305057344 wait complete. 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.848+0000 7f1ebf1ff700 1 Processor -- start 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 -- start start 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 0x7f1eb81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb819dad0 0x7f1eb819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb819e550 con 0x7f1eb81061f0 2023-09-23T05:09:54.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb819e6c0 con 0x7f1eb819dad0 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebf1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb819e880 con 0x7f1eb81010d0 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40878/0 (socket says 172.21.15.5:40878) 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/407743678 learned_addr learned my addr 172.21.15.5:0/407743678 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 0x7f1eb81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.849+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb819dad0 0x7f1eb819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/407743678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb819dad0 msgr2=0x7f1eb819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb819dad0 0x7f1eb819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 msgr2=0x7f1eb81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 0x7f1eb81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/407743678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40095f0 con 0x7f1eb81010d0 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00e9a0 tx=0x7f1eac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1eb5ffb700 1 -- 172.21.15.5:0/407743678 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac009790 con 0x7f1eb81010d0 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1eb5ffb700 1 -- 172.21.15.5:0/407743678 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eac010430 con 0x7f1eb81010d0 2023-09-23T05:09:54.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1eb5ffb700 1 -- 172.21.15.5:0/407743678 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac018680 con 0x7f1eb81010d0 2023-09-23T05:09:54.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb81a5de0 con 0x7f1eb81010d0 2023-09-23T05:09:54.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.850+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb81a6210 con 0x7f1eb81010d0 2023-09-23T05:09:54.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.853+0000 7f1eb5ffb700 1 -- 172.21.15.5:0/407743678 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f1eac0105a0 con 0x7f1eb81010d0 2023-09-23T05:09:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1ed2b53b80 1 Processor -- start 2023-09-23T05:09:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1eb5ffb700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 0x7f1ea80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1eb5ffb700 1 -- 172.21.15.5:0/407743678 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f1eac014070 con 0x7f1eb81010d0 2023-09-23T05:09:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1ed2b53b80 1 -- start start 2023-09-23T05:09:54.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 0x7f1ea80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.855+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f52329460 con 0x563f523cbbd0 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5231ac20 con 0x563f523c0d20 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f52318540 con 0x563f523c0400 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40892/0 (socket says 172.21.15.5:40892) 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/1097517917 learned_addr learned my addr 172.21.15.5:0/1097517917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 0x7f1ea80a3bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000f540 tx=0x7f1eb000b040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/1097517917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 msgr2=0x563f523cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/1097517917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 msgr2=0x563f523cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 -- 172.21.15.5:0/1097517917 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f523768d0 con 0x563f523c0400 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.856+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1eac096910 tx=0x7f1eac099d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265d5cb8dd6fe17b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ea2ffd700 1 -- 172.21.15.5:0/1097517917 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac09e560 con 0x563f523c0400 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ea2ffd700 1 -- 172.21.15.5:0/1097517917 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eac0b1070 con 0x563f523c0400 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ea2ffd700 1 -- 172.21.15.5:0/1097517917 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac09a8d0 con 0x563f523c0400 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1097517917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 msgr2=0x563f523c07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1eac096910 tx=0x7f1eac099d70 comp rx=0 tx=0).stop 2023-09-23T05:09:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1097517917 shutdown_connections 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0400 0x563f523c07e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1097517917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1097517917 >> 172.21.15.5:0/1097517917 conn(0x563f52335260 msgr2=0x563f523bc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1097517917 shutdown_connections 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.857+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1097517917 wait complete. 2023-09-23T05:09:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.858+0000 7f1ed2b53b80 1 -- start start 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.858+0000 7f1ed2b53b80 1 Processor -- start 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 0x563f523c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f52313e30 con 0x563f523c0400 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f522b0710 con 0x563f523c0d20 2023-09-23T05:09:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f522b03b0 con 0x563f523cbbd0 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ed2b53b80 10 client.0 _ll_register_callbacks cb 0x563f5210d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 0x563f523c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40896/0 (socket says 172.21.15.5:40896) 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 -- 172.21.15.5:0/3819076525 learned_addr learned my addr 172.21.15.5:0/3819076525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 -- 172.21.15.5:0/3819076525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 msgr2=0x563f523c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 -- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 msgr2=0x563f523c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 0x563f523c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebd79c700 1 -- 172.21.15.5:0/3819076525 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb81a6210 con 0x563f523cbbd0 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.859+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 0x563f523c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0008a90 tx=0x7f1eb0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaed7593ebd1754d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000e040 con 0x563f523cbbd0 2023-09-23T05:09:54.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f523b4250 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f1eb001c040 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb0017ad0 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1ed2b53b80 10 client.27988 fetch_fsmap learned FSMap version 83 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x563f5213ede0 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.860+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f523b4250 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1eb0021400 con 0x563f523cbbd0 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 10 client.27988 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:09:54.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 20 client.27988 populate_metadata read hostname 'smithi005' 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f523db6c0 0x563f523ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 4 client.27988 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f5230ce10 con 0x563f523db6c0 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f523db6c0 0x563f523ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f523db6c0 0x563f523ddae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a5e22442add00d server_cookie=fb527e835c99b071 in_seq=0 out_seq=0 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.861+0000 7f1e97fff700 10 client.27988 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1e97fff700 1 -- 172.21.15.5:0/3819076525 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f5230ce10 con 0x563f523db6c0 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1e97fff700 10 client.27988 handle_command_reply: tid=0 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 client.27988 shutdown 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 msgr2=0x563f523c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0008a90 tx=0x7f1eb0004ad0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 shutdown_connections 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f523db6c0 0x563f523ddae0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c93f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1e97fff700 0 client.27988 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0400 0x563f523c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/3819076525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0d20 0x563f523c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.862+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 >> 172.21.15.5:0/3819076525 conn(0x563f52335260 msgr2=0x563f523bc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.863+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 shutdown_connections 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.863+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/3819076525 wait complete. 2023-09-23T05:09:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.863+0000 7f1ed2b53b80 20 client.27988 trim_cache size 0 max 16384 2023-09-23T05:09:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 Processor -- start 2023-09-23T05:09:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 -- start start 2023-09-23T05:09:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f52194920 con 0x563f523c0d20 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f523bb520 con 0x563f523c0400 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.880+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f523c47e0 con 0x563f523cbbd0 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36356/0 (socket says 172.21.15.5:36356) 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1ebd79c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 -- 172.21.15.5:0/1393506927 learned_addr learned my addr 172.21.15.5:0/1393506927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 -- 172.21.15.5:0/1393506927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 msgr2=0x563f523c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 -- 172.21.15.5:0/1393506927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 msgr2=0x563f523c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 -- 172.21.15.5:0/1393506927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f523b8480 con 0x563f523c0d20 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009990 tx=0x7f1ea4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89914a331cf9e465 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.881+0000 7f1ea37fe700 1 -- 172.21.15.5:0/1393506927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea401d030 con 0x563f523c0d20 2023-09-23T05:09:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ea37fe700 1 -- 172.21.15.5:0/1393506927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea401e040 con 0x563f523c0d20 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ea37fe700 1 -- 172.21.15.5:0/1393506927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004760 con 0x563f523c0d20 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1393506927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 msgr2=0x563f523c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009990 tx=0x7f1ea4005fd0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1393506927 shutdown_connections 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523cbbd0 0x563f523c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/1393506927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523c0d20 0x563f523c9220 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1393506927 >> 172.21.15.5:0/1393506927 conn(0x563f52335260 msgr2=0x563f523c7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1393506927 shutdown_connections 2023-09-23T05:09:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.882+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/1393506927 wait complete. 2023-09-23T05:09:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.883+0000 7f1ed2b53b80 1 -- start start 2023-09-23T05:09:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.883+0000 7f1ed2b53b80 1 Processor -- start 2023-09-23T05:09:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f523d1010 con 0x563f523cbbd0 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f523d1180 con 0x563f523c0400 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f523d1340 con 0x563f523c0d20 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ed2b53b80 10 client.0 _ll_register_callbacks cb 0x563f5210d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:40912/0 (socket says 172.21.15.5:40912) 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1eb7fff700 1 -- 172.21.15.5:0/2149185378 learned_addr learned my addr 172.21.15.5:0/2149185378 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ebcf9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.884+0000 7f1ebd79c700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 -- 172.21.15.5:0/2149185378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 msgr2=0x563f523d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 -- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 msgr2=0x563f523d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 -- 172.21.15.5:0/2149185378 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb810e5e0 con 0x563f523c0d20 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1eb7fff700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40098e0 tx=0x7f1ea400bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41a89bbc42a1c57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea401d030 con 0x563f523c0d20 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1eac0b1070 con 0x563f523c0d20 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f1ea4005490 con 0x563f523c0d20 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea4032070 con 0x563f523c0d20 2023-09-23T05:09:54.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1ed2b53b80 10 client.27994 fetch_fsmap learned FSMap version 83 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.885+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x563f52127c60 con 0x563f523c0d20 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eac0b1070 con 0x563f523c0d20 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ea400f460 con 0x563f523c0d20 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 10 client.27994 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 10 client.27994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 20 client.27994 populate_metadata read hostname 'smithi005' 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f524303a0 0x563f524327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:09:54.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 4 client.27994 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f5230cb30 con 0x563f524303a0 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.886+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f524303a0 0x563f524327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1e97fff700 10 client.27994 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ebcf9b700 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f524303a0 0x563f524327c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25417d03481a9f2 server_cookie=7c6d05a7c16d34a in_seq=0 out_seq=0 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1e97fff700 1 -- 172.21.15.5:0/2149185378 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563f5230cb30 con 0x563f524303a0 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1e97fff700 10 client.27994 handle_command_reply: tid=0 2023-09-23T05:09:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 client.27994 shutdown 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 msgr2=0x563f523d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40098e0 tx=0x7f1ea400bfa0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 shutdown_connections 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x563f524303a0 0x563f524327c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f523cbbd0 0x563f523d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f523c0400 0x563f523d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 --2- 172.21.15.5:0/2149185378 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x563f523c0d20 0x563f523d2900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1e97fff700 0 client.27994 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:09:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 >> 172.21.15.5:0/2149185378 conn(0x563f52335260 msgr2=0x563f523c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 shutdown_connections 2023-09-23T05:09:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.887+0000 7f1ed2b53b80 1 -- 172.21.15.5:0/2149185378 wait complete. 2023-09-23T05:09:54.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ed2b53b80 20 client.27994 trim_cache size 0 max 16384 2023-09-23T05:09:54.897 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:09:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:09:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:09:54.899 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:09:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 msgr2=0x7f1ea80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 0x7f1ea80a3bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000f540 tx=0x7f1eb000b040 comp rx=0 tx=0).stop 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 msgr2=0x7f1eb81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00e9a0 tx=0x7f1eac00ecb0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 shutdown_connections 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f1ea80a1710 0x7f1ea80a3bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eb819dad0 0x7f1eb819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1eb81010d0 0x7f1eb81991a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 --2- 172.21.15.5:0/407743678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81061f0 0x7f1eb81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:09:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 >> 172.21.15.5:0/407743678 conn(0x7f1eb80fc990 msgr2=0x7f1eb80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:09:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 shutdown_connections 2023-09-23T05:09:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:09:54.888+0000 7f1ebf1ff700 1 -- 172.21.15.5:0/407743678 wait complete. 2023-09-23T05:09:54.907 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:09:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:56 smithi069 ceph-mon[133931]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:56.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:56 smithi005 ceph-mon[97638]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:56.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:56 smithi118 ceph-mon[134297]: pgmap v744: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:09:58 smithi069 ceph-mon[133931]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:58.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:09:58 smithi005 ceph-mon[97638]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:09:58.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:09:58 smithi118 ceph-mon[134297]: pgmap v745: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:00 smithi069 ceph-mon[133931]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:00 smithi069 ceph-mon[133931]: overall HEALTH_OK 2023-09-23T05:10:00.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:00 smithi005 ceph-mon[97638]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:00.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:00 smithi005 ceph-mon[97638]: overall HEALTH_OK 2023-09-23T05:10:00.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:00 smithi118 ceph-mon[134297]: pgmap v746: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:00.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:00 smithi118 ceph-mon[134297]: overall HEALTH_OK 2023-09-23T05:10:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:02 smithi005 ceph-mon[97638]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:02.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:02 smithi118 ceph-mon[134297]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:02.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:02 smithi069 ceph-mon[133931]: pgmap v747: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:04 smithi069 ceph-mon[133931]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:10:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:04 smithi005 ceph-mon[97638]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:10:04.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:04 smithi118 ceph-mon[134297]: pgmap v748: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:10:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:06 smithi069 ceph-mon[133931]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:10:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:06 smithi005 ceph-mon[97638]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:10:06.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:06 smithi118 ceph-mon[134297]: pgmap v749: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:10:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:08 smithi069 ceph-mon[133931]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:10:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:08 smithi005 ceph-mon[97638]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:10:08.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:08 smithi118 ceph-mon[134297]: pgmap v750: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:10:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:10 smithi069 ceph-mon[133931]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:10:10.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:10 smithi005 ceph-mon[97638]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:10:10.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:10 smithi118 ceph-mon[134297]: pgmap v751: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:10:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:12 smithi069 ceph-mon[133931]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:10:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:12 smithi069 ceph-mon[133931]: osdmap e125: 12 total, 12 up, 12 in 2023-09-23T05:10:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:12 smithi005 ceph-mon[97638]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:10:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:12 smithi005 ceph-mon[97638]: osdmap e125: 12 total, 12 up, 12 in 2023-09-23T05:10:12.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:12 smithi118 ceph-mon[134297]: pgmap v752: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:10:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:12 smithi118 ceph-mon[134297]: osdmap e125: 12 total, 12 up, 12 in 2023-09-23T05:10:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:14 smithi069 ceph-mon[133931]: pgmap v754: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:10:14.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:14 smithi005 ceph-mon[97638]: pgmap v754: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:10:14.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:14 smithi118 ceph-mon[134297]: pgmap v754: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-23T05:10:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:16 smithi069 ceph-mon[133931]: pgmap v755: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:10:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:16 smithi005 ceph-mon[97638]: pgmap v755: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:10:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:16 smithi118 ceph-mon[134297]: pgmap v755: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:10:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:18 smithi069 ceph-mon[133931]: pgmap v756: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:18.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:18 smithi005 ceph-mon[97638]: pgmap v756: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:18 smithi118 ceph-mon[134297]: pgmap v756: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:20 smithi069 ceph-mon[133931]: pgmap v757: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:20 smithi005 ceph-mon[97638]: pgmap v757: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:20.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:20 smithi118 ceph-mon[134297]: pgmap v757: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:22 smithi005 ceph-mon[97638]: osdmap e126: 12 total, 12 up, 12 in 2023-09-23T05:10:22.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:22 smithi005 ceph-mon[97638]: pgmap v759: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2023-09-23T05:10:22.560 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:22 smithi069 ceph-mon[133931]: osdmap e126: 12 total, 12 up, 12 in 2023-09-23T05:10:22.560 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:22 smithi069 ceph-mon[133931]: pgmap v759: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2023-09-23T05:10:22.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:22 smithi118 ceph-mon[134297]: osdmap e126: 12 total, 12 up, 12 in 2023-09-23T05:10:22.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:22 smithi118 ceph-mon[134297]: pgmap v759: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 309 B/s wr, 10 op/s 2023-09-23T05:10:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:24 smithi069 ceph-mon[133931]: pgmap v760: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:24.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:24 smithi005 ceph-mon[97638]: pgmap v760: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:24.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:24 smithi118 ceph-mon[134297]: pgmap v760: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:24.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:10:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 -- 172.21.15.5:0/831115114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb810d6a0 msgr2=0x7f8cb810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 --2- 172.21.15.5:0/831115114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb810d6a0 0x7f8cb810fb10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b0a0 tx=0x7f8cb000b3b0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 -- 172.21.15.5:0/831115114 shutdown_connections 2023-09-23T05:10:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 --2- 172.21.15.5:0/831115114 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb810d6a0 0x7f8cb810fb10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 --2- 172.21.15.5:0/831115114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb8073250 0x7f8cb8073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 --2- 172.21.15.5:0/831115114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073b70 0x7f8cb810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 -- 172.21.15.5:0/831115114 >> 172.21.15.5:0/831115114 conn(0x7f8cb80fc990 msgr2=0x7f8cb80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.130+0000 7f8cbefff700 1 -- 172.21.15.5:0/831115114 shutdown_connections 2023-09-23T05:10:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.131+0000 7f8cbefff700 1 -- 172.21.15.5:0/831115114 wait complete. 2023-09-23T05:10:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.131+0000 7f8cbefff700 1 Processor -- start 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 -- start start 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073250 0x7f8cb8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb810d6a0 0x7f8cb819dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb819e270 con 0x7f8cb810d6a0 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb819e3e0 con 0x7f8cb8073250 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbefff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb819e5a0 con 0x7f8cb8073b70 2023-09-23T05:10:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45912/0 (socket says 172.21.15.5:45912) 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbd59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb810d6a0 0x7f8cb819dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cb7fff700 1 -- 172.21.15.5:0/3735604613 learned_addr learned my addr 172.21.15.5:0/3735604613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.132+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073250 0x7f8cb8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 -- 172.21.15.5:0/3735604613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073250 msgr2=0x7f8cb8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073250 0x7f8cb8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 -- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb810d6a0 msgr2=0x7f8cb819dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb810d6a0 0x7f8cb819dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 -- 172.21.15.5:0/3735604613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0009d00 con 0x7f8cb8073b70 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9a0 tx=0x7f8ca400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb5ffb700 1 -- 172.21.15.5:0/3735604613 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4009790 con 0x7f8cb8073b70 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb819e880 con 0x7f8cb8073b70 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb5ffb700 1 -- 172.21.15.5:0/3735604613 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca4010430 con 0x7f8cb8073b70 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cb5ffb700 1 -- 172.21.15.5:0/3735604613 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4018680 con 0x7f8cb8073b70 2023-09-23T05:10:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.133+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb81a60c0 con 0x7f8cb8073b70 2023-09-23T05:10:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.137+0000 7f8cb5ffb700 1 -- 172.21.15.5:0/3735604613 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f8ca40105a0 con 0x7f8cb8073b70 2023-09-23T05:10:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 Processor -- start 2023-09-23T05:10:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 -- start start 2023-09-23T05:10:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4c37460 con 0x55aef4cce400 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4c28c20 con 0x55aef4cced20 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4c26540 con 0x55aef4cd9bd0 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cb5ffb700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 0x7f8ca80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.138+0000 7f8cb5ffb700 1 -- 172.21.15.5:0/3735604613 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f8ca4014070 con 0x7f8cb8073b70 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cbcd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45920/0 (socket says 172.21.15.5:45920) 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 -- 172.21.15.5:0/1533317372 learned_addr learned my addr 172.21.15.5:0/1533317372 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 0x7f8ca80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 -- 172.21.15.5:0/1533317372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 msgr2=0x55aef4cd9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 -- 172.21.15.5:0/1533317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 msgr2=0x55aef4cce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 -- 172.21.15.5:0/1533317372 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef4c848d0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c2d0 tx=0x7f8ca408fb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ca473ac2eee3d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8ca2ffd700 1 -- 172.21.15.5:0/1533317372 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca409a3d0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8ca2ffd700 1 -- 172.21.15.5:0/1533317372 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca409e520 con 0x55aef4cd9bd0 2023-09-23T05:10:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.139+0000 7f8ca2ffd700 1 -- 172.21.15.5:0/1533317372 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca409a630 con 0x55aef4cd9bd0 2023-09-23T05:10:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/1533317372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 msgr2=0x55aef4cdbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 0x7f8ca80a3bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b780 tx=0x7f8cb0006230 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400c2d0 tx=0x7f8ca408fb40 comp rx=0 tx=0).stop 2023-09-23T05:10:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/1533317372 shutdown_connections 2023-09-23T05:10:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cdbfc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/1533317372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/1533317372 >> 172.21.15.5:0/1533317372 conn(0x55aef4c43260 msgr2=0x55aef4cca790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/1533317372 shutdown_connections 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.140+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/1533317372 wait complete. 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 -- start start 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 Processor -- start 2023-09-23T05:10:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cd69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4c21e30 con 0x55aef4cce400 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4bbe710 con 0x55aef4cced20 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4bbe3b0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.141+0000 7f8cd2a07b80 10 client.0 _ll_register_callbacks cb 0x55aef4a1b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45928/0 (socket says 172.21.15.5:45928) 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 -- 172.21.15.5:0/684676005 learned_addr learned my addr 172.21.15.5:0/684676005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cd69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 -- 172.21.15.5:0/684676005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 msgr2=0x55aef4cd6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 -- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 msgr2=0x55aef4cd69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cd69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbd59c700 1 -- 172.21.15.5:0/684676005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb81a60c0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.142+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cd69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0015000 tx=0x7f8cb00099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b3bcdf4b9b5d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0025040 con 0x55aef4cd9bd0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef4cc2250 con 0x55aef4cd9bd0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f8cb00076f0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cb000e040 con 0x55aef4cd9bd0 2023-09-23T05:10:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8cd2a07b80 10 client.28006 fetch_fsmap learned FSMap version 83 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55aef4a4cde0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aef4cc2250 con 0x55aef4cd9bd0 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.143+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8cb0020890 con 0x55aef4cd9bd0 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 10 client.28006 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 10 client.28006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 20 client.28006 populate_metadata read hostname 'smithi005' 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4ce96d0 0x55aef4cebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 4 client.28006 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:10:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef4c1ae10 con 0x55aef4ce96d0 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4ce96d0 0x55aef4cebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4ce96d0 0x55aef4cebaf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888e85a0e7965b28 server_cookie=23767f2763525df2 in_seq=0 out_seq=0 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.144+0000 7f8c97fff700 10 client.28006 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8c97fff700 1 -- 172.21.15.5:0/684676005 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aef4c1ae10 con 0x55aef4ce96d0 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8c97fff700 10 client.28006 handle_command_reply: tid=0 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 client.28006 shutdown 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 msgr2=0x55aef4cd7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0015000 tx=0x7f8cb00099e0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 shutdown_connections 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4ce96d0 0x55aef4cebaf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd7430 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cce400 0x55aef4cd69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8c97fff700 0 client.28006 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/684676005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cced20 0x55aef4cd6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 >> 172.21.15.5:0/684676005 conn(0x55aef4c43260 msgr2=0x55aef4cca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 shutdown_connections 2023-09-23T05:10:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.145+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/684676005 wait complete. 2023-09-23T05:10:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.146+0000 7f8cd2a07b80 20 client.28006 trim_cache size 0 max 16384 2023-09-23T05:10:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.163+0000 7f8cd2a07b80 1 Processor -- start 2023-09-23T05:10:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.163+0000 7f8cd2a07b80 1 -- start start 2023-09-23T05:10:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cce400 0x55aef4cd1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4cd7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4aa2920 con 0x55aef4cced20 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4cd2840 con 0x55aef4cce400 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4cec170 con 0x55aef4cd9bd0 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4cd7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45944/0 (socket says 172.21.15.5:45944) 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbcd9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cce400 0x55aef4cd1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 -- 172.21.15.5:0/2341653140 learned_addr learned my addr 172.21.15.5:0/2341653140 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 -- 172.21.15.5:0/2341653140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cce400 msgr2=0x55aef4cd1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cce400 0x55aef4cd1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 -- 172.21.15.5:0/2341653140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 msgr2=0x55aef4cd7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4cd7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 -- 172.21.15.5:0/2341653140 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef4cc6480 con 0x55aef4cd9bd0 2023-09-23T05:10:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.164+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0006200 tx=0x7f8cb00260b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8538977e7a5070 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8ca37fe700 1 -- 172.21.15.5:0/2341653140 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0008d60 con 0x55aef4cd9bd0 2023-09-23T05:10:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8ca37fe700 1 -- 172.21.15.5:0/2341653140 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb002ee80 con 0x55aef4cd9bd0 2023-09-23T05:10:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8ca37fe700 1 -- 172.21.15.5:0/2341653140 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00050d0 con 0x55aef4cd9bd0 2023-09-23T05:10:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/2341653140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 msgr2=0x55aef4cd5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0006200 tx=0x7f8cb00260b0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/2341653140 shutdown_connections 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cd9bd0 0x55aef4cd5890 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cce400 0x55aef4cd1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/2341653140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4cd7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/2341653140 >> 172.21.15.5:0/2341653140 conn(0x55aef4c43260 msgr2=0x55aef4cd5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/2341653140 shutdown_connections 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.165+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/2341653140 wait complete. 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.166+0000 7f8cd2a07b80 1 -- start start 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.166+0000 7f8cd2a07b80 1 Processor -- start 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4ce05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 0x55aef4cde520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4cdeaf0 con 0x55aef4cced20 2023-09-23T05:10:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4cdec60 con 0x55aef4cd9bd0 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef4cdee20 con 0x55aef4cce400 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cd2a07b80 10 client.0 _ll_register_callbacks cb 0x55aef4a1b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cbcd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cbcd9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:45960/0 (socket says 172.21.15.5:45960) 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4ce05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cbcd9b700 1 -- 172.21.15.5:0/524549909 learned_addr learned my addr 172.21.15.5:0/524549909 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.167+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 0x55aef4cde520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 -- 172.21.15.5:0/524549909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 msgr2=0x55aef4cde520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 0x55aef4cde520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 -- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 msgr2=0x55aef4ce05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4ce05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 -- 172.21.15.5:0/524549909 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aef4a4cde0 con 0x55aef4cce400 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4ce05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbd59c700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 0x55aef4cde520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cbcd9b700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55aef4cec3a0 tx=0x7f8cac004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bee04f37f8fe8e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8c97fff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac017040 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef4c1ab30 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f8cac012d40 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cac0210b0 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cd2a07b80 10 client.28012 fetch_fsmap learned FSMap version 83 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.168+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55aef4a35c60 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aef4c1ab30 con 0x55aef4cce400 2023-09-23T05:10:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8cac010ae0 con 0x55aef4cce400 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 10 client.28012 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 10 client.28012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 20 client.28012 populate_metadata read hostname 'smithi005' 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4d3e720 0x55aef4d40b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 4 client.28012 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aef4a6df70 con 0x55aef4d3e720 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.169+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4d3e720 0x55aef4d40b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cb7fff700 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4d3e720 0x55aef4d40b40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3874a62a04b65f server_cookie=2d446b9218d37fef in_seq=0 out_seq=0 2023-09-23T05:10:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8c97fff700 10 client.28012 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8c97fff700 1 -- 172.21.15.5:0/524549909 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aef4a6df70 con 0x55aef4d3e720 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8c97fff700 10 client.28012 handle_command_reply: tid=0 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 client.28012 shutdown 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 msgr2=0x55aef4ce0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55aef4cec3a0 tx=0x7f8cac004690 comp rx=0 tx=0).stop 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 shutdown_connections 2023-09-23T05:10:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55aef4d3e720 0x55aef4d40b40 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aef4cd9bd0 0x55aef4cde520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55aef4cce400 0x55aef4ce0060 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 --2- 172.21.15.5:0/524549909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aef4cced20 0x55aef4ce05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8c97fff700 0 client.28012 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 >> 172.21.15.5:0/524549909 conn(0x55aef4c43260 msgr2=0x55aef4cd7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 shutdown_connections 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 1 -- 172.21.15.5:0/524549909 wait complete. 2023-09-23T05:10:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.170+0000 7f8cd2a07b80 20 client.28012 trim_cache size 0 max 16384 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:10:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:10:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:10:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:10:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:10:25.181 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:10:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 msgr2=0x7f8ca80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 0x7f8ca80a3bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b780 tx=0x7f8cb0006230 comp rx=0 tx=0).stop 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 msgr2=0x7f8cb81997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9a0 tx=0x7f8ca400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 shutdown_connections 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8ca80a1710 0x7f8ca80a3bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb810d6a0 0x7f8cb819dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cb8073250 0x7f8cb8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 --2- 172.21.15.5:0/3735604613 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8cb8073b70 0x7f8cb81997d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 >> 172.21.15.5:0/3735604613 conn(0x7f8cb80fc990 msgr2=0x7f8cb8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 shutdown_connections 2023-09-23T05:10:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:25.171+0000 7f8cbefff700 1 -- 172.21.15.5:0/3735604613 wait complete. 2023-09-23T05:10:25.190 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:10:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:26 smithi069 ceph-mon[133931]: pgmap v761: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:26 smithi005 ceph-mon[97638]: pgmap v761: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:26.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:26 smithi118 ceph-mon[134297]: pgmap v761: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-23T05:10:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:28 smithi069 ceph-mon[133931]: pgmap v762: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:28 smithi005 ceph-mon[97638]: pgmap v762: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:28 smithi118 ceph-mon[134297]: pgmap v762: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:10:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:30 smithi069 ceph-mon[133931]: pgmap v763: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:10:30.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:30 smithi005 ceph-mon[97638]: pgmap v763: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:10:30.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:30 smithi118 ceph-mon[134297]: pgmap v763: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:10:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:32 smithi005 ceph-mon[97638]: pgmap v764: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:10:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:32 smithi118 ceph-mon[134297]: pgmap v764: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:10:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:32 smithi069 ceph-mon[133931]: pgmap v764: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:10:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:34 smithi069 ceph-mon[133931]: pgmap v765: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:34.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:34 smithi005 ceph-mon[97638]: pgmap v765: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:34 smithi118 ceph-mon[134297]: pgmap v765: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:36 smithi069 ceph-mon[133931]: pgmap v766: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:36 smithi005 ceph-mon[97638]: pgmap v766: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:36.660 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:36 smithi118 ceph-mon[134297]: pgmap v766: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:37.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:37 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:10:37.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:37 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:10:37.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:37 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:10:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:38 smithi069 ceph-mon[133931]: pgmap v767: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:38.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:38 smithi005 ceph-mon[97638]: pgmap v767: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:38 smithi118 ceph-mon[134297]: pgmap v767: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:40 smithi069 ceph-mon[133931]: pgmap v768: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:40.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:40 smithi005 ceph-mon[97638]: pgmap v768: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:40.766 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:40 smithi118 ceph-mon[134297]: pgmap v768: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:42 smithi005 ceph-mon[97638]: pgmap v769: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:42 smithi118 ceph-mon[134297]: pgmap v769: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:42 smithi069 ceph-mon[133931]: pgmap v769: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:10:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:10:43.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:10:43.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:10:43.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:10:43.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:10:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:10:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:10:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:10:44.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:44 smithi005 ceph-mon[97638]: pgmap v770: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:44.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:44 smithi118 ceph-mon[134297]: pgmap v770: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:44 smithi069 ceph-mon[133931]: pgmap v770: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:46 smithi069 ceph-mon[133931]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:46.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:46 smithi005 ceph-mon[97638]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:46 smithi118 ceph-mon[134297]: pgmap v771: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:48 smithi069 ceph-mon[133931]: pgmap v772: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:48 smithi005 ceph-mon[97638]: pgmap v772: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:48 smithi118 ceph-mon[134297]: pgmap v772: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:50 smithi069 ceph-mon[133931]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:50 smithi005 ceph-mon[97638]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:50 smithi118 ceph-mon[134297]: pgmap v773: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:52 smithi005 ceph-mon[97638]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:52.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:52 smithi118 ceph-mon[134297]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:52 smithi069 ceph-mon[133931]: pgmap v774: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:10:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:54 smithi069 ceph-mon[133931]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:54.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:54 smithi005 ceph-mon[97638]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:54.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:54 smithi118 ceph-mon[134297]: pgmap v775: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:10:55.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:10:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 -- 172.21.15.5:0/2500297264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb834101a80 msgr2=0x7fb834105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/2500297264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb834101a80 0x7fb834105ac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb828009990 tx=0x7fb828009ca0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 -- 172.21.15.5:0/2500297264 shutdown_connections 2023-09-23T05:10:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/2500297264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341061f0 0x7fb83410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/2500297264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341010d0 0x7fb8341014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/2500297264 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb834101a80 0x7fb834105ac0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.412+0000 7fb83a9ff700 1 -- 172.21.15.5:0/2500297264 >> 172.21.15.5:0/2500297264 conn(0x7fb8340fc990 msgr2=0x7fb8340fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.413+0000 7fb83a9ff700 1 -- 172.21.15.5:0/2500297264 shutdown_connections 2023-09-23T05:10:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.413+0000 7fb83a9ff700 1 -- 172.21.15.5:0/2500297264 wait complete. 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 Processor -- start 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 -- start start 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341061f0 0x7fb834199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 0x7fb83419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83419e5e0 con 0x7fb8341010d0 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83419e750 con 0x7fb8341061f0 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb83a9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83419e910 con 0x7fb83419db60 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb833fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.414+0000 7fb838f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 0x7fb83419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45416/0 (socket says 172.21.15.5:45416) 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb838f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 0x7fb83419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:43548/0 (socket says 172.21.15.5:43548) 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 -- 172.21.15.5:0/985357412 learned_addr learned my addr 172.21.15.5:0/985357412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb8337fe700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341061f0 0x7fb834199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 -- 172.21.15.5:0/985357412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 msgr2=0x7fb83419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 0x7fb83419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 -- 172.21.15.5:0/985357412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341061f0 msgr2=0x7fb834199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341061f0 0x7fb834199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 -- 172.21.15.5:0/985357412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8280095f0 con 0x7fb8341010d0 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb833fff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9a0 tx=0x7fb82400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb8317fa700 1 -- 172.21.15.5:0/985357412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824009790 con 0x7fb8341010d0 2023-09-23T05:10:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8341a5e70 con 0x7fb8341010d0 2023-09-23T05:10:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.415+0000 7fb8317fa700 1 -- 172.21.15.5:0/985357412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb824010430 con 0x7fb8341010d0 2023-09-23T05:10:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.416+0000 7fb8317fa700 1 -- 172.21.15.5:0/985357412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824018680 con 0x7fb8341010d0 2023-09-23T05:10:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.416+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8341a62f0 con 0x7fb8341010d0 2023-09-23T05:10:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.419+0000 7fb8317fa700 1 -- 172.21.15.5:0/985357412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb8240105a0 con 0x7fb8341010d0 2023-09-23T05:10:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.420+0000 7fb8317fa700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 0x7fb81c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.420+0000 7fb84e3a6b80 1 Processor -- start 2023-09-23T05:10:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.420+0000 7fb8317fa700 1 -- 172.21.15.5:0/985357412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb824014070 con 0x7fb8341010d0 2023-09-23T05:10:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.420+0000 7fb84e3a6b80 1 -- start start 2023-09-23T05:10:55.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.420+0000 7fb8337fe700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 0x7fb81c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a282690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a284fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a1e0460 con 0x564e5a277d20 2023-09-23T05:10:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a1d1c20 con 0x564e5a277400 2023-09-23T05:10:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a1cf540 con 0x564e5a282bd0 2023-09-23T05:10:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:45028/0 (socket says 172.21.15.5:45028) 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb838f9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a284fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 -- 172.21.15.5:0/4249183418 learned_addr learned my addr 172.21.15.5:0/4249183418 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb8337fe700 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a282690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 -- 172.21.15.5:0/4249183418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 msgr2=0x564e5a284fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a284fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb8337fe700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 0x7fb81c0a3bd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb83419a7d0 tx=0x7fb828005040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 -- 172.21.15.5:0/4249183418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 msgr2=0x564e5a282690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.421+0000 7fb833fff700 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a282690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb833fff700 1 -- 172.21.15.5:0/4249183418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e5a22d8d0 con 0x564e5a277400 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb838f9c700 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a284fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb833fff700 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb824096910 tx=0x7fb824099d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2f03842a659867 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb81a7fc700 1 -- 172.21.15.5:0/4249183418 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82409e560 con 0x564e5a277400 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb81a7fc700 1 -- 172.21.15.5:0/4249183418 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8240b1070 con 0x564e5a277400 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.422+0000 7fb81a7fc700 1 -- 172.21.15.5:0/4249183418 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82409a670 con 0x564e5a277400 2023-09-23T05:10:55.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/4249183418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 msgr2=0x564e5a2777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb824096910 tx=0x7fb824099d70 comp rx=0 tx=0).stop 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/4249183418 shutdown_connections 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a284fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a2777e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/4249183418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a282690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/4249183418 >> 172.21.15.5:0/4249183418 conn(0x564e5a1ec260 msgr2=0x564e5a273790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/4249183418 shutdown_connections 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/4249183418 wait complete. 2023-09-23T05:10:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.423+0000 7fb84e3a6b80 1 -- start start 2023-09-23T05:10:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 Processor -- start 2023-09-23T05:10:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277d20 0x564e5a27fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 0x564e5a2803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a1cae30 con 0x564e5a282bd0 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a167710 con 0x564e5a277d20 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a1673b0 con 0x564e5a277400 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb84e3a6b80 10 client.0 _ll_register_callbacks cb 0x564e59fc4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb833fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb833fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:43578/0 (socket says 172.21.15.5:43578) 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb833fff700 1 -- 172.21.15.5:0/621917108 learned_addr learned my addr 172.21.15.5:0/621917108 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb8337fe700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277d20 0x564e5a27fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.424+0000 7fb838f9c700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 0x564e5a2803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 -- 172.21.15.5:0/621917108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277d20 msgr2=0x564e5a27fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277d20 0x564e5a27fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 -- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 msgr2=0x564e5a2803c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 0x564e5a2803c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 -- 172.21.15.5:0/621917108 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c011230 con 0x564e5a277400 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb838f9c700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 0x564e5a2803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb833fff700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e970 tx=0x7fb8240cac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35284109f6eba4b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb8137fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82409e560 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e5a26b250 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb8240987b0 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb824099900 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb84e3a6b80 10 client.28024 fetch_fsmap learned FSMap version 83 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.425+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fb8341a62f0 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e5a26b250 con 0x564e5a277400 2023-09-23T05:10:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8240a2310 con 0x564e5a277400 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 10 client.28024 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 10 client.28024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 20 client.28024 populate_metadata read hostname 'smithi005' 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2922d0 0x564e5a2946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 4 client.28024 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e5a1c3e10 con 0x564e5a2922d0 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.426+0000 7fb8337fe700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2922d0 0x564e5a2946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.427+0000 7fb8337fe700 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2922d0 0x564e5a2946f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee3a99c4d9c7ada server_cookie=3f8f393a29b03ee8 in_seq=0 out_seq=0 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.427+0000 7fb8137fe700 10 client.28024 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.427+0000 7fb8137fe700 1 -- 172.21.15.5:0/621917108 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e5a1c3e10 con 0x564e5a2922d0 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.427+0000 7fb8137fe700 10 client.28024 handle_command_reply: tid=0 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.427+0000 7fb84e3a6b80 1 client.28024 shutdown 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 msgr2=0x564e5a27f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e970 tx=0x7fb8240cac10 comp rx=0 tx=0).stop 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 shutdown_connections 2023-09-23T05:10:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2922d0 0x564e5a2946f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a282bd0 0x564e5a2803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277400 0x564e5a27f940 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb8137fe700 0 client.28024 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/621917108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277d20 0x564e5a27fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 >> 172.21.15.5:0/621917108 conn(0x564e5a1ec260 msgr2=0x564e5a273320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 shutdown_connections 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/621917108 wait complete. 2023-09-23T05:10:55.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.428+0000 7fb84e3a6b80 20 client.28024 trim_cache size 0 max 16384 2023-09-23T05:10:55.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.445+0000 7fb84e3a6b80 1 Processor -- start 2023-09-23T05:10:55.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.445+0000 7fb84e3a6b80 1 -- start start 2023-09-23T05:10:55.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277400 0x564e5a2801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a282bd0 0x564e5a27a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a04b920 con 0x564e5a277400 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a28f060 con 0x564e5a282bd0 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a27b500 con 0x564e5a277d20 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb833fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277400 0x564e5a2801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:43594/0 (socket says 172.21.15.5:43594) 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 -- 172.21.15.5:0/1030160847 learned_addr learned my addr 172.21.15.5:0/1030160847 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb838f9c700 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a282bd0 0x564e5a27a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 -- 172.21.15.5:0/1030160847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a282bd0 msgr2=0x564e5a27a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a282bd0 0x564e5a27a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 -- 172.21.15.5:0/1030160847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277400 msgr2=0x564e5a2801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277400 0x564e5a2801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.446+0000 7fb8337fe700 1 -- 172.21.15.5:0/1030160847 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e59ff5de0 con 0x564e5a277d20 2023-09-23T05:10:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb8337fe700 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b660 tx=0x7fb828022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f991261b6e22f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb81affd700 1 -- 172.21.15.5:0/1030160847 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828036e10 con 0x564e5a277d20 2023-09-23T05:10:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb81affd700 1 -- 172.21.15.5:0/1030160847 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb82801e040 con 0x564e5a277d20 2023-09-23T05:10:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb81affd700 1 -- 172.21.15.5:0/1030160847 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828004c70 con 0x564e5a277d20 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/1030160847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 msgr2=0x564e5a279b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b660 tx=0x7fb828022000 comp rx=0 tx=0).stop 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/1030160847 shutdown_connections 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a282bd0 0x564e5a27a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277400 0x564e5a2801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/1030160847 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a277d20 0x564e5a279b40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/1030160847 >> 172.21.15.5:0/1030160847 conn(0x564e5a1ec260 msgr2=0x564e5a279430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.447+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/1030160847 shutdown_connections 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.448+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/1030160847 wait complete. 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.448+0000 7fb84e3a6b80 1 -- start start 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.448+0000 7fb84e3a6b80 1 Processor -- start 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a289850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a2881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a288890 con 0x564e5a277d20 2023-09-23T05:10:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a288a00 con 0x564e5a277400 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5a2e6100 con 0x564e5a282bd0 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb84e3a6b80 10 client.0 _ll_register_callbacks cb 0x564e59fc4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45452/0 (socket says 172.21.15.5:45452) 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 -- 172.21.15.5:0/855739564 learned_addr learned my addr 172.21.15.5:0/855739564 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb838f9c700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a2881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb833fff700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a289850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 -- 172.21.15.5:0/855739564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 msgr2=0x564e5a2881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a2881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 -- 172.21.15.5:0/855739564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 msgr2=0x564e5a289850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a289850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.449+0000 7fb8337fe700 1 -- 172.21.15.5:0/855739564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83410e5e0 con 0x564e5a277d20 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8337fe700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb828009350 tx=0x7fb82800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb65ac7652a7dee1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8137fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828036570 con 0x564e5a277d20 2023-09-23T05:10:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8240b1070 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb82802c040 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb828004500 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb84e3a6b80 10 client.17946 fetch_fsmap learned FSMap version 83 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x564e59fdec60 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.450+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8240b1070 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb82800f760 con 0x564e5a277d20 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 10 client.17946 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:10:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 20 client.17946 populate_metadata read hostname 'smithi005' 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2e7430 0x564e5a2e9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 4 client.17946 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e5a1c3b30 con 0x564e5a2e7430 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb833fff700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2e7430 0x564e5a2e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb8137fe700 10 client.17946 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:55.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.451+0000 7fb833fff700 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2e7430 0x564e5a2e9850 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c154d4704b44bb server_cookie=846a9cf0552280be in_seq=0 out_seq=0 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb8137fe700 1 -- 172.21.15.5:0/855739564 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e5a1c3b30 con 0x564e5a2e7430 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb8137fe700 10 client.17946 handle_command_reply: tid=0 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 client.17946 shutdown 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 msgr2=0x564e5a287b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb828009350 tx=0x7fb82800bf60 comp rx=0 tx=0).stop 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 shutdown_connections 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x564e5a2e7430 0x564e5a2e9850 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x564e5a282bd0 0x564e5a2881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564e5a277400 0x564e5a289850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 --2- 172.21.15.5:0/855739564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5a277d20 0x564e5a287b80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb8137fe700 0 client.17946 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 >> 172.21.15.5:0/855739564 conn(0x564e5a1ec260 msgr2=0x564e5a279cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 shutdown_connections 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 1 -- 172.21.15.5:0/855739564 wait complete. 2023-09-23T05:10:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.452+0000 7fb84e3a6b80 20 client.17946 trim_cache size 0 max 16384 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:10:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:10:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:10:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:10:55.462 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:10:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 msgr2=0x7fb81c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 0x7fb81c0a3bd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb83419a7d0 tx=0x7fb828005040 comp rx=0 tx=0).stop 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 msgr2=0x7fb8341991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9a0 tx=0x7fb82400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 shutdown_connections 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb81c0a1710 0x7fb81c0a3bd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb83419db60 0x7fb83419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8341010d0 0x7fb8341991c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 --2- 172.21.15.5:0/985357412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb8341061f0 0x7fb834199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:10:55.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 >> 172.21.15.5:0/985357412 conn(0x7fb8340fc990 msgr2=0x7fb8340fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:10:55.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 shutdown_connections 2023-09-23T05:10:55.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:10:55.453+0000 7fb83a9ff700 1 -- 172.21.15.5:0/985357412 wait complete. 2023-09-23T05:10:55.471 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:10:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:56 smithi069 ceph-mon[133931]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:56 smithi005 ceph-mon[97638]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:56.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:56 smithi118 ceph-mon[134297]: pgmap v776: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:10:58 smithi069 ceph-mon[133931]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:58.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:10:58 smithi005 ceph-mon[97638]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:10:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:10:58 smithi118 ceph-mon[134297]: pgmap v777: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-23T05:11:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:00 smithi069 ceph-mon[133931]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:00.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:00 smithi005 ceph-mon[97638]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:00.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:00 smithi118 ceph-mon[134297]: pgmap v778: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:02 smithi005 ceph-mon[97638]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:02.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:02 smithi118 ceph-mon[134297]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:02.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:02 smithi069 ceph-mon[133931]: pgmap v779: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:04 smithi069 ceph-mon[133931]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:11:04.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:04 smithi005 ceph-mon[97638]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:11:04.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:04 smithi118 ceph-mon[134297]: pgmap v780: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:11:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:06 smithi069 ceph-mon[133931]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:11:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:06 smithi005 ceph-mon[97638]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:11:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:06 smithi118 ceph-mon[134297]: pgmap v781: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-23T05:11:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:08 smithi069 ceph-mon[133931]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:11:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:08 smithi005 ceph-mon[97638]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:11:08.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:08 smithi118 ceph-mon[134297]: pgmap v782: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-23T05:11:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:10 smithi069 ceph-mon[133931]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:11:10.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:10 smithi005 ceph-mon[97638]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:11:10.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:10 smithi118 ceph-mon[134297]: pgmap v783: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:11:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:12 smithi069 ceph-mon[133931]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:11:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:12 smithi069 ceph-mon[133931]: osdmap e127: 12 total, 12 up, 12 in 2023-09-23T05:11:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:12 smithi005 ceph-mon[97638]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:11:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:12 smithi005 ceph-mon[97638]: osdmap e127: 12 total, 12 up, 12 in 2023-09-23T05:11:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:12 smithi118 ceph-mon[134297]: pgmap v784: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:11:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:12 smithi118 ceph-mon[134297]: osdmap e127: 12 total, 12 up, 12 in 2023-09-23T05:11:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:14 smithi069 ceph-mon[133931]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:11:14.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:14 smithi005 ceph-mon[97638]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:11:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:14 smithi118 ceph-mon[134297]: pgmap v786: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:11:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:16 smithi069 ceph-mon[133931]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:11:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:16 smithi005 ceph-mon[97638]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:11:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:16 smithi118 ceph-mon[134297]: pgmap v787: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-23T05:11:18.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:18 smithi069 ceph-mon[133931]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:18.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:18 smithi005 ceph-mon[97638]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:18 smithi118 ceph-mon[134297]: pgmap v788: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:20 smithi069 ceph-mon[133931]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:20 smithi005 ceph-mon[97638]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:20.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:20 smithi118 ceph-mon[134297]: pgmap v789: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:22 smithi005 ceph-mon[97638]: osdmap e128: 12 total, 12 up, 12 in 2023-09-23T05:11:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:22 smithi005 ceph-mon[97638]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:11:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:22 smithi118 ceph-mon[134297]: osdmap e128: 12 total, 12 up, 12 in 2023-09-23T05:11:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:22 smithi118 ceph-mon[134297]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:11:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:22 smithi069 ceph-mon[133931]: osdmap e128: 12 total, 12 up, 12 in 2023-09-23T05:11:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:22 smithi069 ceph-mon[133931]: pgmap v791: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:11:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:24 smithi069 ceph-mon[133931]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:24.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:24 smithi005 ceph-mon[97638]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:24.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:24 smithi118 ceph-mon[134297]: pgmap v792: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:11:25.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:11:25.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 -- 172.21.15.5:0/593230505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d81010d0 msgr2=0x7fc2d81014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 --2- 172.21.15.5:0/593230505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d81010d0 0x7fc2d81014b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009990 tx=0x7fc2c8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 -- 172.21.15.5:0/593230505 shutdown_connections 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 --2- 172.21.15.5:0/593230505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81061f0 0x7fc2d810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 --2- 172.21.15.5:0/593230505 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d81010d0 0x7fc2d81014b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 --2- 172.21.15.5:0/593230505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d8101a80 0x7fc2d8105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 -- 172.21.15.5:0/593230505 >> 172.21.15.5:0/593230505 conn(0x7fc2d80fc990 msgr2=0x7fc2d80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 -- 172.21.15.5:0/593230505 shutdown_connections 2023-09-23T05:11:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.693+0000 7fc2dddff700 1 -- 172.21.15.5:0/593230505 wait complete. 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.694+0000 7fc2dddff700 1 Processor -- start 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.694+0000 7fc2dddff700 1 -- start start 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81010d0 0x7fc2d8196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d819c3c0 con 0x7fc2d81061f0 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d819c530 con 0x7fc2d81010d0 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2dddff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d819c6f0 con 0x7fc2d819b940 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46010/0 (socket says 172.21.15.5:46010) 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36884/0 (socket says 172.21.15.5:36884) 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2241066875 learned_addr learned my addr 172.21.15.5:0/2241066875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.695+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81010d0 0x7fc2d8196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2241066875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81010d0 msgr2=0x7fc2d8196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81010d0 0x7fc2d8196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 msgr2=0x7fc2d81974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2241066875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80095f0 con 0x7fc2d819b940 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ef30 tx=0x7fc2d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d4ff9700 1 -- 172.21.15.5:0/2241066875 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0018070 con 0x7fc2d819b940 2023-09-23T05:11:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d4ff9700 1 -- 172.21.15.5:0/2241066875 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d000f040 con 0x7fc2d819b940 2023-09-23T05:11:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2d4ff9700 1 -- 172.21.15.5:0/2241066875 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000cd00 con 0x7fc2d819b940 2023-09-23T05:11:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d81a3bf0 con 0x7fc2d819b940 2023-09-23T05:11:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.696+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d81a4080 con 0x7fc2d819b940 2023-09-23T05:11:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.699+0000 7fc2d4ff9700 1 -- 172.21.15.5:0/2241066875 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fc2d0003a00 con 0x7fc2d819b940 2023-09-23T05:11:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 Processor -- start 2023-09-23T05:11:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 -- start start 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae6fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2d4ff9700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 0x7fc2c00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae657460 con 0x5556ae6eed20 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae648c20 con 0x5556ae6fdfe0 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae646540 con 0x5556ae6ee400 2023-09-23T05:11:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2d4ff9700 1 -- 172.21.15.5:0/2241066875 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc2d00cd200 con 0x7fc2d819b940 2023-09-23T05:11:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.701+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 0x7fc2c00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:34352/0 (socket says 172.21.15.5:34352) 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46012/0 (socket says 172.21.15.5:46012) 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 -- 172.21.15.5:0/4022164243 learned_addr learned my addr 172.21.15.5:0/4022164243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae6fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 -- 172.21.15.5:0/4022164243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 msgr2=0x5556ae6ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 -- 172.21.15.5:0/4022164243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 msgr2=0x5556ae6fdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae6fdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 -- 172.21.15.5:0/4022164243 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ae6a48d0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 0x7fc2c00a3c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b2e0 tx=0x7fc2c8005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.702+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00a1b40 tx=0x7fc2d0099c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da36be6f17498b0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2bdffb700 1 -- 172.21.15.5:0/4022164243 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000ce80 con 0x5556ae6fdfe0 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2bdffb700 1 -- 172.21.15.5:0/4022164243 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d0004ad0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2bdffb700 1 -- 172.21.15.5:0/4022164243 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0053f80 con 0x5556ae6fdfe0 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 -- 172.21.15.5:0/4022164243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 msgr2=0x5556ae7003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00a1b40 tx=0x7fc2d0099c70 comp rx=0 tx=0).stop 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 -- 172.21.15.5:0/4022164243 shutdown_connections 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6fdfe0 0x5556ae7003d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/4022164243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae6fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 -- 172.21.15.5:0/4022164243 >> 172.21.15.5:0/4022164243 conn(0x5556ae663260 msgr2=0x5556ae6ea790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 -- 172.21.15.5:0/4022164243 shutdown_connections 2023-09-23T05:11:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.703+0000 7fc2f1754b80 1 -- 172.21.15.5:0/4022164243 wait complete. 2023-09-23T05:11:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.704+0000 7fc2f1754b80 1 -- start start 2023-09-23T05:11:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.704+0000 7fc2f1754b80 1 Processor -- start 2023-09-23T05:11:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6eed20 0x5556ae6fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae641e30 con 0x5556ae6fdfe0 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae5de710 con 0x5556ae6eed20 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae5de3b0 con 0x5556ae6ee400 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2f1754b80 10 client.0 _ll_register_callbacks cb 0x5556ae43b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2d77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2d77fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46016/0 (socket says 172.21.15.5:46016) 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.705+0000 7fc2d77fe700 1 -- 172.21.15.5:0/423706475 learned_addr learned my addr 172.21.15.5:0/423706475 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6eed20 0x5556ae6fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 -- 172.21.15.5:0/423706475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 msgr2=0x5556ae6fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 -- 172.21.15.5:0/423706475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6eed20 msgr2=0x5556ae6fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6eed20 0x5556ae6fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 -- 172.21.15.5:0/423706475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc019040 con 0x5556ae6fdfe0 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6fb800 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00a1b40 tx=0x7fc2d009a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d76a9377ce512e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000ce60 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556ae6e2250 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.706+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc2d001b320 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d009a470 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 10 client.17958 fetch_fsmap learned FSMap version 83 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fc2d81a4080 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556ae6e2250 con 0x5556ae6fdfe0 2023-09-23T05:11:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2d00a1be0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 10 client.17958 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 20 client.17958 populate_metadata read hostname 'smithi005' 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae70d560 0x5556ae70f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 4 client.17958 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556ae63ae10 con 0x5556ae70d560 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.707+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae70d560 0x5556ae70f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.708+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae70d560 0x5556ae70f980 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400f4edb59c8249c server_cookie=45f11d5899b8136c in_seq=0 out_seq=0 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.708+0000 7fc2b6ffd700 10 client.17958 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/423706475 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556ae63ae10 con 0x5556ae70d560 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2b6ffd700 10 client.17958 handle_command_reply: tid=0 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 client.17958 shutdown 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 msgr2=0x5556ae6fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6fb800 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00a1b40 tx=0x7fc2d009a200 comp rx=0 tx=0).stop 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 shutdown_connections 2023-09-23T05:11:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae70d560 0x5556ae70f980 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6fb800 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6ee400 0x5556ae6fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2b6ffd700 0 client.17958 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/423706475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6eed20 0x5556ae6fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 >> 172.21.15.5:0/423706475 conn(0x5556ae663260 msgr2=0x5556ae6ea320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 shutdown_connections 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 1 -- 172.21.15.5:0/423706475 wait complete. 2023-09-23T05:11:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.709+0000 7fc2f1754b80 20 client.17958 trim_cache size 0 max 16384 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 Processor -- start 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 -- start start 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae6fa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6f8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae4c2920 con 0x5556ae6fdfe0 2023-09-23T05:11:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae6ee270 con 0x5556ae6ee400 2023-09-23T05:11:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae705090 con 0x5556ae6eed20 2023-09-23T05:11:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2d6ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46022/0 (socket says 172.21.15.5:46022) 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2d6ffd700 1 -- 172.21.15.5:0/1579748258 learned_addr learned my addr 172.21.15.5:0/1579748258 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.727+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae6fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 -- 172.21.15.5:0/1579748258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 msgr2=0x5556ae6fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae6fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 -- 172.21.15.5:0/1579748258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 msgr2=0x5556ae6f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 -- 172.21.15.5:0/1579748258 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ae6e6480 con 0x5556ae6eed20 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8068e70 tx=0x7fc2cc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594bf85b300dff25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2be7fc700 1 -- 172.21.15.5:0/1579748258 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc023070 con 0x5556ae6eed20 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2be7fc700 1 -- 172.21.15.5:0/1579748258 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2cc004510 con 0x5556ae6eed20 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.728+0000 7fc2be7fc700 1 -- 172.21.15.5:0/1579748258 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016030 con 0x5556ae6eed20 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- 172.21.15.5:0/1579748258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 msgr2=0x5556ae704b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8068e70 tx=0x7fc2cc0047c0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- 172.21.15.5:0/1579748258 shutdown_connections 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6fdfe0 0x5556ae6f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae6fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/1579748258 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6eed20 0x5556ae704b20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- 172.21.15.5:0/1579748258 >> 172.21.15.5:0/1579748258 conn(0x5556ae663260 msgr2=0x5556ae6f60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- 172.21.15.5:0/1579748258 shutdown_connections 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- 172.21.15.5:0/1579748258 wait complete. 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 -- start start 2023-09-23T05:11:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.729+0000 7fc2f1754b80 1 Processor -- start 2023-09-23T05:11:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae704350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae704890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae703110 con 0x5556ae6eed20 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae703280 con 0x5556ae6ee400 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae703440 con 0x5556ae6fdfe0 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2f1754b80 10 client.0 _ll_register_callbacks cb 0x7fc2cc0116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:46036/0 (socket says 172.21.15.5:46036) 2023-09-23T05:11:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2494246280 learned_addr learned my addr 172.21.15.5:0/2494246280 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2d6ffd700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae704890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.730+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae704350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2494246280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 msgr2=0x5556ae704350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae704350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 msgr2=0x5556ae704890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae704890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 -- 172.21.15.5:0/2494246280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc0110f0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2d7fff700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0004e00 tx=0x7fc2d00b24c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e481a8ebe6ef1319 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001b320 con 0x5556ae6fdfe0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2d0004ad0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fc2d009a470 con 0x5556ae6fdfe0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d000ce60 con 0x5556ae6fdfe0 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2f1754b80 10 client.28048 fetch_fsmap learned FSMap version 83 2023-09-23T05:11:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5556ae46cde0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.731+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d0004ad0 con 0x5556ae6fdfe0 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2d0095c80 con 0x5556ae6fdfe0 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 10 client.28048 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 10 client.28048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 20 client.28048 populate_metadata read hostname 'smithi005' 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae7620d0 0x5556ae7644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 4 client.28048 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556ae63ab30 con 0x5556ae7620d0 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.732+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae7620d0 0x5556ae7644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2b6ffd700 10 client.28048 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2d77fe700 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae7620d0 0x5556ae7644c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8529616eab0e817 server_cookie=ee025b9901918b80 in_seq=0 out_seq=0 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2b6ffd700 1 -- 172.21.15.5:0/2494246280 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5556ae63ab30 con 0x5556ae7620d0 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2b6ffd700 10 client.28048 handle_command_reply: tid=0 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 client.28048 shutdown 2023-09-23T05:11:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 msgr2=0x5556ae702b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0004e00 tx=0x7fc2d00b24c0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 shutdown_connections 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5556ae7620d0 0x5556ae7644c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5556ae6fdfe0 0x5556ae702b10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ae6ee400 0x5556ae704350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 --2- 172.21.15.5:0/2494246280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556ae6eed20 0x5556ae704890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2b6ffd700 0 client.28048 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 >> 172.21.15.5:0/2494246280 conn(0x5556ae663260 msgr2=0x5556ae6fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 shutdown_connections 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 1 -- 172.21.15.5:0/2494246280 wait complete. 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.733+0000 7fc2f1754b80 20 client.28048 trim_cache size 0 max 16384 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:11:25.743 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:11:25.744 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:11:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 msgr2=0x7fc2c00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 0x7fc2c00a3c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b2e0 tx=0x7fc2c8005fd0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 msgr2=0x7fc2d819bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000ef30 tx=0x7fc2d000c3a0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 shutdown_connections 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fc2c00a1790 0x7fc2c00a3c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fc2d819b940 0x7fc2d819bdc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2d81010d0 0x7fc2d8196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 --2- 172.21.15.5:0/2241066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d81061f0 0x7fc2d81974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 >> 172.21.15.5:0/2241066875 conn(0x7fc2d80fc990 msgr2=0x7fc2d8104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 shutdown_connections 2023-09-23T05:11:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:25.734+0000 7fc2dddff700 1 -- 172.21.15.5:0/2241066875 wait complete. 2023-09-23T05:11:25.753 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:11:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:26 smithi069 ceph-mon[133931]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-23T05:11:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:26 smithi005 ceph-mon[97638]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-23T05:11:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:26 smithi118 ceph-mon[134297]: pgmap v793: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-23T05:11:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:28 smithi069 ceph-mon[133931]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:28 smithi005 ceph-mon[97638]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:28 smithi118 ceph-mon[134297]: pgmap v794: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:30 smithi069 ceph-mon[133931]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:30.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:30 smithi005 ceph-mon[97638]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:30.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:30 smithi118 ceph-mon[134297]: pgmap v795: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:11:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:32 smithi005 ceph-mon[97638]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:11:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:32 smithi118 ceph-mon[134297]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:11:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:32 smithi069 ceph-mon[133931]: pgmap v796: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-23T05:11:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:34 smithi069 ceph-mon[133931]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:34.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:34 smithi005 ceph-mon[97638]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:34 smithi118 ceph-mon[134297]: pgmap v797: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:36 smithi069 ceph-mon[133931]: pgmap v798: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:36 smithi005 ceph-mon[97638]: pgmap v798: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:36 smithi118 ceph-mon[134297]: pgmap v798: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:38 smithi069 ceph-mon[133931]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:38.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:38 smithi005 ceph-mon[97638]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:38 smithi118 ceph-mon[134297]: pgmap v799: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:40.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:40 smithi069 ceph-mon[133931]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:40.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:40 smithi005 ceph-mon[97638]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:40 smithi118 ceph-mon[134297]: pgmap v800: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:42 smithi005 ceph-mon[97638]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:42 smithi118 ceph-mon[134297]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:42 smithi069 ceph-mon[133931]: pgmap v801: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:43.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:43 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:11:43.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:43 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:11:43.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:43 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:11:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:44 smithi069 ceph-mon[133931]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:11:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:11:44.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:11:44.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:44 smithi005 ceph-mon[97638]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:11:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:11:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:11:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:44 smithi118 ceph-mon[134297]: pgmap v802: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:11:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:11:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:11:46.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:46 smithi069 ceph-mon[133931]: pgmap v803: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:46.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:46 smithi005 ceph-mon[97638]: pgmap v803: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:46 smithi118 ceph-mon[134297]: pgmap v803: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:48 smithi069 ceph-mon[133931]: pgmap v804: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:48.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:48 smithi005 ceph-mon[97638]: pgmap v804: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:48 smithi118 ceph-mon[134297]: pgmap v804: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:50.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:50 smithi069 ceph-mon[133931]: pgmap v805: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:50 smithi005 ceph-mon[97638]: pgmap v805: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:50 smithi118 ceph-mon[134297]: pgmap v805: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:52 smithi005 ceph-mon[97638]: pgmap v806: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:52.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:52 smithi118 ceph-mon[134297]: pgmap v806: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:52 smithi069 ceph-mon[133931]: pgmap v806: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:54 smithi069 ceph-mon[133931]: pgmap v807: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:54.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:54 smithi005 ceph-mon[97638]: pgmap v807: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:54.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:54 smithi118 ceph-mon[134297]: pgmap v807: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:11:55.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:11:55.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 -- 172.21.15.5:0/4192533453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41061f0 msgr2=0x7f3fa410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/4192533453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41061f0 0x7f3fa410dd70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00b0a0 tx=0x7f3f9c00b3b0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 -- 172.21.15.5:0/4192533453 shutdown_connections 2023-09-23T05:11:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/4192533453 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41061f0 0x7f3fa410dd70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/4192533453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa41010d0 0x7f3fa41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/4192533453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa4101a80 0x7f3fa4105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 -- 172.21.15.5:0/4192533453 >> 172.21.15.5:0/4192533453 conn(0x7f3fa40fc990 msgr2=0x7f3fa40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:55.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 -- 172.21.15.5:0/4192533453 shutdown_connections 2023-09-23T05:11:55.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.991+0000 7f3faa5ff700 1 -- 172.21.15.5:0/4192533453 wait complete. 2023-09-23T05:11:55.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.992+0000 7f3faa5ff700 1 Processor -- start 2023-09-23T05:11:55.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.992+0000 7f3faa5ff700 1 -- start start 2023-09-23T05:11:55.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:55.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 0x7f3fa41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa419dc40 0x7f3fa419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419e6c0 con 0x7f3fa419dc40 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419e830 con 0x7f3fa41061f0 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3faa5ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa419e9f0 con 0x7f3fa41010d0 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56242/0 (socket says 172.21.15.5:56242) 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 -- 172.21.15.5:0/645757419 learned_addr learned my addr 172.21.15.5:0/645757419 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:55.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa419dc40 0x7f3fa419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 0x7f3fa41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 -- 172.21.15.5:0/645757419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 msgr2=0x7f3fa41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 0x7f3fa41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 -- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa419dc40 msgr2=0x7f3fa419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa419dc40 0x7f3fa419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa3fff700 1 -- 172.21.15.5:0/645757419 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c009d00 con 0x7f3fa41010d0 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.993+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 0x7f3fa41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:11:55.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e9a0 tx=0x7f3f9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3fa17fa700 1 -- 172.21.15.5:0/645757419 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94009790 con 0x7f3fa41010d0 2023-09-23T05:11:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3fa17fa700 1 -- 172.21.15.5:0/645757419 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9400cbd0 con 0x7f3fa41010d0 2023-09-23T05:11:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3fa17fa700 1 -- 172.21.15.5:0/645757419 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94018680 con 0x7f3fa41010d0 2023-09-23T05:11:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa41a5f50 con 0x7f3fa41010d0 2023-09-23T05:11:55.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.994+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa41a6380 con 0x7f3fa41010d0 2023-09-23T05:11:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:55.999+0000 7f3fa17fa700 1 -- 172.21.15.5:0/645757419 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f3f9400cd40 con 0x7f3fa41010d0 2023-09-23T05:11:55.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 Processor -- start 2023-09-23T05:11:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 -- start start 2023-09-23T05:11:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eae67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eaf1bd0 0x56019eaf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ea4f460 con 0x56019eaf1bd0 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ea40c20 con 0x56019eae6400 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.000+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ea3e540 con 0x56019eae6d20 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa17fa700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 0x7f3f8c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56244/0 (socket says 172.21.15.5:56244) 2023-09-23T05:11:56.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eaf1bd0 0x56019eaf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4037100966 learned_addr learned my addr 172.21.15.5:0/4037100966 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eae67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa17fa700 1 -- 172.21.15.5:0/645757419 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3f94014070 con 0x7f3fa41010d0 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 0x7f3f8c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4037100966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 msgr2=0x56019eae67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eae67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4037100966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eaf1bd0 msgr2=0x56019eaf3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eaf1bd0 0x56019eaf3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4037100966 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019ea9c8d0 con 0x56019eae6d20 2023-09-23T05:11:56.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.001+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f94096e60 tx=0x7f3f9409a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944f04d04e507971 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:56.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3f8a7fc700 1 -- 172.21.15.5:0/4037100966 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9409aaf0 con 0x56019eae6d20 2023-09-23T05:11:56.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3f8a7fc700 1 -- 172.21.15.5:0/4037100966 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9409e540 con 0x56019eae6d20 2023-09-23T05:11:56.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3f8a7fc700 1 -- 172.21.15.5:0/4037100966 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9409a8b0 con 0x56019eae6d20 2023-09-23T05:11:56.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 0x7f3f8c0a3c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0062a0 tx=0x7f3f9c006210 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4037100966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 msgr2=0x56019eaf1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3f94096e60 tx=0x7f3f9409a2f0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4037100966 shutdown_connections 2023-09-23T05:11:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eaf1bd0 0x56019eaf3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eae67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4037100966 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eaf1690 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4037100966 >> 172.21.15.5:0/4037100966 conn(0x56019ea5b260 msgr2=0x56019eae2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:56.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.002+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4037100966 shutdown_connections 2023-09-23T05:11:56.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.003+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4037100966 wait complete. 2023-09-23T05:11:56.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.003+0000 7f3fbdfb6b80 1 -- start start 2023-09-23T05:11:56.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.003+0000 7f3fbdfb6b80 1 Processor -- start 2023-09-23T05:11:56.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eaf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6d20 0x56019eaee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ea39e30 con 0x56019eae6d20 2023-09-23T05:11:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e9d6710 con 0x56019eae6400 2023-09-23T05:11:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e9d63b0 con 0x56019eaf1bd0 2023-09-23T05:11:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.004+0000 7f3fbdfb6b80 10 client.0 _ll_register_callbacks cb 0x56019e833800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:11:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa8b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa8b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56252/0 (socket says 172.21.15.5:56252) 2023-09-23T05:11:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eaf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa8b9c700 1 -- 172.21.15.5:0/4186261875 learned_addr learned my addr 172.21.15.5:0/4186261875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6d20 0x56019eaee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4186261875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 msgr2=0x56019eaeee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6d20 msgr2=0x56019eaee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6d20 0x56019eaee830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 -- 172.21.15.5:0/4186261875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa41a6380 con 0x56019eae6400 2023-09-23T05:11:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eaf15b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3f94096e60 tx=0x7f3f9409e2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554987de47c97fe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3f837fe700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:11:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940b1070 con 0x56019eae6400 2023-09-23T05:11:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56019eada250 con 0x56019eae6400 2023-09-23T05:11:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.005+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3f9409a3f0 con 0x56019eae6400 2023-09-23T05:11:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f940a2780 con 0x56019eae6400 2023-09-23T05:11:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 10 client.28016 fetch_fsmap learned FSMap version 83 2023-09-23T05:11:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56019e864de0 con 0x56019eae6400 2023-09-23T05:11:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56019eada250 con 0x56019eae6400 2023-09-23T05:11:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f940a22f0 con 0x56019eae6400 2023-09-23T05:11:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 10 client.28016 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:11:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 10 client.28016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:11:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 20 client.28016 populate_metadata read hostname 'smithi005' 2023-09-23T05:11:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb01200 0x56019eb03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 4 client.28016 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:11:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.006+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56019ea32e10 con 0x56019eb01200 2023-09-23T05:11:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.007+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb01200 0x56019eb03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.007+0000 7f3f837fe700 10 client.28016 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.007+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb01200 0x56019eb03620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb6d676b09aad9f server_cookie=2ed506c90c0d8470 in_seq=0 out_seq=0 2023-09-23T05:11:56.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3f837fe700 1 -- 172.21.15.5:0/4186261875 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56019ea32e10 con 0x56019eb01200 2023-09-23T05:11:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3f837fe700 10 client.28016 handle_command_reply: tid=0 2023-09-23T05:11:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 client.28016 shutdown 2023-09-23T05:11:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 msgr2=0x56019eaf15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eaf15b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3f94096e60 tx=0x7f3f9409e2e0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 shutdown_connections 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb01200 0x56019eb03620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eaf1bd0 0x56019eaeee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eae6400 0x56019eaf15b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/4186261875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6d20 0x56019eaee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3f837fe700 0 client.28016 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 >> 172.21.15.5:0/4186261875 conn(0x56019ea5b260 msgr2=0x56019eae2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 shutdown_connections 2023-09-23T05:11:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/4186261875 wait complete. 2023-09-23T05:11:56.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.008+0000 7f3fbdfb6b80 20 client.28016 trim_cache size 0 max 16384 2023-09-23T05:11:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 Processor -- start 2023-09-23T05:11:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 -- start start 2023-09-23T05:11:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eae25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e8ba920 con 0x56019eae6400 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eaf8c00 con 0x56019eaf1bd0 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.026+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eaea700 con 0x56019eae6d20 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56260/0 (socket says 172.21.15.5:56260) 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34624/0 (socket says 172.21.15.5:34624) 2023-09-23T05:11:56.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa37fe700 1 -- 172.21.15.5:0/3833032928 learned_addr learned my addr 172.21.15.5:0/3833032928 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eae25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 -- 172.21.15.5:0/3833032928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 msgr2=0x56019eae20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 -- 172.21.15.5:0/3833032928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 msgr2=0x56019eae25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eae25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 -- 172.21.15.5:0/3833032928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019e84dc60 con 0x56019eae6400 2023-09-23T05:11:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400c5b0 tx=0x7f3f940a2900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf16fb4f9f41b864 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.027+0000 7f3f8affd700 1 -- 172.21.15.5:0/3833032928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940b1070 con 0x56019eae6400 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3f8affd700 1 -- 172.21.15.5:0/3833032928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f940a2780 con 0x56019eae6400 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3f8affd700 1 -- 172.21.15.5:0/3833032928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9409a860 con 0x56019eae6400 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/3833032928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 msgr2=0x56019eaf86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400c5b0 tx=0x7f3f940a2900 comp rx=0 tx=0).stop 2023-09-23T05:11:56.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/3833032928 shutdown_connections 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eae25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eaf86c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/3833032928 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eae20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/3833032928 >> 172.21.15.5:0/3833032928 conn(0x56019ea5b260 msgr2=0x56019eae19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/3833032928 shutdown_connections 2023-09-23T05:11:56.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.028+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/3833032928 wait complete. 2023-09-23T05:11:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.029+0000 7f3fbdfb6b80 1 -- start start 2023-09-23T05:11:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.029+0000 7f3fbdfb6b80 1 Processor -- start 2023-09-23T05:11:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eb54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eb49de0 con 0x56019eae6400 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eb54560 con 0x56019eaf1bd0 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eb546d0 con 0x56019eae6d20 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fbdfb6b80 10 client.0 _ll_register_callbacks cb 0x7f3f98009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:11:56.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34634/0 (socket says 172.21.15.5:34634) 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa3fff700 1 -- 172.21.15.5:0/2746192709 learned_addr learned my addr 172.21.15.5:0/2746192709 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:56276/0 (socket says 172.21.15.5:56276) 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa8b9c700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eb54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 -- 172.21.15.5:0/2746192709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 msgr2=0x56019eb54020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eb54020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 -- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 msgr2=0x56019eb49190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa37fe700 1 -- 172.21.15.5:0/2746192709 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f980044d0 con 0x56019eae6d20 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.030+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3fa37fe700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x56019eae3cc0 tx=0x7f3f98009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a28643bfb36bce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3f837fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98016070 con 0x56019eae6d20 2023-09-23T05:11:56.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56019ea32b30 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f3f9801f510 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f98003680 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3fbdfb6b80 10 client.28066 fetch_fsmap learned FSMap version 83 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x56019e864de0 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.031+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56019ea32b30 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3f98011770 con 0x56019eae6d20 2023-09-23T05:11:56.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 10 client.28066 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 10 client.28066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 20 client.28066 populate_metadata read hostname 'smithi005' 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb55f80 0x56019eb583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 4 client.28066 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56019e885f70 con 0x56019eb55f80 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb55f80 0x56019eb583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3f837fe700 10 client.28066 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:56.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.032+0000 7f3fa3fff700 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb55f80 0x56019eb583a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4d6d06232d4c0b server_cookie=d0eb72185f74eb5f in_seq=0 out_seq=0 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3f837fe700 1 -- 172.21.15.5:0/2746192709 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56019e885f70 con 0x56019eb55f80 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3f837fe700 10 client.28066 handle_command_reply: tid=0 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 client.28066 shutdown 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 msgr2=0x56019eb496d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x56019eae3cc0 tx=0x7f3f98009bd0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 shutdown_connections 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x56019eb55f80 0x56019eb583a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56019eaf1bd0 0x56019eb54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56019eae6400 0x56019eb49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 --2- 172.21.15.5:0/2746192709 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x56019eae6d20 0x56019eb496d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3f837fe700 0 client.28066 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 >> 172.21.15.5:0/2746192709 conn(0x56019ea5b260 msgr2=0x56019eae2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 shutdown_connections 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 1 -- 172.21.15.5:0/2746192709 wait complete. 2023-09-23T05:11:56.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.033+0000 7f3fbdfb6b80 20 client.28066 trim_cache size 0 max 16384 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:11:56.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:11:56.045 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:11:56.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:11:56.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:11:56.045 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:11:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 msgr2=0x7f3f8c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 0x7f3f8c0a3c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0062a0 tx=0x7f3f9c006210 comp rx=0 tx=0).stop 2023-09-23T05:11:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 msgr2=0x7f3fa41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:11:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e9a0 tx=0x7f3f9400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 shutdown_connections 2023-09-23T05:11:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f3f8c0a1790 0x7f3f8c0a3c50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa419dc40 0x7f3fa419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f3fa41010d0 0x7f3fa41992a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 --2- 172.21.15.5:0/645757419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fa41061f0 0x7f3fa41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:11:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 >> 172.21.15.5:0/645757419 conn(0x7f3fa40fc990 msgr2=0x7f3fa40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:11:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 shutdown_connections 2023-09-23T05:11:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:11:56.034+0000 7f3faa5ff700 1 -- 172.21.15.5:0/645757419 wait complete. 2023-09-23T05:11:56.052 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:11:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:56 smithi069 ceph-mon[133931]: pgmap v808: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:56.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:56 smithi005 ceph-mon[97638]: pgmap v808: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:56.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:56 smithi118 ceph-mon[134297]: pgmap v808: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:11:58.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:11:58 smithi069 ceph-mon[133931]: pgmap v809: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:58.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:11:58 smithi005 ceph-mon[97638]: pgmap v809: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:11:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:11:58 smithi118 ceph-mon[134297]: pgmap v809: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:00.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:00 smithi069 ceph-mon[133931]: pgmap v810: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:00.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:00 smithi005 ceph-mon[97638]: pgmap v810: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:00.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:00 smithi118 ceph-mon[134297]: pgmap v810: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:02 smithi005 ceph-mon[97638]: pgmap v811: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:02.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:02 smithi118 ceph-mon[134297]: pgmap v811: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:02.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:02 smithi069 ceph-mon[133931]: pgmap v811: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:04 smithi069 ceph-mon[133931]: pgmap v812: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:04.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:04 smithi005 ceph-mon[97638]: pgmap v812: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:04.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:04 smithi118 ceph-mon[134297]: pgmap v812: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:06.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:06 smithi069 ceph-mon[133931]: pgmap v813: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:06 smithi005 ceph-mon[97638]: pgmap v813: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:06 smithi118 ceph-mon[134297]: pgmap v813: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:08.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:08 smithi069 ceph-mon[133931]: pgmap v814: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-09-23T05:12:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:08 smithi005 ceph-mon[97638]: pgmap v814: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-09-23T05:12:08.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:08 smithi118 ceph-mon[134297]: pgmap v814: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-09-23T05:12:10.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:10 smithi069 ceph-mon[133931]: pgmap v815: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:12:10.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:10 smithi005 ceph-mon[97638]: pgmap v815: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:12:10.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:10 smithi118 ceph-mon[134297]: pgmap v815: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-09-23T05:12:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:12 smithi069 ceph-mon[133931]: pgmap v816: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:12.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:12 smithi069 ceph-mon[133931]: osdmap e129: 12 total, 12 up, 12 in 2023-09-23T05:12:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:12 smithi005 ceph-mon[97638]: pgmap v816: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:12.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:12 smithi005 ceph-mon[97638]: osdmap e129: 12 total, 12 up, 12 in 2023-09-23T05:12:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:12 smithi118 ceph-mon[134297]: pgmap v816: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-09-23T05:12:12.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:12 smithi118 ceph-mon[134297]: osdmap e129: 12 total, 12 up, 12 in 2023-09-23T05:12:14.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:14 smithi069 ceph-mon[133931]: pgmap v818: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:12:14.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:14 smithi005 ceph-mon[97638]: pgmap v818: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:12:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:14 smithi118 ceph-mon[134297]: pgmap v818: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:12:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:16 smithi069 ceph-mon[133931]: pgmap v819: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:16.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:16 smithi069 ceph-mon[133931]: osdmap e130: 12 total, 12 up, 12 in 2023-09-23T05:12:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:16 smithi005 ceph-mon[97638]: pgmap v819: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:16 smithi005 ceph-mon[97638]: osdmap e130: 12 total, 12 up, 12 in 2023-09-23T05:12:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:16 smithi118 ceph-mon[134297]: pgmap v819: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:16 smithi118 ceph-mon[134297]: osdmap e130: 12 total, 12 up, 12 in 2023-09-23T05:12:17.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:17 smithi069 ceph-mon[133931]: pgmap v821: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:12:17.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:17 smithi005 ceph-mon[97638]: pgmap v821: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:12:17.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:17 smithi118 ceph-mon[134297]: pgmap v821: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:12:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:20 smithi069 ceph-mon[133931]: pgmap v822: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2023-09-23T05:12:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:20 smithi005 ceph-mon[97638]: pgmap v822: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2023-09-23T05:12:20.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:20 smithi118 ceph-mon[134297]: pgmap v822: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2023-09-23T05:12:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:22 smithi005 ceph-mon[97638]: pgmap v823: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:12:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:22 smithi118 ceph-mon[134297]: pgmap v823: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:12:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:22 smithi069 ceph-mon[133931]: pgmap v823: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2023-09-23T05:12:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:24 smithi069 ceph-mon[133931]: pgmap v824: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:24.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:24 smithi005 ceph-mon[97638]: pgmap v824: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:24.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:24 smithi118 ceph-mon[134297]: pgmap v824: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:12:26.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:12:26.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 -- 172.21.15.5:0/2434191225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 msgr2=0x7fb32c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 --2- 172.21.15.5:0/2434191225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c10dd70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb32400b0a0 tx=0x7fb32400b3b0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 -- 172.21.15.5:0/2434191225 shutdown_connections 2023-09-23T05:12:26.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 --2- 172.21.15.5:0/2434191225 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c10dd70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 --2- 172.21.15.5:0/2434191225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 --2- 172.21.15.5:0/2434191225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c101a80 0x7fb32c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 -- 172.21.15.5:0/2434191225 >> 172.21.15.5:0/2434191225 conn(0x7fb32c0fc990 msgr2=0x7fb32c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 -- 172.21.15.5:0/2434191225 shutdown_connections 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.273+0000 7fb3331ff700 1 -- 172.21.15.5:0/2434191225 wait complete. 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 Processor -- start 2023-09-23T05:12:26.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 -- start start 2023-09-23T05:12:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 0x7fb32c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19c4f0 con 0x7fb32c1010d0 2023-09-23T05:12:26.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19c660 con 0x7fb32c19ba70 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.274+0000 7fb3331ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c19c820 con 0x7fb32c1061f0 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54744/0 (socket says 172.21.15.5:54744) 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 -- 172.21.15.5:0/3400823082 learned_addr learned my addr 172.21.15.5:0/3400823082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb33179c700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 0x7fb32c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 -- 172.21.15.5:0/3400823082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 msgr2=0x7fb32c19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 0x7fb32c19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 -- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 msgr2=0x7fb32c1970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 -- 172.21.15.5:0/3400823082 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb320009520 con 0x7fb32c1061f0 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb33179c700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 0x7fb32c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb330f9b700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb32bfff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb31800e970 tx=0x7fb31800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.275+0000 7fb329ffb700 1 -- 172.21.15.5:0/3400823082 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318009790 con 0x7fb32c1061f0 2023-09-23T05:12:26.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.276+0000 7fb329ffb700 1 -- 172.21.15.5:0/3400823082 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb31800eec0 con 0x7fb32c1061f0 2023-09-23T05:12:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.276+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb324009d00 con 0x7fb32c1061f0 2023-09-23T05:12:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.276+0000 7fb329ffb700 1 -- 172.21.15.5:0/3400823082 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318005590 con 0x7fb32c1061f0 2023-09-23T05:12:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.276+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c1a40d0 con 0x7fb32c1061f0 2023-09-23T05:12:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.279+0000 7fb329ffb700 1 -- 172.21.15.5:0/3400823082 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fb31800cbd0 con 0x7fb32c1061f0 2023-09-23T05:12:26.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 Processor -- start 2023-09-23T05:12:26.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 -- start start 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a466d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a467f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a45d6460 con 0x5600a466d400 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a45c7c20 con 0x5600a466dd20 2023-09-23T05:12:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a45c5540 con 0x5600a467cfe0 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.281+0000 7fb329ffb700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 0x7fb31c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a467f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:41178/0 (socket says 172.21.15.5:41178) 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 -- 172.21.15.5:0/357769957 learned_addr learned my addr 172.21.15.5:0/357769957 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb33179c700 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a466d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb329ffb700 1 -- 172.21.15.5:0/3400823082 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb318014070 con 0x7fb32c1061f0 2023-09-23T05:12:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb33179c700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 0x7fb31c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 -- 172.21.15.5:0/357769957 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 msgr2=0x5600a467f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a467f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 -- 172.21.15.5:0/357769957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 msgr2=0x5600a466d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a466d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 -- 172.21.15.5:0/357769957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a46238d0 con 0x5600a466dd20 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb32bfff700 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a467f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.282+0000 7fb330f9b700 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb320011760 tx=0x7fb320011a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edbb7c9128f3fae7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb33179c700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 0x7fb31c0a3bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fd0 tx=0x7fb32400b700 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb316ffd700 1 -- 172.21.15.5:0/357769957 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000f4e0 con 0x5600a466dd20 2023-09-23T05:12:26.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb316ffd700 1 -- 172.21.15.5:0/357769957 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb32000fb00 con 0x5600a466dd20 2023-09-23T05:12:26.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb316ffd700 1 -- 172.21.15.5:0/357769957 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320019b90 con 0x5600a466dd20 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb346bb1b80 1 -- 172.21.15.5:0/357769957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 msgr2=0x5600a467caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.283+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb320011760 tx=0x7fb320011a70 comp rx=0 tx=0).stop 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 -- 172.21.15.5:0/357769957 shutdown_connections 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a467f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a466d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/357769957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a467caa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 -- 172.21.15.5:0/357769957 >> 172.21.15.5:0/357769957 conn(0x5600a45e2260 msgr2=0x5600a4669790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 -- 172.21.15.5:0/357769957 shutdown_connections 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 -- 172.21.15.5:0/357769957 wait complete. 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 -- start start 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.284+0000 7fb346bb1b80 1 Processor -- start 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466d400 0x5600a4679d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a467cfe0 0x5600a467a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a45c0e30 con 0x5600a467cfe0 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a455d710 con 0x5600a466d400 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a455d3b0 con 0x5600a466dd20 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb346bb1b80 10 client.0 _ll_register_callbacks cb 0x5600a43ba800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb32bfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54760/0 (socket says 172.21.15.5:54760) 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb32bfff700 1 -- 172.21.15.5:0/4026709785 learned_addr learned my addr 172.21.15.5:0/4026709785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:26.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb33179c700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a467cfe0 0x5600a467a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.285+0000 7fb330f9b700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466d400 0x5600a4679d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 -- 172.21.15.5:0/4026709785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466d400 msgr2=0x5600a4679d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466d400 0x5600a4679d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 -- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a467cfe0 msgr2=0x5600a467a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a467cfe0 0x5600a467a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 -- 172.21.15.5:0/4026709785 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c1a40d0 con 0x5600a466dd20 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb32bfff700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb31809a400 tx=0x7fb318004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=932ae2c9fc754ee4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb30bfff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318016530 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a4661250 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb3180b1070 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb31809ab70 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb346bb1b80 10 client.28078 fetch_fsmap learned FSMap version 83 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fb320011440 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.286+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600a4661250 con 0x5600a466dd20 2023-09-23T05:12:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb318099960 con 0x5600a466dd20 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 10 client.28078 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 10 client.28078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 20 client.28078 populate_metadata read hostname 'smithi005' 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a468cba0 0x5600a468efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 4 client.28078 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a45b9e10 con 0x5600a468cba0 2023-09-23T05:12:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.287+0000 7fb330f9b700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a468cba0 0x5600a468efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb330f9b700 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a468cba0 0x5600a468efc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6ac38244e9ae91 server_cookie=89326d4ff685e166 in_seq=0 out_seq=0 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb30bfff700 10 client.28078 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb30bfff700 1 -- 172.21.15.5:0/4026709785 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5600a45b9e10 con 0x5600a468cba0 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb30bfff700 10 client.28078 handle_command_reply: tid=0 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 client.28078 shutdown 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 msgr2=0x5600a467a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb31809a400 tx=0x7fb318004ef0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 shutdown_connections 2023-09-23T05:12:26.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a468cba0 0x5600a468efc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb30bfff700 0 client.28078 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.288+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a467cfe0 0x5600a467a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466d400 0x5600a4679d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/4026709785 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a466dd20 0x5600a467a260 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 >> 172.21.15.5:0/4026709785 conn(0x5600a45e2260 msgr2=0x5600a4669380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 shutdown_connections 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 1 -- 172.21.15.5:0/4026709785 wait complete. 2023-09-23T05:12:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.289+0000 7fb346bb1b80 20 client.28078 trim_cache size 0 max 16384 2023-09-23T05:12:26.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 Processor -- start 2023-09-23T05:12:26.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 -- start start 2023-09-23T05:12:26.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4676fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4677510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a4441920 con 0x5600a466d400 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a466e100 con 0x5600a466dd20 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a468c980 con 0x5600a467cfe0 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb33179c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4677510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58408/0 (socket says 172.21.15.5:58408) 2023-09-23T05:12:26.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb330f9b700 1 -- 172.21.15.5:0/984399397 learned_addr learned my addr 172.21.15.5:0/984399397 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:26.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.307+0000 7fb32bfff700 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4676fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 -- 172.21.15.5:0/984399397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 msgr2=0x5600a4677510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4677510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 -- 172.21.15.5:0/984399397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 msgr2=0x5600a4676fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4676fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 -- 172.21.15.5:0/984399397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a43ebde0 con 0x5600a466d400 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb330f9b700 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb320011fd0 tx=0x7fb320019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cec6d41384aebf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb3177fe700 1 -- 172.21.15.5:0/984399397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320019530 con 0x5600a466d400 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb3177fe700 1 -- 172.21.15.5:0/984399397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb320005b00 con 0x5600a466d400 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.308+0000 7fb3177fe700 1 -- 172.21.15.5:0/984399397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320005710 con 0x5600a466d400 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- 172.21.15.5:0/984399397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 msgr2=0x5600a4683b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb320011fd0 tx=0x7fb320019710 comp rx=0 tx=0).stop 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- 172.21.15.5:0/984399397 shutdown_connections 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4677510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a4683b20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/984399397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4676fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- 172.21.15.5:0/984399397 >> 172.21.15.5:0/984399397 conn(0x5600a45e2260 msgr2=0x5600a4675040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- 172.21.15.5:0/984399397 shutdown_connections 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- 172.21.15.5:0/984399397 wait complete. 2023-09-23T05:12:26.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 -- start start 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.309+0000 7fb346bb1b80 1 Processor -- start 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4681b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4682050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a4683ee0 con 0x5600a466d400 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a4682590 con 0x5600a466dd20 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a4682750 con 0x5600a467cfe0 2023-09-23T05:12:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb346bb1b80 10 client.0 _ll_register_callbacks cb 0x7fb31809a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb330f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58424/0 (socket says 172.21.15.5:58424) 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb330f9b700 1 -- 172.21.15.5:0/2635226322 learned_addr learned my addr 172.21.15.5:0/2635226322 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb32bfff700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4681b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.310+0000 7fb33179c700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4682050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 -- 172.21.15.5:0/2635226322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 msgr2=0x5600a4682050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4682050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 -- 172.21.15.5:0/2635226322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 msgr2=0x5600a4681b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4681b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 -- 172.21.15.5:0/2635226322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a46238d0 con 0x5600a466d400 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb32bfff700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4681b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb330f9b700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb32001c040 tx=0x7fb320005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976ac7d5e922acb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb30bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320005500 con 0x5600a466d400 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a45b9b30 con 0x5600a466d400 2023-09-23T05:12:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.311+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fb320005660 con 0x5600a466d400 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32000f4e0 con 0x5600a466d400 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 10 client.18000 fetch_fsmap learned FSMap version 83 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5600a43d4c60 con 0x5600a466d400 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600a45b9b30 con 0x5600a466d400 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb320003680 con 0x5600a466d400 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 10 client.18000 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:12:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 20 client.18000 populate_metadata read hostname 'smithi005' 2023-09-23T05:12:26.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a46e1a40 0x5600a46e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:26.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 4 client.18000 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:12:26.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600a440cf70 con 0x5600a46e1a40 2023-09-23T05:12:26.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.312+0000 7fb32bfff700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a46e1a40 0x5600a46e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:26.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb32bfff700 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a46e1a40 0x5600a46e3e60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e60069aeb3bf74 server_cookie=b8df722fc80a69f in_seq=0 out_seq=0 2023-09-23T05:12:26.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb30bfff700 10 client.18000 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb30bfff700 1 -- 172.21.15.5:0/2635226322 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5600a440cf70 con 0x5600a46e1a40 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb30bfff700 10 client.18000 handle_command_reply: tid=0 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 client.18000 shutdown 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 msgr2=0x5600a46835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb32001c040 tx=0x7fb320005b00 comp rx=0 tx=0).stop 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 shutdown_connections 2023-09-23T05:12:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x5600a46e1a40 0x5600a46e3e60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x5600a467cfe0 0x5600a4682050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600a466d400 0x5600a46835f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb30bfff700 0 client.18000 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 --2- 172.21.15.5:0/2635226322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5600a466dd20 0x5600a4681b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 >> 172.21.15.5:0/2635226322 conn(0x5600a45e2260 msgr2=0x5600a4675040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.313+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 shutdown_connections 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb346bb1b80 1 -- 172.21.15.5:0/2635226322 wait complete. 2023-09-23T05:12:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb346bb1b80 20 client.18000 trim_cache size 0 max 16384 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:12:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:12:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:12:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:12:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:12:26.325 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:12:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 msgr2=0x7fb31c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 0x7fb31c0a3bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb324009fd0 tx=0x7fb32400b700 comp rx=0 tx=0).stop 2023-09-23T05:12:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 msgr2=0x7fb32c197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb31800e970 tx=0x7fb31800ec80 comp rx=0 tx=0).stop 2023-09-23T05:12:26.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 shutdown_connections 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fb31c0a1710 0x7fb31c0a3bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb32c19ba70 0x7fb32c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb32c1010d0 0x7fb32c1970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 --2- 172.21.15.5:0/3400823082 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fb32c1061f0 0x7fb32c197610 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.314+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 >> 172.21.15.5:0/3400823082 conn(0x7fb32c0fc990 msgr2=0x7fb32c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.315+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 shutdown_connections 2023-09-23T05:12:26.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:26.315+0000 7fb3331ff700 1 -- 172.21.15.5:0/3400823082 wait complete. 2023-09-23T05:12:26.334 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:12:26.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:26 smithi005 ceph-mon[97638]: pgmap v825: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:12:26.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:26 smithi069 ceph-mon[133931]: pgmap v825: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:12:26.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:26 smithi118 ceph-mon[134297]: pgmap v825: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:12:28.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:28 smithi069 ceph-mon[133931]: pgmap v826: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:12:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:28 smithi005 ceph-mon[97638]: pgmap v826: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:12:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:28 smithi118 ceph-mon[134297]: pgmap v826: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:12:30.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:30 smithi069 ceph-mon[133931]: pgmap v827: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:30.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:30 smithi005 ceph-mon[97638]: pgmap v827: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:30.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:30 smithi118 ceph-mon[134297]: pgmap v827: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:32 smithi005 ceph-mon[97638]: pgmap v828: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:32 smithi118 ceph-mon[134297]: pgmap v828: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:32 smithi069 ceph-mon[133931]: pgmap v828: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:12:34.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:34 smithi069 ceph-mon[133931]: pgmap v829: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:34.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:34 smithi005 ceph-mon[97638]: pgmap v829: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:34 smithi118 ceph-mon[134297]: pgmap v829: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:36.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:36 smithi069 ceph-mon[133931]: pgmap v830: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:36.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:36 smithi005 ceph-mon[97638]: pgmap v830: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:36.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:36 smithi118 ceph-mon[134297]: pgmap v830: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:38.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:38 smithi069 ceph-mon[133931]: pgmap v831: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:38.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:38 smithi005 ceph-mon[97638]: pgmap v831: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:38 smithi118 ceph-mon[134297]: pgmap v831: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:40.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:40 smithi005 ceph-mon[97638]: pgmap v832: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:40.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:40 smithi118 ceph-mon[134297]: pgmap v832: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:40.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:40 smithi069 ceph-mon[133931]: pgmap v832: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:42.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:42 smithi005 ceph-mon[97638]: pgmap v833: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:42 smithi118 ceph-mon[134297]: pgmap v833: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:42 smithi069 ceph-mon[133931]: pgmap v833: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:44 smithi069 ceph-mon[133931]: pgmap v834: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:44 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:12:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:44 smithi005 ceph-mon[97638]: pgmap v834: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:44 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:12:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:44 smithi118 ceph-mon[134297]: pgmap v834: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:44 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:12:46.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:46 smithi069 ceph-mon[133931]: pgmap v835: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:46.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:46 smithi005 ceph-mon[97638]: pgmap v835: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:46.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:46 smithi118 ceph-mon[134297]: pgmap v835: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:48 smithi069 ceph-mon[133931]: pgmap v836: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:48 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:48.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:48 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:48 smithi005 ceph-mon[97638]: pgmap v836: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:48 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:48 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:48 smithi118 ceph-mon[134297]: pgmap v836: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:48 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:48 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: pgmap v837: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:12:50.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:12:50.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: pgmap v837: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:12:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: pgmap v837: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:50.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:50.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:12:50.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:12:50.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:12:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:52 smithi005 ceph-mon[97638]: pgmap v838: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:52.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:52 smithi118 ceph-mon[134297]: pgmap v838: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:52 smithi069 ceph-mon[133931]: pgmap v838: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:54.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:54 smithi069 ceph-mon[133931]: pgmap v839: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:54.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:54 smithi005 ceph-mon[97638]: pgmap v839: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:54.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:54 smithi118 ceph-mon[134297]: pgmap v839: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:56.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:12:56.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:56 smithi069 ceph-mon[133931]: pgmap v840: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.557+0000 7fe0f2dff700 1 -- 172.21.15.5:0/420518042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec1061f0 msgr2=0x7fe0ec10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.557+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/420518042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec10dd70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009f90 tx=0x7fe0e400b330 comp rx=0 tx=0).stop 2023-09-23T05:12:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 -- 172.21.15.5:0/420518042 shutdown_connections 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/420518042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec10dd70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/420518042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/420518042 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec101a80 0x7fe0ec105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 -- 172.21.15.5:0/420518042 >> 172.21.15.5:0/420518042 conn(0x7fe0ec0fc990 msgr2=0x7fe0ec0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 -- 172.21.15.5:0/420518042 shutdown_connections 2023-09-23T05:12:56.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.558+0000 7fe0f2dff700 1 -- 172.21.15.5:0/420518042 wait complete. 2023-09-23T05:12:56.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 Processor -- start 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 -- start start 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec19da90 0x7fe0ec19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19e510 con 0x7fe0ec1010d0 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19e680 con 0x7fe0ec19da90 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f2dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19e840 con 0x7fe0ec1061f0 2023-09-23T05:12:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.559+0000 7fe0f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51244/0 (socket says 172.21.15.5:51244) 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39062/0 (socket says 172.21.15.5:39062) 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0f0b9b700 1 -- 172.21.15.5:0/601627607 learned_addr learned my addr 172.21.15.5:0/601627607 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0f139c700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec19da90 0x7fe0ec19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 -- 172.21.15.5:0/601627607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec19da90 msgr2=0x7fe0ec19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec19da90 0x7fe0ec19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 -- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 msgr2=0x7fe0ec1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 -- 172.21.15.5:0/601627607 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e00095f0 con 0x7fe0ec1061f0 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9c0 tx=0x7fe0d800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.560+0000 7fe0e9ffb700 1 -- 172.21.15.5:0/601627607 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d800c9c0 con 0x7fe0ec1061f0 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.561+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4009c40 con 0x7fe0ec1061f0 2023-09-23T05:12:56.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.561+0000 7fe0e9ffb700 1 -- 172.21.15.5:0/601627607 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d8010430 con 0x7fe0ec1061f0 2023-09-23T05:12:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.561+0000 7fe0e9ffb700 1 -- 172.21.15.5:0/601627607 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8018680 con 0x7fe0ec1061f0 2023-09-23T05:12:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.561+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec1a6140 con 0x7fe0ec1061f0 2023-09-23T05:12:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.564+0000 7fe0e9ffb700 1 -- 172.21.15.5:0/601627607 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe0d800cb20 con 0x7fe0ec1061f0 2023-09-23T05:12:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.565+0000 7fe1067f7b80 1 Processor -- start 2023-09-23T05:12:56.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.565+0000 7fe1067f7b80 1 -- start start 2023-09-23T05:12:56.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0e9ffb700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 0x7fe0dc0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d1ff460 con 0x55f89d2a1bd0 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d1f0c20 con 0x55f89d296400 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d1ee540 con 0x55f89d296d20 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0e9ffb700 1 -- 172.21.15.5:0/601627607 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe0d8014070 con 0x7fe0ec1061f0 2023-09-23T05:12:56.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0ebfff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39078/0 (socket says 172.21.15.5:39078) 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0ebfff700 1 -- 172.21.15.5:0/410400722 learned_addr learned my addr 172.21.15.5:0/410400722 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0f139c700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 0x7fe0dc0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0ebfff700 1 -- 172.21.15.5:0/410400722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 msgr2=0x55f89d2967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.566+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0f139c700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0ebfff700 1 -- 172.21.15.5:0/410400722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 msgr2=0x55f89d2a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0ebfff700 1 -- 172.21.15.5:0/410400722 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f89d24c8d0 con 0x55f89d296d20 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0f139c700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80a2120 tx=0x7fe0d8099e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=389b841b718effd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0d6ffd700 1 -- 172.21.15.5:0/410400722 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8063c10 con 0x55f89d296d20 2023-09-23T05:12:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0d6ffd700 1 -- 172.21.15.5:0/410400722 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0d8062eb0 con 0x55f89d296d20 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.567+0000 7fe0d6ffd700 1 -- 172.21.15.5:0/410400722 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8098400 con 0x55f89d296d20 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- 172.21.15.5:0/410400722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 msgr2=0x55f89d2a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe0f139c700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 0x7fe0dc0a3c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009f60 tx=0x7fe0e4009450 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80a2120 tx=0x7fe0d8099e00 comp rx=0 tx=0).stop 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- 172.21.15.5:0/410400722 shutdown_connections 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/410400722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d296d20 0x55f89d2a1690 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- 172.21.15.5:0/410400722 >> 172.21.15.5:0/410400722 conn(0x55f89d20b260 msgr2=0x55f89d292790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- 172.21.15.5:0/410400722 shutdown_connections 2023-09-23T05:12:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- 172.21.15.5:0/410400722 wait complete. 2023-09-23T05:12:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.568+0000 7fe1067f7b80 1 -- start start 2023-09-23T05:12:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 Processor -- start 2023-09-23T05:12:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d2a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d29e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d1e9e30 con 0x55f89d296400 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d186710 con 0x55f89d296d20 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d1863b0 con 0x55f89d2a1bd0 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe1067f7b80 10 client.0 _ll_register_callbacks cb 0x55f89cfe3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.569+0000 7fe0f139c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d29e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d2a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0f139c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39094/0 (socket says 172.21.15.5:39094) 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0f139c700 1 -- 172.21.15.5:0/520232918 learned_addr learned my addr 172.21.15.5:0/520232918 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 -- 172.21.15.5:0/520232918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 msgr2=0x55f89d29f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 -- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 msgr2=0x55f89d2a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d2a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 -- 172.21.15.5:0/520232918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec1a6140 con 0x55f89d296d20 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d29e970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe0d809e5d0 tx=0x7fe0d8063e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab0f88306530ec2d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d809a2c0 con 0x55f89d296d20 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f89d28a250 con 0x55f89d296d20 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.570+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mon.2 v2:172.21.15.69:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe0d809e410 con 0x55f89d296d20 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d809a9c0 con 0x55f89d296d20 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 10 client.28052 fetch_fsmap learned FSMap version 83 2023-09-23T05:12:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55f89d014de0 con 0x55f89d296d20 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mon.2 v2:172.21.15.69:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f89d28a250 con 0x55f89d296d20 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mon.2 v2:172.21.15.69:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe0d809e020 con 0x55f89d296d20 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 10 client.28052 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 10 client.28052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 20 client.28052 populate_metadata read hostname 'smithi005' 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d2b1210 0x55f89d2b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 4 client.28052 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:12:56.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.571+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f89d1e2e10 con 0x55f89d2b1210 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.572+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d2b1210 0x55f89d2b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.572+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d2b1210 0x55f89d2b3600 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0478eabb2b455cb server_cookie=b07de4fe70952f4b in_seq=0 out_seq=0 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.572+0000 7fe0cbfff700 10 client.28052 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe0cbfff700 1 -- 172.21.15.5:0/520232918 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f89d1e2e10 con 0x55f89d2b1210 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe0cbfff700 10 client.28052 handle_command_reply: tid=0 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 client.28052 shutdown 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 msgr2=0x55f89d29e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d29e970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe0d809e5d0 tx=0x7fe0d8063e50 comp rx=0 tx=0).stop 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 shutdown_connections 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d2b1210 0x55f89d2b3600 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d2a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/520232918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d29e970 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe0cbfff700 0 client.28052 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 >> 172.21.15.5:0/520232918 conn(0x55f89d20b260 msgr2=0x55f89d292320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 shutdown_connections 2023-09-23T05:12:56.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 1 -- 172.21.15.5:0/520232918 wait complete. 2023-09-23T05:12:56.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.573+0000 7fe1067f7b80 20 client.28052 trim_cache size 0 max 16384 2023-09-23T05:12:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 Processor -- start 2023-09-23T05:12:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 -- start start 2023-09-23T05:12:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d299cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d06a920 con 0x55f89d296400 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d29f920 con 0x55f89d296d20 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d28a140 con 0x55f89d2a1bd0 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe0f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.591+0000 7fe0f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51276/0 (socket says 172.21.15.5:51276) 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 -- 172.21.15.5:0/3959585532 learned_addr learned my addr 172.21.15.5:0/3959585532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f139c700 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d299cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 -- 172.21.15.5:0/3959585532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 msgr2=0x55f89d29a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 -- 172.21.15.5:0/3959585532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 msgr2=0x55f89d299cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d299cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 -- 172.21.15.5:0/3959585532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f89d28e480 con 0x55f89d296400 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005b40 tx=0x7fe0e0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a6ba16676e1793 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0d77fe700 1 -- 172.21.15.5:0/3959585532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0025030 con 0x55f89d296400 2023-09-23T05:12:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0d77fe700 1 -- 172.21.15.5:0/3959585532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e0018040 con 0x55f89d296400 2023-09-23T05:12:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.592+0000 7fe0d77fe700 1 -- 172.21.15.5:0/3959585532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0017040 con 0x55f89d296400 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 -- 172.21.15.5:0/3959585532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 msgr2=0x55f89d299330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005b40 tx=0x7fe0e0009310 comp rx=0 tx=0).stop 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 -- 172.21.15.5:0/3959585532 shutdown_connections 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d29a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296400 0x55f89d299330 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/3959585532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296d20 0x55f89d299cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 -- 172.21.15.5:0/3959585532 >> 172.21.15.5:0/3959585532 conn(0x55f89d20b260 msgr2=0x55f89d290f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 -- 172.21.15.5:0/3959585532 shutdown_connections 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.593+0000 7fe1067f7b80 1 -- 172.21.15.5:0/3959585532 wait complete. 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 -- start start 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 Processor -- start 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296d20 0x55f89d2a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d2a6e10 con 0x55f89d296d20 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d2a6f80 con 0x55f89d296400 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89d2a7140 con 0x55f89d2a1bd0 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe1067f7b80 10 client.0 _ll_register_callbacks cb 0x7fe0d809a090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe0f139c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe0f139c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39118/0 (socket says 172.21.15.5:39118) 2023-09-23T05:12:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.594+0000 7fe0f139c700 1 -- 172.21.15.5:0/406574081 learned_addr learned my addr 172.21.15.5:0/406574081 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0ebfff700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296d20 0x55f89d2a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 -- 172.21.15.5:0/406574081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 msgr2=0x55f89d2a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2a81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 -- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296d20 msgr2=0x55f89d2a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296d20 0x55f89d2a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 -- 172.21.15.5:0/406574081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e0012570 con 0x55f89d2a1bd0 2023-09-23T05:12:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0f139c700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4008010 tx=0x7fe0e4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5327071501e926f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.595+0000 7fe0cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40052c0 con 0x55f89d2a1bd0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0d8062eb0 con 0x55f89d2a1bd0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe0e4005420 con 0x55f89d2a1bd0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e40055e0 con 0x55f89d2a1bd0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 10 client.28102 fetch_fsmap learned FSMap version 83 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55f89cffdc60 con 0x55f89d2a1bd0 2023-09-23T05:12:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e403b020 con 0x55f89d2a1bd0 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe0e4032bb0 con 0x55f89d2a1bd0 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 10 client.28102 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 10 client.28102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 20 client.28102 populate_metadata read hostname 'smithi005' 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d306280 0x55f89d3086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 4 client.28102 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:12:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.596+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f89d1e2b30 con 0x55f89d306280 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d306280 0x55f89d3086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe0f0b9b700 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d306280 0x55f89d3086a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73438d20e2e9b67 server_cookie=9ab9fb2e654dd132 in_seq=0 out_seq=0 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe0cbfff700 10 client.28102 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe0cbfff700 1 -- 172.21.15.5:0/406574081 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f89d1e2b30 con 0x55f89d306280 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe0cbfff700 10 client.28102 handle_command_reply: tid=0 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.597+0000 7fe1067f7b80 1 client.28102 shutdown 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 msgr2=0x55f89d2a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4008010 tx=0x7fe0e4009920 comp rx=0 tx=0).stop 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 shutdown_connections 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x55f89d306280 0x55f89d3086a0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x55f89d2a1bd0 0x55f89d2a6700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f89d296400 0x55f89d2a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 --2- 172.21.15.5:0/406574081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f89d296d20 0x55f89d2a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe0cbfff700 0 client.28102 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 >> 172.21.15.5:0/406574081 conn(0x55f89d20b260 msgr2=0x55f89d2919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 shutdown_connections 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 1 -- 172.21.15.5:0/406574081 wait complete. 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe1067f7b80 20 client.28102 trim_cache size 0 max 16384 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:12:56.608 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:12:56.609 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:12:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 msgr2=0x7fe0dc0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.598+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 0x7fe0dc0a3c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009f60 tx=0x7fe0e4009450 comp rx=0 tx=0).stop 2023-09-23T05:12:56.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 msgr2=0x7fe0ec199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e9c0 tx=0x7fe0d800ecd0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 shutdown_connections 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe0dc0a1790 0x7fe0dc0a3c50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0ec19da90 0x7fe0ec19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec1010d0 0x7fe0ec1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 --2- 172.21.15.5:0/601627607 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe0ec1061f0 0x7fe0ec199630 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 >> 172.21.15.5:0/601627607 conn(0x7fe0ec0fc990 msgr2=0x7fe0ec10c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:12:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 shutdown_connections 2023-09-23T05:12:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:12:56.599+0000 7fe0f2dff700 1 -- 172.21.15.5:0/601627607 wait complete. 2023-09-23T05:12:56.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:56 smithi005 ceph-mon[97638]: pgmap v840: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:56.618 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:12:56.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:56 smithi118 ceph-mon[134297]: pgmap v840: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:12:58.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:58 smithi005 ceph-mon[97638]: pgmap v841: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:58.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:58 smithi118 ceph-mon[134297]: pgmap v841: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:58.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:58 smithi069 ceph-mon[133931]: pgmap v841: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:12:59.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:12:59 smithi005 ceph-mon[97638]: pgmap v842: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:59.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:12:59 smithi118 ceph-mon[134297]: pgmap v842: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:12:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:12:59 smithi069 ceph-mon[133931]: pgmap v842: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:02.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:02 smithi005 ceph-mon[97638]: pgmap v843: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:02.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:02 smithi118 ceph-mon[134297]: pgmap v843: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:02.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:02 smithi069 ceph-mon[133931]: pgmap v843: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:04.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:04 smithi069 ceph-mon[133931]: pgmap v844: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:04.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:04 smithi005 ceph-mon[97638]: pgmap v844: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:04.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:04 smithi118 ceph-mon[134297]: pgmap v844: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:06.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:06 smithi005 ceph-mon[97638]: pgmap v845: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:13:06.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:06 smithi118 ceph-mon[134297]: pgmap v845: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:13:06.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:06 smithi069 ceph-mon[133931]: pgmap v845: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:13:08.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:08 smithi005 ceph-mon[97638]: pgmap v846: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:13:08.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:08 smithi118 ceph-mon[134297]: pgmap v846: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:13:08.742 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:08 smithi069 ceph-mon[133931]: pgmap v846: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:13:09.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:09 smithi005 ceph-mon[97638]: pgmap v847: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:13:09.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:09 smithi118 ceph-mon[134297]: pgmap v847: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:13:09.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:09 smithi069 ceph-mon[133931]: pgmap v847: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:13:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:11 smithi005 ceph-mon[97638]: osdmap e131: 12 total, 12 up, 12 in 2023-09-23T05:13:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:11 smithi005 ceph-mon[97638]: pgmap v849: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:11 smithi118 ceph-mon[134297]: osdmap e131: 12 total, 12 up, 12 in 2023-09-23T05:13:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:11 smithi118 ceph-mon[134297]: pgmap v849: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:11 smithi069 ceph-mon[133931]: osdmap e131: 12 total, 12 up, 12 in 2023-09-23T05:13:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:11 smithi069 ceph-mon[133931]: pgmap v849: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:14.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:14 smithi005 ceph-mon[97638]: pgmap v850: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:14.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:14 smithi118 ceph-mon[134297]: pgmap v850: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:14.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:14 smithi069 ceph-mon[133931]: pgmap v850: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:13:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:16 smithi005 ceph-mon[97638]: pgmap v851: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:16.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:16 smithi005 ceph-mon[97638]: osdmap e132: 12 total, 12 up, 12 in 2023-09-23T05:13:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:16 smithi118 ceph-mon[134297]: pgmap v851: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:16.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:16 smithi118 ceph-mon[134297]: osdmap e132: 12 total, 12 up, 12 in 2023-09-23T05:13:16.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:16 smithi069 ceph-mon[133931]: pgmap v851: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:16.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:16 smithi069 ceph-mon[133931]: osdmap e132: 12 total, 12 up, 12 in 2023-09-23T05:13:17.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:17 smithi005 ceph-mon[97638]: pgmap v853: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:13:17.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:17 smithi118 ceph-mon[134297]: pgmap v853: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:13:17.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:17 smithi069 ceph-mon[133931]: pgmap v853: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:13:20.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:20 smithi005 ceph-mon[97638]: pgmap v854: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:13:20.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:20 smithi118 ceph-mon[134297]: pgmap v854: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:13:20.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:20 smithi069 ceph-mon[133931]: pgmap v854: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:13:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:22 smithi005 ceph-mon[97638]: pgmap v855: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:22 smithi118 ceph-mon[134297]: pgmap v855: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:22 smithi069 ceph-mon[133931]: pgmap v855: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:24.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:24 smithi005 ceph-mon[97638]: pgmap v856: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:24.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:24 smithi118 ceph-mon[134297]: pgmap v856: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:24.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:24 smithi069 ceph-mon[133931]: pgmap v856: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:13:25.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:25 smithi005 ceph-mon[97638]: pgmap v857: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:13:25.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:25 smithi118 ceph-mon[134297]: pgmap v857: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:13:25.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:25 smithi069 ceph-mon[133931]: pgmap v857: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-23T05:13:26.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-23T05:13:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 -- 172.21.15.5:0/148543098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1701010d0 msgr2=0x7fa1701014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 --2- 172.21.15.5:0/148543098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1701010d0 0x7fa1701014b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa160009990 tx=0x7fa160009ca0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 -- 172.21.15.5:0/148543098 shutdown_connections 2023-09-23T05:13:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 --2- 172.21.15.5:0/148543098 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa17010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 --2- 172.21.15.5:0/148543098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1701010d0 0x7fa1701014b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 --2- 172.21.15.5:0/148543098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa170101a80 0x7fa170105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 -- 172.21.15.5:0/148543098 >> 172.21.15.5:0/148543098 conn(0x7fa1700fc990 msgr2=0x7fa1700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 -- 172.21.15.5:0/148543098 shutdown_connections 2023-09-23T05:13:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.839+0000 7fa1769ff700 1 -- 172.21.15.5:0/148543098 wait complete. 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.840+0000 7fa1769ff700 1 Processor -- start 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.840+0000 7fa1769ff700 1 -- start start 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1701010d0 0x7fa1701991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17019db60 0x7fa17019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17019e5e0 con 0x7fa17019db60 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17019e750 con 0x7fa1701010d0 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa1769ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17019e910 con 0x7fa1701061f0 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48884/0 (socket says 172.21.15.5:48884) 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 -- 172.21.15.5:0/1864198722 learned_addr learned my addr 172.21.15.5:0/1864198722 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa174f9c700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17019db60 0x7fa17019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa16ffff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1701010d0 0x7fa1701991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 -- 172.21.15.5:0/1864198722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1701010d0 msgr2=0x7fa1701991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1701010d0 0x7fa1701991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 -- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17019db60 msgr2=0x7fa17019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17019db60 0x7fa17019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.841+0000 7fa167fff700 1 -- 172.21.15.5:0/1864198722 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600095f0 con 0x7fa1701061f0 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa167fff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e970 tx=0x7fa15800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa16dffb700 1 -- 172.21.15.5:0/1864198722 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158009790 con 0x7fa1701061f0 2023-09-23T05:13:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1701a5e70 con 0x7fa1701061f0 2023-09-23T05:13:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa16dffb700 1 -- 172.21.15.5:0/1864198722 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15800c7e0 con 0x7fa1701061f0 2023-09-23T05:13:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa16dffb700 1 -- 172.21.15.5:0/1864198722 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580054e0 con 0x7fa1701061f0 2023-09-23T05:13:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.842+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1701a6340 con 0x7fa1701061f0 2023-09-23T05:13:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.847+0000 7fa16dffb700 1 -- 172.21.15.5:0/1864198722 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa15800c950 con 0x7fa1701061f0 2023-09-23T05:13:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 Processor -- start 2023-09-23T05:13:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 -- start start 2023-09-23T05:13:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700ca37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 0x561700cae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700c0c460 con 0x561700ca3d20 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700bfdc20 con 0x561700ca3400 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.848+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700bfb540 con 0x561700caebd0 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa16dffb700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 0x7fa15c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa16ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 0x561700cae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48890/0 (socket says 172.21.15.5:48890) 2023-09-23T05:13:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 -- 172.21.15.5:0/237568600 learned_addr learned my addr 172.21.15.5:0/237568600 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa174f9c700 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700ca37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa16dffb700 1 -- 172.21.15.5:0/1864198722 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa158014070 con 0x7fa1701061f0 2023-09-23T05:13:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa174f9c700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 0x7fa15c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 -- 172.21.15.5:0/237568600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 msgr2=0x561700ca37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700ca37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 -- 172.21.15.5:0/237568600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 msgr2=0x561700cae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 0x561700cae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 -- 172.21.15.5:0/237568600 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561700c598d0 con 0x561700caebd0 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa16ffff700 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 0x561700cae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:13:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.849+0000 7fa167fff700 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa15800cc00 tx=0x7fa15800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3867b0fb615ffd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa1667fc700 1 -- 172.21.15.5:0/237568600 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800cc50 con 0x561700caebd0 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa1667fc700 1 -- 172.21.15.5:0/237568600 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15809e580 con 0x561700caebd0 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa1667fc700 1 -- 172.21.15.5:0/237568600 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158016ea0 con 0x561700caebd0 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/237568600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 msgr2=0x561700cb0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa174f9c700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 0x7fa15c0a3c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa168009ec0 tx=0x7fa168004b40 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa15800cc00 tx=0x7fa15800cf50 comp rx=0 tx=0).stop 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/237568600 shutdown_connections 2023-09-23T05:13:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700caebd0 0x561700cb0fc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700ca37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/237568600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3d20 0x561700cae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.850+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/237568600 >> 172.21.15.5:0/237568600 conn(0x561700c18260 msgr2=0x561700c9f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.851+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/237568600 shutdown_connections 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.851+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/237568600 wait complete. 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.851+0000 7fa18a3ebb80 1 -- start start 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.851+0000 7fa18a3ebb80 1 Processor -- start 2023-09-23T05:13:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 0x561700cae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 0x561700cabf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700bf6e30 con 0x561700ca3400 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700b93710 con 0x561700caebd0 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700b933b0 con 0x561700ca3d20 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa18a3ebb80 10 client.0 _ll_register_callbacks cb 0x5617009f0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:13:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa16ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 0x561700cae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa174f9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 0x561700cabf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48898/0 (socket says 172.21.15.5:48898) 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa167fff700 1 -- 172.21.15.5:0/1698267982 learned_addr learned my addr 172.21.15.5:0/1698267982 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa16ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 0x561700cae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56898/0 (socket says 172.21.15.5:56898) 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa167fff700 1 -- 172.21.15.5:0/1698267982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 msgr2=0x561700cabf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.852+0000 7fa167fff700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 0x561700cabf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa167fff700 1 -- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 msgr2=0x561700cae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa167fff700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 0x561700cae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa167fff700 1 -- 172.21.15.5:0/1698267982 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17010e5e0 con 0x561700ca3d20 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa174f9c700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 0x561700cabf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa167fff700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa15809a3c0 tx=0x7fa15809a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd683dc317bfb47d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa14f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580a27c0 con 0x561700ca3d20 2023-09-23T05:13:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561700c97250 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa15809ab30 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa158016dd0 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa18a3ebb80 10 client.28114 fetch_fsmap learned FSMap version 83 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x7fa1701a6340 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.853+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561700c97250 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa158099920 con 0x561700ca3d20 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 10 client.28114 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:13:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 10 client.28114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 20 client.28114 populate_metadata read hostname 'smithi005' 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700cbe200 0x561700cc05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 4 client.28114 mds_command: new command op to 24577 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561700befe10 con 0x561700cbe200 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.854+0000 7fa16ffff700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700cbe200 0x561700cc05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa16ffff700 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700cbe200 0x561700cc05f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9c054ef1d3b30e server_cookie=740333d92c76c6f8 in_seq=0 out_seq=0 2023-09-23T05:13:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa14f7fe700 10 client.28114 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:13:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa14f7fe700 1 -- 172.21.15.5:0/1698267982 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561700befe10 con 0x561700cbe200 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa14f7fe700 10 client.28114 handle_command_reply: tid=0 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa18a3ebb80 1 client.28114 shutdown 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 msgr2=0x561700cab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.855+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa15809a3c0 tx=0x7fa15809a4a0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 shutdown_connections 2023-09-23T05:13:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700cbe200 0x561700cc05f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700caebd0 0x561700cabf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa14f7fe700 0 client.28114 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700ca3400 0x561700cae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/1698267982 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab860 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 >> 172.21.15.5:0/1698267982 conn(0x561700c18260 msgr2=0x561700c9f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 shutdown_connections 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/1698267982 wait complete. 2023-09-23T05:13:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.856+0000 7fa18a3ebb80 20 client.28114 trim_cache size 0 max 16384 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.873+0000 7fa18a3ebb80 1 Processor -- start 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 -- start start 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700cab020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700ca8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700a77920 con 0x561700caebd0 2023-09-23T05:13:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700ca4100 con 0x561700ca3400 2023-09-23T05:13:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700ca4300 con 0x561700ca3d20 2023-09-23T05:13:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa174f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700ca8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48912/0 (socket says 172.21.15.5:48912) 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 -- 172.21.15.5:0/2435556514 learned_addr learned my addr 172.21.15.5:0/2435556514 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa16ffff700 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700cab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 -- 172.21.15.5:0/2435556514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 msgr2=0x561700cab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700cab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.874+0000 7fa167fff700 1 -- 172.21.15.5:0/2435556514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 msgr2=0x561700ca8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa167fff700 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700ca8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa167fff700 1 -- 172.21.15.5:0/2435556514 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561700a21de0 con 0x561700ca3d20 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa167fff700 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa158098780 tx=0x7fa15800cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec3218f8e9466764 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa166ffd700 1 -- 172.21.15.5:0/2435556514 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15807b030 con 0x561700ca3d20 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa166ffd700 1 -- 172.21.15.5:0/2435556514 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15809ab30 con 0x561700ca3d20 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.875+0000 7fa166ffd700 1 -- 172.21.15.5:0/2435556514 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15809e140 con 0x561700ca3d20 2023-09-23T05:13:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/2435556514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 msgr2=0x561700cab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa158098780 tx=0x7fa15800cf50 comp rx=0 tx=0).stop 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/2435556514 shutdown_connections 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700ca8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700cab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/2435556514 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700cab560 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/2435556514 >> 172.21.15.5:0/2435556514 conn(0x561700c18260 msgr2=0x561700ca6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/2435556514 shutdown_connections 2023-09-23T05:13:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/2435556514 wait complete. 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 -- start start 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.876+0000 7fa18a3ebb80 1 Processor -- start 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700d0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700cb1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700cb1c60 con 0x561700caebd0 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700cb1dd0 con 0x561700ca3400 2023-09-23T05:13:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x561700cb1f90 con 0x561700ca3d20 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa18a3ebb80 10 client.0 _ll_register_callbacks cb 0x7fa158005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48916/0 (socket says 172.21.15.5:48916) 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa174f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700cb1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 -- 172.21.15.5:0/3093105271 learned_addr learned my addr 172.21.15.5:0/3093105271 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa16ffff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700d0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 -- 172.21.15.5:0/3093105271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 msgr2=0x561700d0a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700d0a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 -- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 msgr2=0x561700cb1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700cb1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.877+0000 7fa167fff700 1 -- 172.21.15.5:0/3093105271 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1701a6340 con 0x561700ca3d20 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa167fff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa15809a130 tx=0x7fa1580a2610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901adb56e6034b82 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.118:3300/0 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15807b070 con 0x561700ca3d20 2023-09-23T05:13:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mon.1 v2:172.21.15.118:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa15807c070 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa15809e580 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1580a2430 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 10 client.28120 fetch_fsmap learned FSMap version 83 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x561700a0ac60 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mon.1 v2:172.21.15.118:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15809e580 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mon.1 v2:172.21.15.118:3300/0 5 ==== fsmap(e 83) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa158004a30 con 0x561700ca3d20 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 10 client.28120 fetch_fsmap finished waiting for FSMap version 83 2023-09-23T05:13:26.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 10 client.28120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.g 2023-09-23T05:13:26.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 20 client.28120 populate_metadata read hostname 'smithi005' 2023-09-23T05:13:26.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700d12d50 0x561700d15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:26.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 4 client.28120 mds_command: new command op to 24577 tid=0[{"prefix": "scrub status"}] 2023-09-23T05:13:26.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.878+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 --> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561700befb30 con 0x561700d12d50 2023-09-23T05:13:26.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa16ffff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700d12d50 0x561700d15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:26.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa14f7fe700 10 client.28120 ms_handle_connect on v2:172.21.15.5:6838/3661239153 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa16ffff700 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700d12d50 0x561700d15140 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32b7ed78c7db5c4 server_cookie=771fcf2e36387161 in_seq=0 out_seq=0 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa14f7fe700 1 -- 172.21.15.5:0/3093105271 <== mds.0 v2:172.21.15.5:6838/3661239153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561700befb30 con 0x561700d12d50 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa14f7fe700 10 client.28120 handle_command_reply: tid=0 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.879+0000 7fa18a3ebb80 1 client.28120 shutdown 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 msgr2=0x561700d0a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa15809a130 tx=0x7fa1580a2610 comp rx=0 tx=0).stop 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 shutdown_connections 2023-09-23T05:13:26.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:6838/3661239153,v1:172.21.15.5:6839/3661239153] conn(0x561700d12d50 0x561700d15140 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561700caebd0 0x561700cb1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561700ca3400 0x561700d0a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa14f7fe700 0 client.28120 ms_handle_reset on v2:172.21.15.5:6838/3661239153 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 --2- 172.21.15.5:0/3093105271 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x561700ca3d20 0x561700d0a810 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 >> 172.21.15.5:0/3093105271 conn(0x561700c18260 msgr2=0x561700c9f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 shutdown_connections 2023-09-23T05:13:26.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 1 -- 172.21.15.5:0/3093105271 wait complete. 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa18a3ebb80 20 client.28120 trim_cache size 0 max 16384 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stdout: "0983c5c5-78b1-4320-8f7e-c19542831fa5": { 2023-09-23T05:13:26.890 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-09-23T05:13:26.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0983c5c5-78b1-4320-8f7e-c19542831fa5", 2023-09-23T05:13:26.891 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-09-23T05:13:26.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:26.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:26.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 msgr2=0x7fa15c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 0x7fa15c0a3c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa168009ec0 tx=0x7fa168004b40 comp rx=0 tx=0).stop 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 msgr2=0x7fa170199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.880+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e970 tx=0x7fa15800ec80 comp rx=0 tx=0).stop 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 shutdown_connections 2023-09-23T05:13:26.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa15c0a1790 0x7fa15c0a3c50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17019db60 0x7fa17019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1701010d0 0x7fa1701991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 --2- 172.21.15.5:0/1864198722 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa1701061f0 0x7fa170199700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 >> 172.21.15.5:0/1864198722 conn(0x7fa1700fc990 msgr2=0x7fa1700fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 shutdown_connections 2023-09-23T05:13:26.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:26.881+0000 7fa1769ff700 1 -- 172.21.15.5:0/1864198722 wait complete. 2023-09-23T05:13:26.898 INFO:tasks.cephfs.filesystem:scrub status for tag:0983c5c5-78b1-4320-8f7e-c19542831fa5 - {'path': '/', 'tag': '0983c5c5-78b1-4320-8f7e-c19542831fa5', 'options': 'recursive,force'} 2023-09-23T05:13:26.899 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-23T05:13:26.908 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-23T05:13:26.911 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-23T05:13:26.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T05:13:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 -- 172.21.15.5:0/3786055544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50101a60 msgr2=0x7fcb50105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 --2- 172.21.15.5:0/3786055544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50101a60 0x7fcb50105aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009990 tx=0x7fcb40009ca0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 -- 172.21.15.5:0/3786055544 shutdown_connections 2023-09-23T05:13:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 --2- 172.21.15.5:0/3786055544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb501061d0 0x7fcb5010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 --2- 172.21.15.5:0/3786055544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501010b0 0x7fcb50101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 --2- 172.21.15.5:0/3786055544 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb50101a60 0x7fcb50105aa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 -- 172.21.15.5:0/3786055544 >> 172.21.15.5:0/3786055544 conn(0x7fcb500fc930 msgr2=0x7fcb500fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:27.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 -- 172.21.15.5:0/3786055544 shutdown_connections 2023-09-23T05:13:27.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.137+0000 7fcb585ff700 1 -- 172.21.15.5:0/3786055544 wait complete. 2023-09-23T05:13:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.138+0000 7fcb585ff700 1 Processor -- start 2023-09-23T05:13:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.138+0000 7fcb585ff700 1 -- start start 2023-09-23T05:13:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501061d0 0x7fcb50199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 0x7fcb5019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019e5f0 con 0x7fcb5019db70 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019e760 con 0x7fcb501061d0 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb585ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019e920 con 0x7fcb501010b0 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb56b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 0x7fcb5019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48934/0 (socket says 172.21.15.5:48934) 2023-09-23T05:13:27.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 -- 172.21.15.5:0/225803471 learned_addr learned my addr 172.21.15.5:0/225803471 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb55b9a700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501061d0 0x7fcb50199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 -- 172.21.15.5:0/225803471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501061d0 msgr2=0x7fcb50199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501061d0 0x7fcb50199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 -- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 msgr2=0x7fcb5019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 0x7fcb5019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb5639b700 1 -- 172.21.15.5:0/225803471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400095f0 con 0x7fcb501010b0 2023-09-23T05:13:27.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.139+0000 7fcb56b9c700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 0x7fcb5019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb5639b700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9a0 tx=0x7fcb4c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c009790 con 0x7fcb501010b0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c010430 con 0x7fcb501010b0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb501a5e80 con 0x7fcb501010b0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c018680 con 0x7fcb501010b0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.140+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501a62b0 con 0x7fcb501010b0 2023-09-23T05:13:27.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.141+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb50067ca0 con 0x7fcb501010b0 2023-09-23T05:13:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.143+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fcb4c0105a0 con 0x7fcb501010b0 2023-09-23T05:13:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.145+0000 7fcb477fe700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 0x7fcb3c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.145+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fcb4c014070 con 0x7fcb501010b0 2023-09-23T05:13:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.145+0000 7fcb55b9a700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 0x7fcb3c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.145+0000 7fcb55b9a700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 0x7fcb3c0a3bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5019a7e0 tx=0x7fcb40005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:27.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.147+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fcb4c0966a0 con 0x7fcb501010b0 2023-09-23T05:13:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.358+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb50101be0 con 0x7fcb501010b0 2023-09-23T05:13:27.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.360+0000 7fcb477fe700 1 -- 172.21.15.5:0/225803471 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+25333 (secure 0 0 0) 0x7fcb4c099cc0 con 0x7fcb501010b0 2023-09-23T05:13:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:13:27.361 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":132,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T05:13:16.079643+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3587586220":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2853321891":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1767282859":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/734408633":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T05:13:27.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 msgr2=0x7fcb3c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 0x7fcb3c0a3bd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb5019a7e0 tx=0x7fcb40005fd0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 msgr2=0x7fcb501991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9a0 tx=0x7fcb4c00ecb0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 shutdown_connections 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fcb3c0a1710 0x7fcb3c0a3bd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb5019db70 0x7fcb5019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fcb501010b0 0x7fcb501991d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 --2- 172.21.15.5:0/225803471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb501061d0 0x7fcb50199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 >> 172.21.15.5:0/225803471 conn(0x7fcb500fc930 msgr2=0x7fcb500fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 shutdown_connections 2023-09-23T05:13:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.363+0000 7fcb585ff700 1 -- 172.21.15.5:0/225803471 wait complete. 2023-09-23T05:13:27.382 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T05:13:27.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T05:13:27.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 -- 172.21.15.5:0/3224234428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10141010d0 msgr2=0x7f10141014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 --2- 172.21.15.5:0/3224234428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10141010d0 0x7f10141014b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f10040098f0 tx=0x7f1004009c00 comp rx=0 tx=0).stop 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 -- 172.21.15.5:0/3224234428 shutdown_connections 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 --2- 172.21.15.5:0/3224234428 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141061f0 0x7f101410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 --2- 172.21.15.5:0/3224234428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10141010d0 0x7f10141014b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 --2- 172.21.15.5:0/3224234428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1014101a80 0x7f1014105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 -- 172.21.15.5:0/3224234428 >> 172.21.15.5:0/3224234428 conn(0x7f10140fc990 msgr2=0x7f10140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:27.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 -- 172.21.15.5:0/3224234428 shutdown_connections 2023-09-23T05:13:27.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.590+0000 7f1019bff700 1 -- 172.21.15.5:0/3224234428 wait complete. 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.591+0000 7f1019bff700 1 Processor -- start 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.591+0000 7f1019bff700 1 -- start start 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10141061f0 0x7f1014073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1014073ba0 0x7f10141a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1014075070 con 0x7f1014073ba0 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141a9f80 con 0x7f10141061f0 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1019bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141aa0c0 con 0x7f10141010d0 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48950/0 (socket says 172.21.15.5:48950) 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 -- 172.21.15.5:0/3198891907 learned_addr learned my addr 172.21.15.5:0/3198891907 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f1013fff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1014073ba0 0x7f10141a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f100bfff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10141061f0 0x7f1014073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 -- 172.21.15.5:0/3198891907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10141061f0 msgr2=0x7f1014073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10141061f0 0x7f1014073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 -- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1014073ba0 msgr2=0x7f10141a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1014073ba0 0x7f10141a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.592+0000 7f10137fe700 1 -- 172.21.15.5:0/3198891907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c009d00 con 0x7f10141010d0 2023-09-23T05:13:27.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f10137fe700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1004009810 tx=0x7f100400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100401c070 con 0x7f10141010d0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100400ba40 con 0x7f10141010d0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004009520 con 0x7f10141010d0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004004930 con 0x7f10141010d0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.593+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10141aa5d0 con 0x7f10141010d0 2023-09-23T05:13:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.594+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014067ca0 con 0x7f10141010d0 2023-09-23T05:13:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.596+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f1004020070 con 0x7f10141010d0 2023-09-23T05:13:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.597+0000 7f10117fa700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 0x7f10000a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.598+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f10040ce590 con 0x7f10141010d0 2023-09-23T05:13:27.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.598+0000 7f100bfff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 0x7f10000a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:27.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.598+0000 7f100bfff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 0x7f10000a3bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1014074910 tx=0x7f0ffc009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:27.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.599+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f1004097440 con 0x7f10141010d0 2023-09-23T05:13:27.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.812+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10141019f0 con 0x7f10141010d0 2023-09-23T05:13:27.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.813+0000 7f10117fa700 1 -- 172.21.15.5:0/3198891907 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1004027020 con 0x7f10141010d0 2023-09-23T05:13:27.813 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T05:13:27.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 msgr2=0x7f10000a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 0x7f10000a3bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1014074910 tx=0x7f0ffc009260 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 msgr2=0x7f1014073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1004009810 tx=0x7f100400fa00 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 shutdown_connections 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10000a1710 0x7f10000a3bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1014073ba0 0x7f10141a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f10141010d0 0x7f1014073120 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 --2- 172.21.15.5:0/3198891907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10141061f0 0x7f1014073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 >> 172.21.15.5:0/3198891907 conn(0x7f10140fc990 msgr2=0x7f10140fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:27.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 shutdown_connections 2023-09-23T05:13:27.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:27.815+0000 7f1019bff700 1 -- 172.21.15.5:0/3198891907 wait complete. 2023-09-23T05:13:27.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T05:13:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 -- 172.21.15.5:0/2061351520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 msgr2=0x7fa64410dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 --2- 172.21.15.5:0/2061351520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa64410dd70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009f90 tx=0x7fa63c00b330 comp rx=0 tx=0).stop 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 -- 172.21.15.5:0/2061351520 shutdown_connections 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 --2- 172.21.15.5:0/2061351520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa64410dd70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 --2- 172.21.15.5:0/2061351520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 0x7fa6441014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 --2- 172.21.15.5:0/2061351520 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa644101a80 0x7fa644105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 -- 172.21.15.5:0/2061351520 >> 172.21.15.5:0/2061351520 conn(0x7fa6440fc990 msgr2=0x7fa6440fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 -- 172.21.15.5:0/2061351520 shutdown_connections 2023-09-23T05:13:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.045+0000 7fa6495ff700 1 -- 172.21.15.5:0/2061351520 wait complete. 2023-09-23T05:13:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.046+0000 7fa6495ff700 1 Processor -- start 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.046+0000 7fa6495ff700 1 -- start start 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 0x7fa6441992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa6441997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419e6d0 con 0x7fa6441061f0 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419e840 con 0x7fa6441010d0 2023-09-23T05:13:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6495ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419ea00 con 0x7fa64419dc50 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6427fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa6441997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48968/0 (socket says 172.21.15.5:48968) 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 -- 172.21.15.5:0/295459881 learned_addr learned my addr 172.21.15.5:0/295459881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa642ffd700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 0x7fa6441992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 -- 172.21.15.5:0/295459881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 msgr2=0x7fa6441992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 0x7fa6441992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 -- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 msgr2=0x7fa6441997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa6441997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6437fe700 1 -- 172.21.15.5:0/295459881 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380095f0 con 0x7fa64419dc50 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.047+0000 7fa6427fc700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa6441997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa6437fe700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009f60 tx=0x7fa63c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00e040 con 0x7fa64419dc50 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63c004e20 con 0x7fa64419dc50 2023-09-23T05:13:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c009c40 con 0x7fa64419dc50 2023-09-23T05:13:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0052a0 con 0x7fa64419dc50 2023-09-23T05:13:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.048+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6441a61f0 con 0x7fa64419dc50 2023-09-23T05:13:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.049+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa644067ca0 con 0x7fa64419dc50 2023-09-23T05:13:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.051+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa63c01d470 con 0x7fa64419dc50 2023-09-23T05:13:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.052+0000 7fa62bfff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 0x7fa62c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.053+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fa63c0cf470 con 0x7fa64419dc50 2023-09-23T05:13:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.053+0000 7fa642ffd700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 0x7fa62c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.053+0000 7fa642ffd700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 0x7fa62c0a3bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa634009c60 tx=0x7fa634009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.056+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa63c098440 con 0x7fa64419dc50 2023-09-23T05:13:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.274+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa644101b20 con 0x7fa64419dc50 2023-09-23T05:13:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.275+0000 7fa62bfff700 1 -- 172.21.15.5:0/295459881 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa63c09ba60 con 0x7fa64419dc50 2023-09-23T05:13:28.275 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T05:13:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 msgr2=0x7fa62c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 0x7fa62c0a3bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa634009c60 tx=0x7fa634009260 comp rx=0 tx=0).stop 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 msgr2=0x7fa64419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009f60 tx=0x7fa63c004c80 comp rx=0 tx=0).stop 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 shutdown_connections 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa62c0a1710 0x7fa62c0a3bd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa64419dc50 0x7fa64419e0d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6441010d0 0x7fa6441992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 --2- 172.21.15.5:0/295459881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6441061f0 0x7fa6441997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 >> 172.21.15.5:0/295459881 conn(0x7fa6440fc990 msgr2=0x7fa6440fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 shutdown_connections 2023-09-23T05:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.277+0000 7fa6495ff700 1 -- 172.21.15.5:0/295459881 wait complete. 2023-09-23T05:13:28.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T05:13:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.503+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2941788645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc4101a80 msgr2=0x7fdfc4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.503+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2941788645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc4101a80 0x7fdfc4105ac0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009990 tx=0x7fdfb8009ca0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2941788645 shutdown_connections 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2941788645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2941788645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 0x7fdfc41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2941788645 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc4101a80 0x7fdfc4105ac0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2941788645 >> 172.21.15.5:0/2941788645 conn(0x7fdfc40fc990 msgr2=0x7fdfc40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2941788645 shutdown_connections 2023-09-23T05:13:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.504+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2941788645 wait complete. 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 Processor -- start 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 -- start start 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 0x7fdfc41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419e5c0 con 0x7fdfc41061f0 2023-09-23T05:13:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419e730 con 0x7fdfc41010d0 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.505+0000 7fdfc93ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc419e8f0 con 0x7fdfc419db40 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc27fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:48984/0 (socket says 172.21.15.5:48984) 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 -- 172.21.15.5:0/2912934262 learned_addr learned my addr 172.21.15.5:0/2912934262 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc2ffd700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 0x7fdfc41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 -- 172.21.15.5:0/2912934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 msgr2=0x7fdfc41991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 0x7fdfc41991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 -- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 msgr2=0x7fdfc41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 -- 172.21.15.5:0/2912934262 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009c40 con 0x7fdfc419db40 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc27fc700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfc37fe700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c370 tx=0x7fdfbc00c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.506+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc017070 con 0x7fdfc419db40 2023-09-23T05:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.507+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfbc00f040 con 0x7fdfc419db40 2023-09-23T05:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.507+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb80095f0 con 0x7fdfc419db40 2023-09-23T05:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.507+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00e050 con 0x7fdfc419db40 2023-09-23T05:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.507+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc41a6140 con 0x7fdfc419db40 2023-09-23T05:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.508+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc4067ca0 con 0x7fdfc419db40 2023-09-23T05:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.509+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fdfbc004bd0 con 0x7fdfc419db40 2023-09-23T05:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.511+0000 7fdfabfff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 0x7fdfac0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.511+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fdfbc0ccfe0 con 0x7fdfc419db40 2023-09-23T05:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.511+0000 7fdfc2ffd700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 0x7fdfac0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.512+0000 7fdfc2ffd700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 0x7fdfac0a3c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009c60 tx=0x7fdfb4009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.513+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fdfbc095e20 con 0x7fdfc419db40 2023-09-23T05:13:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:28 smithi005 ceph-mon[97638]: pgmap v858: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:13:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/225803471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:13:28.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:28 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3198891907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:13:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:28 smithi118 ceph-mon[134297]: pgmap v858: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:13:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/225803471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:13:28.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:28 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3198891907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:13:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.726+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfc4101b20 con 0x7fdfc419db40 2023-09-23T05:13:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.726+0000 7fdfabfff700 1 -- 172.21.15.5:0/2912934262 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfbc099440 con 0x7fdfc419db40 2023-09-23T05:13:28.727 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T05:13:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 msgr2=0x7fdfac0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 0x7fdfac0a3c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009c60 tx=0x7fdfb4009260 comp rx=0 tx=0).stop 2023-09-23T05:13:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 msgr2=0x7fdfc419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c370 tx=0x7fdfbc00c680 comp rx=0 tx=0).stop 2023-09-23T05:13:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 shutdown_connections 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fdfac0a1790 0x7fdfac0a3c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fdfc419db40 0x7fdfc419dfc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc41010d0 0x7fdfc41991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 --2- 172.21.15.5:0/2912934262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfc41061f0 0x7fdfc41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 >> 172.21.15.5:0/2912934262 conn(0x7fdfc40fc990 msgr2=0x7fdfc40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 shutdown_connections 2023-09-23T05:13:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.729+0000 7fdfc93ff700 1 -- 172.21.15.5:0/2912934262 wait complete. 2023-09-23T05:13:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:28 smithi069 ceph-mon[133931]: pgmap v858: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 92 B/s wr, 9 op/s 2023-09-23T05:13:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/225803471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:13:28.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:28 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3198891907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:13:28.747 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T05:13:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 -- 172.21.15.5:0/1495532843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c101a80 msgr2=0x7f444c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 --2- 172.21.15.5:0/1495532843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c101a80 0x7f444c105ac0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 -- 172.21.15.5:0/1495532843 shutdown_connections 2023-09-23T05:13:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 --2- 172.21.15.5:0/1495532843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 0x7f444c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 --2- 172.21.15.5:0/1495532843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 0x7f444c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 --2- 172.21.15.5:0/1495532843 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c101a80 0x7f444c105ac0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 -- 172.21.15.5:0/1495532843 >> 172.21.15.5:0/1495532843 conn(0x7f444c0fc990 msgr2=0x7f444c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 -- 172.21.15.5:0/1495532843 shutdown_connections 2023-09-23T05:13:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.954+0000 7f44509ff700 1 -- 172.21.15.5:0/1495532843 wait complete. 2023-09-23T05:13:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.955+0000 7f44509ff700 1 Processor -- start 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 -- start start 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 0x7f444c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 0x7f444c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19c3e0 con 0x7f444c1061f0 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19c550 con 0x7f444c1010d0 2023-09-23T05:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f44509ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c19c710 con 0x7f444c19b960 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f444ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f444a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 0x7f444c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f444ad9d700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49002/0 (socket says 172.21.15.5:49002) 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f444ad9d700 1 -- 172.21.15.5:0/283931319 learned_addr learned my addr 172.21.15.5:0/283931319 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.956+0000 7f4443fff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 0x7f444c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 -- 172.21.15.5:0/283931319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 msgr2=0x7f444c197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 0x7f444c197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 -- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 msgr2=0x7f444c1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 0x7f444c1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 -- 172.21.15.5:0/283931319 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009520 con 0x7f444c19b960 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f444ad9d700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f444400ef30 tx=0x7f444400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444018070 con 0x7f444c19b960 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444400f040 con 0x7f444c19b960 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44340095f0 con 0x7f444c19b960 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400cd00 con 0x7f444c19b960 2023-09-23T05:13:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.957+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c1a3f10 con 0x7f444c19b960 2023-09-23T05:13:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.958+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c067ca0 con 0x7f444c19b960 2023-09-23T05:13:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.960+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f4444003a00 con 0x7f444c19b960 2023-09-23T05:13:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.962+0000 7f44437fe700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 0x7f44380a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.962+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f44440cdbd0 con 0x7f444c19b960 2023-09-23T05:13:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.962+0000 7f444a59c700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 0x7f44380a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.963+0000 7f444a59c700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 0x7f44380a3bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f443c009df0 tx=0x7f443c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:28.965+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f4444096a80 con 0x7f444c19b960 2023-09-23T05:13:29.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.177+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f444c101b20 con 0x7f444c19b960 2023-09-23T05:13:29.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.178+0000 7f44437fe700 1 -- 172.21.15.5:0/283931319 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v132) v1 ==== 103+0+11 (secure 0 0 0) 0x7f444409a0a0 con 0x7f444c19b960 2023-09-23T05:13:29.178 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T05:13:29.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 msgr2=0x7f44380a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 0x7f44380a3bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f443c009df0 tx=0x7f443c009260 comp rx=0 tx=0).stop 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 msgr2=0x7f444c19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f444400ef30 tx=0x7f444400c3a0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 shutdown_connections 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f44380a1710 0x7f44380a3bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f444c19b960 0x7f444c19bde0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f444c1010d0 0x7f444c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 --2- 172.21.15.5:0/283931319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f444c1061f0 0x7f444c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 >> 172.21.15.5:0/283931319 conn(0x7f444c0fc990 msgr2=0x7f444c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 shutdown_connections 2023-09-23T05:13:29.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.180+0000 7f44509ff700 1 -- 172.21.15.5:0/283931319 wait complete. 2023-09-23T05:13:29.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-23T05:13:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 -- 172.21.15.5:0/4193385291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901010b0 msgr2=0x7f8a90101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 --2- 172.21.15.5:0/4193385291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901010b0 0x7f8a90101490 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8a840098f0 tx=0x7f8a84009c00 comp rx=0 tx=0).stop 2023-09-23T05:13:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 -- 172.21.15.5:0/4193385291 shutdown_connections 2023-09-23T05:13:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 --2- 172.21.15.5:0/4193385291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a901061d0 0x7f8a9010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 --2- 172.21.15.5:0/4193385291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901010b0 0x7f8a90101490 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 --2- 172.21.15.5:0/4193385291 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a90101a60 0x7f8a90105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 -- 172.21.15.5:0/4193385291 >> 172.21.15.5:0/4193385291 conn(0x7f8a900fc930 msgr2=0x7f8a900fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.403+0000 7f8a971ff700 1 -- 172.21.15.5:0/4193385291 shutdown_connections 2023-09-23T05:13:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.404+0000 7f8a971ff700 1 -- 172.21.15.5:0/4193385291 wait complete. 2023-09-23T05:13:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.405+0000 7f8a971ff700 1 Processor -- start 2023-09-23T05:13:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.405+0000 7f8a971ff700 1 -- start start 2023-09-23T05:13:29.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 0x7f8a901975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9019ba20 0x7f8a9019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019c4a0 con 0x7f8a901061d0 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019c610 con 0x7f8a9019ba20 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a971ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019c7d0 con 0x7f8a901010b0 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:49016/0 (socket says 172.21.15.5:49016) 2023-09-23T05:13:29.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 -- 172.21.15.5:0/3397984056 learned_addr learned my addr 172.21.15.5:0/3397984056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a8ffff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 0x7f8a901975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a9579c700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9019ba20 0x7f8a9019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 -- 172.21.15.5:0/3397984056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9019ba20 msgr2=0x7f8a9019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9019ba20 0x7f8a9019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 -- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 msgr2=0x7f8a901975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 0x7f8a901975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a94f9b700 1 -- 172.21.15.5:0/3397984056 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0095f0 con 0x7f8a901010b0 2023-09-23T05:13:29.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.406+0000 7f8a8ffff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 0x7f8a901975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a94f9b700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009810 tx=0x7f8a8400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8401c070 con 0x7f8a901010b0 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a84005040 con 0x7f8a901010b0 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a84009520 con 0x7f8a901010b0 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84004a10 con 0x7f8a901010b0 2023-09-23T05:13:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.407+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a901a3f60 con 0x7f8a901010b0 2023-09-23T05:13:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.408+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90067ca0 con 0x7f8a901010b0 2023-09-23T05:13:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.410+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f8a84020070 con 0x7f8a901010b0 2023-09-23T05:13:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.411+0000 7f8a8dffb700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 0x7f8a800a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:13:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.411+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f8a840ce710 con 0x7f8a901010b0 2023-09-23T05:13:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.412+0000 7f8a8ffff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 0x7f8a800a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:13:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.412+0000 7f8a8ffff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 0x7f8a800a3bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a90198690 tx=0x7f8a7c009310 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:13:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.413+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8a840975c0 con 0x7f8a901010b0 2023-09-23T05:13:29.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:29 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/295459881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:29 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2912934262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:29 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/283931319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:29 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/295459881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:29 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2912934262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:29 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/283931319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:13:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.689+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f8a901983d0 con 0x7f8a901010b0 2023-09-23T05:13:29.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.700+0000 7f8a8dffb700 1 -- 172.21.15.5:0/3397984056 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+734588 (secure 0 0 0) 0x7f8a8409abe0 con 0x7f8a901010b0 2023-09-23T05:13:29.710 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15594, 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2023-09-23T05:13:29.711 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2064728089 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-09-23T05:13:29.712 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2064728089 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/2064728089", 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2023-09-23T04:46:16.587279+0000", 2023-09-23T05:13:29.713 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540138322906710015, 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25577, 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540138322906710015, 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-09-23T05:13:29.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.718 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.719 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.722 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-09-23T05:13:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-09-23T05:13:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-09-23T05:13:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-09-23T05:13:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-09-23T05:13:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T05:13:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-09-23T05:13:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-09-23T05:13:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T05:13:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T05:13:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2023-09-23T05:13:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-09-23T05:13:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T05:13:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-09-23T05:13:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-09-23T05:13:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T05:13:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-09-23T05:13:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T05:13:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.816 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-09-23T05:13:29.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-09-23T05:13:29.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T05:13:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-09-23T05:13:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-09-23T05:13:29.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T05:13:29.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-09-23T05:13:29.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2023-09-23T05:13:29.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T05:13:29.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.841 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:29.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T05:13:29.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.858 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:29.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.861 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.862 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-09-23T05:13:29.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T05:13:29.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.869 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-09-23T05:13:29.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-09-23T05:13:29.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:29.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.887 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:29.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.892 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:29.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-09-23T05:13:29.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T05:13:29.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:29.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.901 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T05:13:29.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T05:13:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T05:13:29.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.923 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T05:13:29.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T05:13:29.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:29.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.934 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T05:13:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T05:13:29.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T05:13:29.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T05:13:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T05:13:29.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T05:13:29.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T05:13:29.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T05:13:29.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.970 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T05:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:29.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-09-23T05:13:29.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:29.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:29.994 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:29.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:29.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T05:13:29.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:30.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.016 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.022 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-09-23T05:13:30.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.033 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-09-23T05:13:30.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-09-23T05:13:30.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T05:13:30.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-09-23T05:13:30.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T05:13:30.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.041 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.043 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.046 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.057 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-09-23T05:13:30.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.066 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.067 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.075 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:30.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.086 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.091 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.101 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-09-23T05:13:30.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-09-23T05:13:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T05:13:30.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.110 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.113 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.126 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-09-23T05:13:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-09-23T05:13:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-09-23T05:13:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T05:13:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-09-23T05:13:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T05:13:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.234 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.243 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.247 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.250 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T05:13:30.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-09-23T05:13:30.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.259 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:30.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.263 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:30.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.272 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-09-23T05:13:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.286 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.287 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.291 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.298 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.300 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.304 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.308 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-09-23T05:13:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-09-23T05:13:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-09-23T05:13:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-09-23T05:13:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-09-23T05:13:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.371 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-09-23T05:13:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-09-23T05:13:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-09-23T05:13:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T05:13:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-09-23T05:13:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-09-23T05:13:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-09-23T05:13:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-09-23T05:13:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-09-23T05:13:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:30.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25621, 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540138322906710015, 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.494 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.499 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-09-23T05:13:30.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.505 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-09-23T05:13:30.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.514 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-09-23T05:13:30.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T05:13:30.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-09-23T05:13:30.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.526 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-09-23T05:13:30.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-09-23T05:13:30.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-09-23T05:13:30.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T05:13:30.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.533 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.537 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-09-23T05:13:30.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-09-23T05:13:30.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.550 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-09-23T05:13:30.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.558 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-09-23T05:13:30.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2023-09-23T05:13:30.565 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-09-23T05:13:30.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T05:13:30.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.573 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-09-23T05:13:30.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-09-23T05:13:30.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.582 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-09-23T05:13:30.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-09-23T05:13:30.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.593 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-09-23T05:13:30.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-09-23T05:13:30.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-09-23T05:13:30.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-09-23T05:13:30.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.613 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.616 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-09-23T05:13:30.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-09-23T05:13:30.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T05:13:30.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-09-23T05:13:30.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-09-23T05:13:30.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-09-23T05:13:30.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.653 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T05:13:30.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.670 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.671 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-09-23T05:13:30.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T05:13:30.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.679 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T05:13:30.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T05:13:30.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.688 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T05:13:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T05:13:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T05:13:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T05:13:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T05:13:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T05:13:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T05:13:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T05:13:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T05:13:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-09-23T05:13:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-09-23T05:13:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.794 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.798 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-09-23T05:13:30.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-09-23T05:13:30.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-09-23T05:13:30.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-09-23T05:13:30.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.816 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.819 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-09-23T05:13:30.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T05:13:30.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-09-23T05:13:30.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.843 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.847 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-09-23T05:13:30.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-09-23T05:13:30.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-09-23T05:13:30.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.858 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.862 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.867 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.872 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.877 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:30.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.886 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.890 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-09-23T05:13:30.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.897 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-09-23T05:13:30.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-09-23T05:13:30.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-09-23T05:13:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.929 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.931 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-09-23T05:13:30.934 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.936 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.940 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.958 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.960 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.967 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.969 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.974 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.975 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.979 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:30.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.984 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:30.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-09-23T05:13:30.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T05:13:30.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:30.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:30.993 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:30.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:30.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:30.998 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:30.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.003 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.010 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-09-23T05:13:31.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.015 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.020 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.029 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:31.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:31.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:31.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-09-23T05:13:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-09-23T05:13:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-09-23T05:13:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-09-23T05:13:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-09-23T05:13:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-09-23T05:13:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-09-23T05:13:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-09-23T05:13:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-09-23T05:13:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-09-23T05:13:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-09-23T05:13:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-09-23T05:13:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-09-23T05:13:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-09-23T05:13:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-09-23T05:13:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-09-23T05:13:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-09-23T05:13:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-09-23T05:13:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-09-23T05:13:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-09-23T05:13:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-09-23T05:13:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2023-09-23T05:13:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-09-23T05:13:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-09-23T05:13:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-09-23T05:13:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-09-23T05:13:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-09-23T05:13:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-23T05:13:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-23T05:13:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-09-23T05:13:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-23T05:13:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2023-09-23T05:13:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-09-23T05:13:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-23T05:13:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-09-23T05:13:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-09-23T05:13:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-09-23T05:13:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-09-23T05:13:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-09-23T05:13:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-23T05:13:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2023-09-23T05:13:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-09-23T05:13:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-09-23T05:13:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-23T05:13:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-09-23T05:13:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-09-23T05:13:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-23T05:13:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-09-23T05:13:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-09-23T05:13:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-09-23T05:13:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-23T05:13:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-09-23T05:13:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-09-23T05:13:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-09-23T05:13:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-23T05:13:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-09-23T05:13:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-09-23T05:13:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-09-23T05:13:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-23T05:13:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-23T05:13:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-23T05:13:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-23T05:13:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-23T05:13:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-23T05:13:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-23T05:13:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-23T05:13:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-09-23T05:13:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-23T05:13:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-09-23T05:13:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-09-23T05:13:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-09-23T05:13:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-09-23T05:13:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-09-23T05:13:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-09-23T05:13:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-23T05:13:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-09-23T05:13:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-09-23T05:13:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-09-23T05:13:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-09-23T05:13:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-09-23T05:13:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-09-23T05:13:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-23T05:13:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-09-23T05:13:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-09-23T05:13:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-09-23T05:13:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-09-23T05:13:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-09-23T05:13:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-09-23T05:13:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-09-23T05:13:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-09-23T05:13:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-09-23T05:13:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-09-23T05:13:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-09-23T05:13:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-23T05:13:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-09-23T05:13:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-09-23T05:13:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-09-23T05:13:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-09-23T05:13:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-09-23T05:13:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-09-23T05:13:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-09-23T05:13:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-09-23T05:13:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-09-23T05:13:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-09-23T05:13:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-09-23T05:13:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-09-23T05:13:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-23T05:13:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-09-23T05:13:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-09-23T05:13:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-09-23T05:13:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-09-23T05:13:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-09-23T05:13:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-09-23T05:13:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-09-23T05:13:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-09-23T05:13:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-09-23T05:13:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-09-23T05:13:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-09-23T05:13:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2023-09-23T05:13:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-09-23T05:13:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2023-09-23T05:13:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-09-23T05:13:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-09-23T05:13:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-09-23T05:13:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-09-23T05:13:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 92, 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2023-09-23T05:13:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4093205639 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2697986932 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3939262460 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 848098975 2023-09-23T05:13:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1766530097 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-09-23T05:13:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3533078788 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-09-23T05:13:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:32.043 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:30 smithi118 ceph-mon[134297]: pgmap v859: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.043 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:30 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3397984056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T05:13:32.044 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:29 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/295459881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:13:32.044 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:29 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2912934262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:13:32.044 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:29 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/283931319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:13:32.044 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:30 smithi069 ceph-mon[133931]: pgmap v859: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.044 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:30 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3397984056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 msgr2=0x7f8a800a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 0x7f8a800a3bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a90198690 tx=0x7f8a7c009310 comp rx=0 tx=0).stop 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 msgr2=0x7f8a90197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009810 tx=0x7f8a8400fa00 comp rx=0 tx=0).stop 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 shutdown_connections 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8a800a1710 0x7f8a800a3bd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9019ba20 0x7f8a9019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8a901010b0 0x7f8a90197080 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 --2- 172.21.15.5:0/3397984056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a901061d0 0x7f8a901975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:13:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 >> 172.21.15.5:0/3397984056 conn(0x7f8a900fc930 msgr2=0x7f8a900fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:13:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 shutdown_connections 2023-09-23T05:13:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:13:29.707+0000 7f8a971ff700 1 -- 172.21.15.5:0/3397984056 wait complete. 2023-09-23T05:13:32.052 DEBUG:tasks.check_counter:Getting stats from a 2023-09-23T05:13:32.052 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-09-23T05:13:32.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:30 smithi005 ceph-mon[97638]: pgmap v859: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:30 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3397984056' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-23T05:13:32.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:32 smithi005 ceph-mon[97638]: pgmap v860: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:32 smithi118 ceph-mon[134297]: pgmap v860: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:32 smithi069 ceph-mon[133931]: pgmap v860: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:13:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T05:13:34.244 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3682, 2023-09-23T05:13:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 3614, 2023-09-23T05:13:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 124638723, 2023-09-23T05:13:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1256423, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.607477681, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.175235926, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.301581726, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.102679636, 2023-09-23T05:13:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3613, 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.370616658, 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000102578 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2072576, 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 295744 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1537, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1896, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 117837318, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1264310, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 9, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 7, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.360582866, 2023-09-23T05:13:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.066768200, 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.229467508, 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.045933716, 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1894, 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.186110525, 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000098263 2023-09-23T05:13:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-23T05:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 788112 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1667, 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1669, 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 182246519, 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 518449, 2023-09-23T05:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.430249486, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.043487692, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.315516100, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.057938534, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1667, 2023-09-23T05:13:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.138787251, 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000083255 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-09-23T05:13:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 10177, 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.304857658, 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000029955 2023-09-23T05:13:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2023-09-23T05:13:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2023-09-23T05:13:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 13688, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 91, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 12673, 2023-09-23T05:13:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 924, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 13557, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 4, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 42, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2023-09-23T05:13:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2023-09-23T05:13:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2023-09-23T05:13:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T05:13:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2023-09-23T05:13:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2023-09-23T05:13:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:13:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2023-09-23T05:13:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 83225, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2023-09-23T05:13:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 123, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 8, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2023-09-23T05:13:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 425090896, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 425090322, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 83225 2023-09-23T05:13:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2023-09-23T05:13:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1507, 2023-09-23T05:13:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 13678, 2023-09-23T05:13:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 12171, 2023-09-23T05:13:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 96, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 108, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 12, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 13688, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 13974, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 286, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2023-09-23T05:13:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 69192, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 207152 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2023-09-23T05:13:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2023-09-23T05:13:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2023-09-23T05:13:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2023-09-23T05:13:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2023-09-23T05:13:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 3, 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2023-09-23T05:13:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 4794, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 305605, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 2025, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-09-23T05:13:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 195, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 13593103, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 99199, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 6287, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 6287, 2023-09-23T05:13:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6287, 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 24.250529221, 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.003857249 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6287, 2023-09-23T05:13:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 6288 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 6287, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 6287, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 3695, 2023-09-23T05:13:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 2590, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2023-09-23T05:13:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2023-09-23T05:13:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2023-09-23T05:13:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2023-09-23T05:13:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 132, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 45, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 11, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 11, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2023-09-23T05:13:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2023-09-23T05:13:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2023-09-23T05:13:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 6882, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 424070460, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 6882, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6882, 2023-09-23T05:13:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 424070460, 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 6287, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 420905524, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6287, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 420905524, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 6287, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 6287, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6287, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 6287, 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2023-09-23T05:13:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2023-09-23T05:13:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2023-09-23T05:13:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x56200297ca00": { 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:34.333 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:34.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:34 smithi005 ceph-mon[97638]: pgmap v861: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:34 smithi118 ceph-mon[134297]: pgmap v861: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:34.690 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-09-23T05:13:34.691 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-09-23T05:13:34.691 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-09-23T05:13:34.691 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:13:34.691 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:13:34.691 DEBUG:tasks.check_counter:Getting stats from d 2023-09-23T05:13:34.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-09-23T05:13:34.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:34 smithi069 ceph-mon[133931]: pgmap v861: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T05:13:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:35 smithi005 ceph-mon[97638]: pgmap v862: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:35.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:35 smithi118 ceph-mon[134297]: pgmap v862: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:35.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:35 smithi069 ceph-mon[133931]: pgmap v862: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:36.874 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 492, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 481, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1640288, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 290448, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-09-23T05:13:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.093481757, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.037189192, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.045968210, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.011393277, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 480, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.052962338, 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000110338 2023-09-23T05:13:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1640288, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 290448 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 368, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 294496, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.035414338, 2023-09-23T05:13:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.026170161, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003408387, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 365, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.036464333, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000099902 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 294496 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 2944, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001296211, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000069741, 2023-09-23T05:13:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.001184510, 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.001189148, 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000396382 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 149745, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-09-23T05:13:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 495, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1570782, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 495, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 495, 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1570782, 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:36.894 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:37.321 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-09-23T05:13:37.322 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-09-23T05:13:37.322 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-09-23T05:13:37.322 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-09-23T05:13:37.322 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-09-23T05:13:37.322 DEBUG:tasks.check_counter:Getting stats from g 2023-09-23T05:13:37.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-09-23T05:13:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T05:13:38.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:38 smithi005 ceph-mon[97638]: pgmap v863: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:38.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:38 smithi118 ceph-mon[134297]: pgmap v863: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:38.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:38 smithi069 ceph-mon[133931]: pgmap v863: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:39.618 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 591585, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 632523, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 119111961, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 471505716, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 20, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2023-09-23T05:13:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 45.404124369, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 18.759985391, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 27.617186830, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 4.190316206, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 632523, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 58.615180450, 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000092668 2023-09-23T05:13:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1137306, 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.203538376, 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000178 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 857424, 2023-09-23T05:13:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 289184 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 398341, 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 526039, 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 127788481, 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 598580453, 2023-09-23T05:13:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 22, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 46.568710986, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 27.528365218, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 21.695469493, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 4.068099020, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 526041, 2023-09-23T05:13:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 122.202817899, 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000232306 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 509251, 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.198144299, 2023-09-23T05:13:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000389 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2163680, 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 361696 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1919790, 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1950527, 2023-09-23T05:13:39.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 319031287, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 602489702, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 180, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 9, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 114.246359266, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 50.463672848, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 79.255648706, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 2.404580101, 2023-09-23T05:13:39.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1950520, 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 59.807245315, 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000030662 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3793408, 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.587174784, 2023-09-23T05:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000154 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 45776, 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 820736 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-09-23T05:13:39.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 113282, 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 94.720740259, 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000836149 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 35010, 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 8.799739613, 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000251349 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 179757, 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 155485, 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 155485, 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1179.702190956, 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.007587241 2023-09-23T05:13:39.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 24272, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 15, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 83, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 5, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 2, 2023-09-23T05:13:39.632 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 1057, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 387, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 612, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 58, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 504, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 2573, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 96, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 96, 2023-09-23T05:13:39.633 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 17, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 254681, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 170989, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 24258, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 127, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 919, 2023-09-23T05:13:39.634 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 30, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 1880, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 10, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 352, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 517, 2023-09-23T05:13:39.635 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 749, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 227408842, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 7, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 517, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:39.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 232, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 13, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 219, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 285, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 31, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 60543, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 88787, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:39.637 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 602, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 87992, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 19404, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 4407, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 84797 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 21, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 149, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 77, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 271, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 345, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:39.639 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 20432, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 20432, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2023-09-23T05:13:39.640 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 13, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 31, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 7, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.641 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 125269, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 50413, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 49161, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 76108, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 1252, 2023-09-23T05:13:39.642 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 227, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 99, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 97, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 130, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 7, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 2, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 351104920, 2023-09-23T05:13:39.643 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 854273934, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 0, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 60867, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 378.956595090, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.006225977 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 0 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.644 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 811, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 22608, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 21797, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 123, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 301, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 178, 2023-09-23T05:13:39.645 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 1057, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 44067, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 43010, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 96, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 22478, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 22382, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 81792, 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 207152 2023-09-23T05:13:39.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 260332, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 28834, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 179757, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 2354, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 58869, 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2023-09-23T05:13:39.647 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 21078, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 90.255638775, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.004281983 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 31219, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 227.419802338, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.007284660 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.648 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.649 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 30131, 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 113.435481131, 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.003764743 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:39.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.651 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:39.652 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 27, 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.008181582, 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000303021 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 119, 2023-09-23T05:13:39.654 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.726083524, 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.006101542 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 27, 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 14.791732582, 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.547841947 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 9, 2023-09-23T05:13:39.656 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.011291628, 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001254625 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 48128, 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 392.651101799, 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.008158475 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.657 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3800, 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 236.931551196, 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.062350408 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.658 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2023-09-23T05:13:39.659 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1, 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000965962, 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000965962 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 20, 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2.410557203, 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.120527860 2023-09-23T05:13:39.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 43, 2023-09-23T05:13:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1.155364693, 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.026868946 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 444, 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1.271790207, 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002864392 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.663 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 7, 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.021271762, 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.003038823 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.664 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 20432, 2023-09-23T05:13:39.665 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 98.611376574, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.004826320 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.666 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 5, 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 3, 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2023-09-23T05:13:39.667 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 184, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 92, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 1627, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 360, 2023-09-23T05:13:39.668 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 360, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:39.669 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:39.670 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:39.671 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:39.672 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 349652, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 461, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 440, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 5, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-09-23T05:13:39.673 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 195, 2023-09-23T05:13:39.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 3367023, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 29531, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:39.675 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 105401, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 854802220, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 37, 2023-09-23T05:13:39.676 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 105401, 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 105401, 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1079.156071649, 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.010238575 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2023-09-23T05:13:39.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 105401, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 106367 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 105401, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 1200, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 104201, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2023-09-23T05:13:39.678 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 0, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 278, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 652, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 61506, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 404, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2023-09-23T05:13:39.679 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 5, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 41676, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 530, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 448, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:39.680 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:39.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2023-09-23T05:13:39.682 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2023-09-23T05:13:39.683 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 132, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 50, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 12, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 12, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2023-09-23T05:13:39.684 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 633, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 33, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 202 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 1, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2023-09-23T05:13:39.685 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 2 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T05:13:39.686 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 5, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 20432, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:39.687 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2909712, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 308368277, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2909712, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 2909712, 2023-09-23T05:13:39.688 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 308368277, 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:39.689 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:39.690 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 105401, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 857031677, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 105401, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 857031677, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2023-09-23T05:13:39.692 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 105401, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 105401, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 105401, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 105401, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.693 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:39.694 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 20432, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 2229784, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 20432, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 639, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2229784, 2023-09-23T05:13:39.695 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55cab15d8640": { 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:39.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 125269, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 850079630, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 125269, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 60867, 2023-09-23T05:13:39.697 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 850079630, 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:39.698 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:39.699 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:39 smithi118 ceph-mon[134297]: pgmap v864: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:39.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:39 smithi005 ceph-mon[97638]: pgmap v864: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:39.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:39 smithi069 ceph-mon[133931]: pgmap v864: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:40.050 INFO:tasks.check_counter:Daemon mds.g mds.exported=30 2023-09-23T05:13:40.050 INFO:tasks.check_counter:Daemon mds.g mds.imported=10 2023-09-23T05:13:40.050 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=7 2023-09-23T05:13:40.050 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=27 2023-09-23T05:13:40.050 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=20 2023-09-23T05:13:40.050 DEBUG:tasks.check_counter:Getting stats from j 2023-09-23T05:13:40.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-09-23T05:13:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.a/config 2023-09-23T05:13:42.284 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-09-23T05:13:42.284 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:42.284 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 2944, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.004109536, 2023-09-23T05:13:42.285 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000081600, 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.001245337, 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.001421174, 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000473724 2023-09-23T05:13:42.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2023-09-23T05:13:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 495, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 484, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1641104, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 292320, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-09-23T05:13:42.288 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.092625985, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.037670660, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.048450623, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.011395521, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 483, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.055544631, 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000114999 2023-09-23T05:13:42.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1641104, 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 292320 2023-09-23T05:13:42.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 368, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 217744, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:42.291 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.034956138, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.026784432, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002581462, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 365, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.053815459, 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000147439 2023-09-23T05:13:42.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 217744 2023-09-23T05:13:42.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:42.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:42.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:42.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148977, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-09-23T05:13:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 497, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1571145, 2023-09-23T05:13:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 497, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 497, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1571145, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-09-23T05:13:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-09-23T05:13:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-09-23T05:13:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-09-23T05:13:42.304 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-09-23T05:13:42.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:42 smithi005 ceph-mon[97638]: pgmap v865: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:42.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:42 smithi118 ceph-mon[134297]: pgmap v865: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:42.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:42 smithi069 ceph-mon[133931]: pgmap v865: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:42.885 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-09-23T05:13:42.886 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-09-23T05:13:42.886 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-09-23T05:13:42.886 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-09-23T05:13:42.886 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-09-23T05:13:42.886 DEBUG:tasks.check_counter:Getting stats from b 2023-09-23T05:13:42.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-09-23T05:13:43.526 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T05:13:44.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:44 smithi069 ceph-mon[133931]: pgmap v866: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:44.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:44 smithi005 ceph-mon[97638]: pgmap v866: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:44.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:44 smithi118 ceph-mon[134297]: pgmap v866: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:44.783 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-23T05:13:44.783 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:44.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 172066, 2023-09-23T05:13:44.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 144992, 2023-09-23T05:13:44.783 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 58224109, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 227591543, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 16, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 9.277491573, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 3.312261695, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 5.599789110, 2023-09-23T05:13:44.784 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 1.364944202, 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 144993, 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 6.754118823, 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000046582 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:44.785 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 241275, 2023-09-23T05:13:44.786 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.032217444, 2023-09-23T05:13:44.786 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000133 2023-09-23T05:13:44.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 6256, 2023-09-23T05:13:44.786 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 16720 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 87844, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 77726, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 31498704, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 269791347, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 33, 2023-09-23T05:13:44.787 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 5.296652088, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 1.697950599, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 2.438339956, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 1.464373146, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 77725, 2023-09-23T05:13:44.788 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 6.527159538, 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000083977 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 97147, 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.013887497, 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000142 2023-09-23T05:13:44.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1984, 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 128896 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 280309, 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 395953, 2023-09-23T05:13:44.790 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 94837627, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 294557089, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 9, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 14.436790127, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 6.578453788, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 9.688965444, 2023-09-23T05:13:44.791 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.633795205, 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 395948, 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 32.301268140, 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000081579 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:44.792 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 357255, 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.084797040, 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000237 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2097104, 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 991600 2023-09-23T05:13:44.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2023-09-23T05:13:44.794 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 64340, 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 66.266380361, 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001029940 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:44.795 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 43650, 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 4.969622591, 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000113851 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.796 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 137163, 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 125451, 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 125451, 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 552.664600246, 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.004405422 2023-09-23T05:13:44.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 11712, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 2, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 20, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 1, 2023-09-23T05:13:44.798 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 1, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 1014, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 142, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 715, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 157, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 810, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 915, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 126, 2023-09-23T05:13:44.799 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 126, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 39, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 237081, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 133286, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 11687, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 66, 2023-09-23T05:13:44.800 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 3601, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 1000, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 1, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 20, 2023-09-23T05:13:44.801 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 789, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 16, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 829, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 150, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 749, 2023-09-23T05:13:44.802 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 227408842, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 7, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 150, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:44.803 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 150, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 10, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 25134, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 94293, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:44.804 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 711, 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 51771, 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 18693, 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 3849, 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 96553 2023-09-23T05:13:44.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 4, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 205, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 123, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 226, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 172, 2023-09-23T05:13:44.806 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 22871, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 22871, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2023-09-23T05:13:44.807 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 21, 2023-09-23T05:13:44.808 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 2, 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2023-09-23T05:13:44.809 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 117793, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 27145, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 24581, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 93212, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 2564, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 163, 2023-09-23T05:13:44.810 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 35, 2023-09-23T05:13:44.811 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 32, 2023-09-23T05:13:44.811 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 131, 2023-09-23T05:13:44.811 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 8, 2023-09-23T05:13:44.811 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 3, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 46688004, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 498942523, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 35233, 2023-09-23T05:13:44.812 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 295.985169973, 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.008400793 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 722, 2023-09-23T05:13:44.813 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 24195, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 23473, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 142, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 281, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 139, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 1014, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 39439, 2023-09-23T05:13:44.814 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 38425, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 126, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 23669, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 23543, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 71300, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 198960 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2023-09-23T05:13:44.815 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 206810, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 3306, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 137163, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 3278, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 19414, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 22850, 2023-09-23T05:13:44.816 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 79.125135460, 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003462806 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 5082, 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 111.569194563, 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.021953796 2023-09-23T05:13:44.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2023-09-23T05:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.820 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2023-09-23T05:13:44.820 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 31376, 2023-09-23T05:13:44.820 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 68.113859076, 2023-09-23T05:13:44.820 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002170890 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2023-09-23T05:13:44.821 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000185312, 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000185312 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.823 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.825 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.826 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 36133, 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 212.583292191, 2023-09-23T05:13:44.827 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005883355 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6258, 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 19.802548748, 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.003164357 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.828 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.829 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.830 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2023-09-23T05:13:44.831 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 32, 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.938817052, 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.029338032 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 848, 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.899050445, 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001060200 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.833 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.834 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 22871, 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 59.632517399, 2023-09-23T05:13:44.835 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002607341 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.836 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 4, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 2, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2023-09-23T05:13:44.837 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 17278, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 8639, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 1632, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:44.838 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:44.839 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:44.840 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:44.841 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:44.842 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 322559, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 554, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 440, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 5, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:44.843 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 195, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:44.844 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 2976843, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 27256, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.845 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 83585, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 498786479, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 28, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 83585, 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2023-09-23T05:13:44.846 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 83585, 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 974.116469640, 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.011654201 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 83585, 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 84089 2023-09-23T05:13:44.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 83585, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 938, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 82647, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 21, 2023-09-23T05:13:44.848 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 784, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 36018, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 460, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 1, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 45889, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2023-09-23T05:13:44.849 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 150, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 42, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2023-09-23T05:13:44.850 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2023-09-23T05:13:44.851 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2023-09-23T05:13:44.852 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 132, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2023-09-23T05:13:44.853 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 45, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 12, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 496, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 6, 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 222 2023-09-23T05:13:44.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2023-09-23T05:13:44.855 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 5, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 22871, 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:44.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 540215, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 139126443, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:44.857 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 540215, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 540215, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 139126443, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.858 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:44.859 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 83585, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 501282278, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 83585, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 501282278, 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:44.860 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:44.861 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 83585, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 83585, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 83585, 2023-09-23T05:13:44.862 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 83585, 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:44.863 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 22871, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 2495891, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 22871, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:44.864 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 785, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2495891, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:44.865 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x55c9533f2640": { 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 117793, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 494748219, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:44.866 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 117793, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 35233, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 494748219, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:44.867 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.868 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:44.868 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-23T05:13:45.170 INFO:tasks.check_counter:Daemon mds.b mds.exported=20 2023-09-23T05:13:45.170 INFO:tasks.check_counter:Daemon mds.b mds.imported=16 2023-09-23T05:13:45.170 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2023-09-23T05:13:45.170 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:13:45.170 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:13:45.170 DEBUG:tasks.check_counter:Getting stats from e 2023-09-23T05:13:45.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-09-23T05:13:45.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:45 smithi069 ceph-mon[133931]: pgmap v867: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:45.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:45 smithi005 ceph-mon[97638]: pgmap v867: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:45.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:45 smithi118 ceph-mon[134297]: pgmap v867: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:45.752 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T05:13:46.959 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-23T05:13:46.959 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:46.959 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3526, 2023-09-23T05:13:46.959 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 3457, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 116637578, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1209622, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.391741081, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.117595277, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.162772458, 2023-09-23T05:13:46.960 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.094359633, 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3457, 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.182374905, 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000052755 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.961 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2078320, 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 300880 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1257, 2023-09-23T05:13:46.962 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1623, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 142933373, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1182079, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 7, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.175165784, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.036942152, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.095577281, 2023-09-23T05:13:46.963 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.032019542, 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1620, 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.079711846, 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000049204 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.964 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 791712 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2308, 2023-09-23T05:13:46.965 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2310, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 197725403, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 717795, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.276524460, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.040869096, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.158331131, 2023-09-23T05:13:46.966 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.062553660, 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2308, 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.129590930, 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000056148 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.967 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2023-09-23T05:13:46.968 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 10817, 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.257138957, 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000023771 2023-09-23T05:13:46.969 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.970 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.971 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2023-09-23T05:13:46.972 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 21953, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 2365, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 3172, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 16416, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 18574, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 194, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2023-09-23T05:13:46.973 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 16, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:46.974 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:46.975 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:46.976 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2023-09-23T05:13:46.977 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.978 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:46.979 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:46.980 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2023-09-23T05:13:46.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 111833, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2023-09-23T05:13:46.982 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 130, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 11, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 42528701, 2023-09-23T05:13:46.983 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 457599603, 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 457599029, 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 111833 2023-09-23T05:13:46.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 994, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 22188, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 21194, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 63, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 68, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 5, 2023-09-23T05:13:46.985 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 21953, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 22282, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 329, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 73880, 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 207152 2023-09-23T05:13:46.986 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2023-09-23T05:13:46.987 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.989 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.990 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.991 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.992 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.993 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.994 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.995 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.996 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2023-09-23T05:13:46.997 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:46.999 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.001 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.002 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.003 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2023-09-23T05:13:47.004 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.006 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 3, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 2, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2023-09-23T05:13:47.007 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 5331, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:47.008 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:47.009 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:47.010 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:47.011 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:47.012 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 282593, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 1285, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:47.013 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 195, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:47.014 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 17968277, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 115930, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-09-23T05:13:47.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 6485, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 21, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 3, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 6485, 2023-09-23T05:13:47.016 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6485, 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 16.501634950, 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002544585 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6485, 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 6486 2023-09-23T05:13:47.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 6485, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 6485, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 3742, 2023-09-23T05:13:47.018 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 2741, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2023-09-23T05:13:47.019 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 1, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:47.020 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2023-09-23T05:13:47.021 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2023-09-23T05:13:47.022 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 132, 2023-09-23T05:13:47.023 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 45, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 11, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 12, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 1, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2023-09-23T05:13:47.024 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2023-09-23T05:13:47.026 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:47.027 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 7087, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 456625401, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 7087, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 7087, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 456625401, 2023-09-23T05:13:47.028 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:47.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6485, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 453425270, 2023-09-23T05:13:47.030 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6485, 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 453425270, 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.031 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:47.032 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6485, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 6485, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6485, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 6485, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:47.034 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.035 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x562ba9da2960": { 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:47.036 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:47.037 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:47.038 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:47.038 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:47.038 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.038 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:47.038 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-23T05:13:47.396 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-23T05:13:47.396 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-23T05:13:47.396 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-09-23T05:13:47.396 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:13:47.396 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:13:47.397 DEBUG:tasks.check_counter:Getting stats from h 2023-09-23T05:13:47.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-09-23T05:13:47.952 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T05:13:48.457 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:48 smithi069 ceph-mon[133931]: pgmap v868: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:48.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:48 smithi005 ceph-mon[97638]: pgmap v868: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:48.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:48 smithi118 ceph-mon[134297]: pgmap v868: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:49.169 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-23T05:13:49.169 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:49.169 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4119, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4052, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 237993588, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1394381, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.480728094, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.133987110, 2023-09-23T05:13:49.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.212682220, 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.112622972, 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4051, 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.202106695, 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000049890 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:49.171 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2075264, 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 299968 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:49.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2014, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2378, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 330506318, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1418298, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 7, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.334006112, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.047686100, 2023-09-23T05:13:49.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.217705705, 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.052015940, 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2376, 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.121011780, 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000050930 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:49.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 793744 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:49.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1595, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1597, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 285619804, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 496052, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.263582454, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.026368928, 2023-09-23T05:13:49.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.184975673, 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.041228438, 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1595, 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.057941870, 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000036327 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:49.177 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-09-23T05:13:49.178 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 11551, 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.318416490, 2023-09-23T05:13:49.179 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000027566 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.180 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.181 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2023-09-23T05:13:49.182 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 18499, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 625, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 11658, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 6216, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 17605, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 2847, 2023-09-23T05:13:49.183 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 13, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:49.184 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2023-09-23T05:13:49.185 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 749, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 227408842, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 7, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:49.186 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T05:13:49.187 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2023-09-23T05:13:49.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2023-09-23T05:13:49.189 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:49.190 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:49.191 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2023-09-23T05:13:49.192 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 125269, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 130, 2023-09-23T05:13:49.193 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 12, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 351104920, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 854273934, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 854273798, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.194 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 125269 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 772, 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 22425, 2023-09-23T05:13:49.195 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 21653, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 65, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 175, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 110, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 18499, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 23140, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 4641, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2023-09-23T05:13:49.196 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 74796, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 207152 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2023-09-23T05:13:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.198 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.199 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2023-09-23T05:13:49.200 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2023-09-23T05:13:49.202 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.204 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.205 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.206 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2023-09-23T05:13:49.207 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.209 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.210 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.211 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:49.212 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.214 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:49.216 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2023-09-23T05:13:49.217 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 5, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 3, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 3, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 4411, 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:49.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:49.219 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:49.222 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 275597, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 1122, 2023-09-23T05:13:49.223 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:49.224 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 195, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 15204035, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 98286, 2023-09-23T05:13:49.225 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2023-09-23T05:13:49.226 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 7122, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 26, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 4, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 7122, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 7122, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 20.158553904, 2023-09-23T05:13:49.227 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.002830462 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 7122, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 7123 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 7122, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 7122, 2023-09-23T05:13:49.228 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 4137, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 2982, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2023-09-23T05:13:49.229 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:49.230 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 2, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2023-09-23T05:13:49.231 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2023-09-23T05:13:49.232 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2023-09-23T05:13:49.233 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 132, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 45, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 11, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 11, 2023-09-23T05:13:49.234 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2023-09-23T05:13:49.235 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2023-09-23T05:13:49.236 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:49.237 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 7724, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 853391433, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 7724, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:49.238 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 7724, 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 853391433, 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:49.240 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 7122, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 850079640, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 7122, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 850079640, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.241 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:49.242 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 7122, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 7122, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 7122, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 7122, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.243 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:49.244 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:49.245 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x557eeb7f8960": { 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:49.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:49.247 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:49.248 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-23T05:13:49.549 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:49 smithi069 ceph-mon[133931]: pgmap v869: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:49.595 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-09-23T05:13:49.595 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-09-23T05:13:49.595 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2023-09-23T05:13:49.596 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:13:49.596 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:13:49.596 DEBUG:tasks.check_counter:Getting stats from k 2023-09-23T05:13:49.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-09-23T05:13:49.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:49 smithi005 ceph-mon[97638]: pgmap v869: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:49.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:49 smithi118 ceph-mon[134297]: pgmap v869: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:50.148 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.b/config 2023-09-23T05:13:50.541 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:50 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:13:50.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:50 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:13:50.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:50 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-23T05:13:51.367 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2023-09-23T05:13:51.367 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3105, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 3038, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 44795138, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1080739, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.340992756, 2023-09-23T05:13:51.368 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.112230920, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.119802638, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.093232334, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3037, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.167769977, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000055242 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.369 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2076464, 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 302928 2023-09-23T05:13:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:13:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1765, 2023-09-23T05:13:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2134, 2023-09-23T05:13:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 130489956, 2023-09-23T05:13:51.371 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1342185, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 9, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 7, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.209005786, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.039469054, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.109692146, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.046341273, 2023-09-23T05:13:51.372 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2131, 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.109572936, 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000051418 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.373 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 793824 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1918, 2023-09-23T05:13:51.374 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1918, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 149463959, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 595703, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.223540556, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.042021724, 2023-09-23T05:13:51.375 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.117201024, 2023-09-23T05:13:51.376 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.051291000, 2023-09-23T05:13:51.376 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:13:51.376 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1917, 2023-09-23T05:13:51.376 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.096544146, 2023-09-23T05:13:51.376 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000050362 2023-09-23T05:13:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:13:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.377 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.378 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-09-23T05:13:51.378 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2023-09-23T05:13:51.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.379 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2023-09-23T05:13:51.379 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2023-09-23T05:13:51.379 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2023-09-23T05:13:51.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.380 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2023-09-23T05:13:51.380 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:51.380 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:51.380 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 10371, 2023-09-23T05:13:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.278651451, 2023-09-23T05:13:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000026868 2023-09-23T05:13:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.382 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2023-09-23T05:13:51.382 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2023-09-23T05:13:51.382 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2023-09-23T05:13:51.382 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.383 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.383 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.383 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.384 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2023-09-23T05:13:51.384 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2023-09-23T05:13:51.384 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2023-09-23T05:13:51.384 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2023-09-23T05:13:51.384 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2023-09-23T05:13:51.385 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 16426, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 29, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 13624, 2023-09-23T05:13:51.386 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 2773, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 16384, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 2720, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 24, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2023-09-23T05:13:51.387 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2023-09-23T05:13:51.388 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 0, 2023-09-23T05:13:51.389 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:13:51.390 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T05:13:51.391 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2023-09-23T05:13:51.392 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:13:51.393 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:13:51.394 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.395 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 100628, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2023-09-23T05:13:51.396 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 130, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 10, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2023-09-23T05:13:51.397 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 21761723, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 325119915, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 325119341, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.398 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 100628 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 1258, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 19077, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 17819, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 64, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 69, 2023-09-23T05:13:51.399 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 5, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 16426, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 19219, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 2793, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 74076, 2023-09-23T05:13:51.400 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 207152 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2023-09-23T05:13:51.401 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.402 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.403 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.404 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2023-09-23T05:13:51.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.406 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2023-09-23T05:13:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.409 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2023-09-23T05:13:51.409 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.409 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.409 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.409 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.410 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2023-09-23T05:13:51.410 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.410 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.410 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.411 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2023-09-23T05:13:51.411 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.411 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.412 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.412 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2023-09-23T05:13:51.412 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.413 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.413 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.413 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2023-09-23T05:13:51.413 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.414 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.414 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.414 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2023-09-23T05:13:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.416 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2023-09-23T05:13:51.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.416 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.417 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.417 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2023-09-23T05:13:51.417 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.417 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.419 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2023-09-23T05:13:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.421 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.422 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.423 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2023-09-23T05:13:51.424 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.426 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 3, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 3, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2023-09-23T05:13:51.427 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 4416, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2023-09-23T05:13:51.428 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:13:51.429 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2023-09-23T05:13:51.430 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:13:51.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:13:51.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 289019, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 1475, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2023-09-23T05:13:51.433 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 10016, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 195, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:13:51.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 14644769, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 98409, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2023-09-23T05:13:51.435 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 6178, 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 21, 2023-09-23T05:13:51.436 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 3, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 6178, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6178, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 11.529744755, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001866258 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2023-09-23T05:13:51.437 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 6178, 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 6179 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 6178, 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 6178, 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2023-09-23T05:13:51.438 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 3554, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 2622, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2023-09-23T05:13:51.439 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 1, 2023-09-23T05:13:51.440 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:13:51.441 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2023-09-23T05:13:51.442 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2023-09-23T05:13:51.443 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 132, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 45, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 11, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 11, 2023-09-23T05:13:51.444 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2023-09-23T05:13:51.445 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.446 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.447 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 6784, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 324105162, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:51.448 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 6784, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6784, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 324105162, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.449 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:51.450 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6178, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 320954421, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6178, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 320954421, 2023-09-23T05:13:51.451 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:51.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 6178, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 6178, 2023-09-23T05:13:51.453 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 6178, 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 6178, 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:51.455 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.456 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x561599f2a960": { 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2023-09-23T05:13:51.457 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2023-09-23T05:13:51.458 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2023-09-23T05:13:51.459 INFO:teuthology.orchestra.run.smithi069.stdout:} 2023-09-23T05:13:51.707 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-09-23T05:13:51.708 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-09-23T05:13:51.708 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-09-23T05:13:51.708 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:13:51.708 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:13:51.708 DEBUG:tasks.check_counter:Getting stats from c 2023-09-23T05:13:51.709 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-09-23T05:13:52.237 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:52 smithi118 ceph-mon[134297]: pgmap v870: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:52.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:52 smithi005 ceph-mon[97638]: pgmap v870: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:52.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:52 smithi069 ceph-mon[133931]: pgmap v870: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:52.704 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T05:13:53.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:53 smithi005 ceph-mon[97638]: pgmap v871: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:53.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:53 smithi118 ceph-mon[134297]: pgmap v871: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:53.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:53 smithi069 ceph-mon[133931]: pgmap v871: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:55.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:55 smithi005 ceph-mon[97638]: pgmap v872: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:55.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:55 smithi118 ceph-mon[134297]: pgmap v872: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:55.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:55 smithi069 ceph-mon[133931]: pgmap v872: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:13:57.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:57 smithi005 ceph-mon[97638]: pgmap v873: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:57.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:57 smithi118 ceph-mon[134297]: pgmap v873: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:57.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:57 smithi069 ceph-mon[133931]: pgmap v873: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-23T05:13:59.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:13:59 smithi005 ceph-mon[97638]: pgmap v874: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:59.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:13:59 smithi118 ceph-mon[134297]: pgmap v874: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:13:59.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:13:59 smithi069 ceph-mon[133931]: pgmap v874: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-23T05:14:01.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:01.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:01.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:14:01.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:14:01.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:01.913 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:01 smithi118 ceph-mon[134297]: pgmap v875: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:14:01.951 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-09-23T05:14:01.951 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:14:01.952 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 52614, 2023-09-23T05:14:01.952 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 51940, 2023-09-23T05:14:01.952 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 19548099, 2023-09-23T05:14:01.952 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 160957248, 2023-09-23T05:14:01.953 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 17, 2023-09-23T05:14:01.953 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 4, 2023-09-23T05:14:01.953 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 3.888682853, 2023-09-23T05:14:01.953 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 1.499941220, 2023-09-23T05:14:01.953 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 1.529598873, 2023-09-23T05:14:01.954 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.985760681, 2023-09-23T05:14:01.954 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:01.954 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 51938, 2023-09-23T05:14:01.954 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 10.906038890, 2023-09-23T05:14:01.954 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000209981 2023-09-23T05:14:01.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.955 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:01.955 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 49310, 2023-09-23T05:14:01.955 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.009369200, 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000190 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 680512 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.956 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 46795, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 53286, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 15482644, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 121712764, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 10, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 7, 2023-09-23T05:14:01.957 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 3.314473404, 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 1.129301949, 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 1.453963706, 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.818305414, 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 53286, 2023-09-23T05:14:01.958 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 11.410705784, 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000214140 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 50765, 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.010294276, 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000202 2023-09-23T05:14:01.959 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 2099936, 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 334704 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 5160502, 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 5245146, 2023-09-23T05:14:01.960 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 898354263, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 1101697419, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 34, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 6, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 178.710367071, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 76.282509013, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 125.662781111, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.659097489, 2023-09-23T05:14:01.961 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 5245143, 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 117.419625770, 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000022386 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 10121366, 2023-09-23T05:14:01.962 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.907816795, 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000089 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 127536 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2023-09-23T05:14:01.963 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 49320, 2023-09-23T05:14:01.964 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 47.305222585, 2023-09-23T05:14:01.965 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000959148 2023-09-23T05:14:01.965 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:01.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.965 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 33149, 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 3.643877631, 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000109924 2023-09-23T05:14:01.966 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 123546, 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 118937, 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2023-09-23T05:14:01.967 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 118937, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 368.809219003, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003100878 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 4609, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 3, 2023-09-23T05:14:01.968 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2023-09-23T05:14:01.969 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 38, 2023-09-23T05:14:01.969 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 3, 2023-09-23T05:14:01.969 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2023-09-23T05:14:01.969 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 1400, 2023-09-23T05:14:01.969 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 223, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 1117, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 60, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 1081, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 213, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 97, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 97, 2023-09-23T05:14:01.970 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 32, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 209871, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 107348, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 4609, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 36, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 1352, 2023-09-23T05:14:01.971 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 2, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 94, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 10, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 538, 2023-09-23T05:14:01.972 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 90, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 749, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 227408842, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 7, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 90, 2023-09-23T05:14:01.973 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:14:01.974 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 90, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 55, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 18768, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 74355, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 325, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 38668, 2023-09-23T05:14:01.975 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 11297, 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 2338, 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 98394 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 206, 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 118, 2023-09-23T05:14:01.976 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 67, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 138, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 251, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 17768, 2023-09-23T05:14:01.977 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 17768, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2023-09-23T05:14:01.978 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2023-09-23T05:14:01.979 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:14:01.979 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 5, 2023-09-23T05:14:01.979 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2023-09-23T05:14:01.979 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 3, 2023-09-23T05:14:01.980 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2023-09-23T05:14:01.980 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2023-09-23T05:14:01.980 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.980 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2023-09-23T05:14:01.981 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2023-09-23T05:14:01.981 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 100628, 2023-09-23T05:14:01.981 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 14339, 2023-09-23T05:14:01.981 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 12291, 2023-09-23T05:14:01.981 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 88337, 2023-09-23T05:14:01.982 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2023-09-23T05:14:01.982 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 2048, 2023-09-23T05:14:01.982 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 144, 2023-09-23T05:14:01.982 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 16, 2023-09-23T05:14:01.983 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 14, 2023-09-23T05:14:01.983 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 130, 2023-09-23T05:14:01.983 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 8, 2023-09-23T05:14:01.983 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2023-09-23T05:14:01.983 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 2, 2023-09-23T05:14:01.984 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 21761723, 2023-09-23T05:14:01.984 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 325119915, 2023-09-23T05:14:01.984 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 0, 2023-09-23T05:14:01.984 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2023-09-23T05:14:01.984 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 25871, 2023-09-23T05:14:01.985 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 192.124839453, 2023-09-23T05:14:01.985 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.007426262 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 0 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 1264, 2023-09-23T05:14:01.986 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 19180, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 17916, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 117, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 190, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 73, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 1400, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 34042, 2023-09-23T05:14:01.987 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 32642, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 97, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 20420, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 20323, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 71628, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 198960 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2023-09-23T05:14:01.988 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 189502, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 25, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 123546, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 1375, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 12614, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 17774, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 70.094810342, 2023-09-23T05:14:01.989 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003943671 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 941, 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 5.352339717, 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.005687927 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.990 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2023-09-23T05:14:01.991 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.992 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.992 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.992 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.992 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2023-09-23T05:14:01.992 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 30145, 2023-09-23T05:14:01.993 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 69.844105279, 2023-09-23T05:14:01.993 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002316938 2023-09-23T05:14:01.993 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.993 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2023-09-23T05:14:01.993 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.994 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.994 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.994 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.994 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2023-09-23T05:14:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.995 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2023-09-23T05:14:01.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.996 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.996 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.996 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.996 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2023-09-23T05:14:01.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.997 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.997 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.997 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.998 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2023-09-23T05:14:01.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.998 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.998 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:01.999 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:01.999 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2023-09-23T05:14:01.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:01.999 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:01.999 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.000 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.000 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2023-09-23T05:14:02.000 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.000 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.000 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.001 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.001 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2023-09-23T05:14:02.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.001 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.001 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.002 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.002 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2023-09-23T05:14:02.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.002 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.002 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.003 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.003 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2023-09-23T05:14:02.003 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2, 2023-09-23T05:14:02.003 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.001452506, 2023-09-23T05:14:02.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000726253 2023-09-23T05:14:02.004 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.004 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2023-09-23T05:14:02.004 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 49670, 2023-09-23T05:14:02.004 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 158.882080551, 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003198753 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2370, 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 11.240988733, 2023-09-23T05:14:02.005 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.004743033 2023-09-23T05:14:02.006 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.006 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2023-09-23T05:14:02.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.006 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.006 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.007 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.008 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2023-09-23T05:14:02.009 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 23, 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.608530749, 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.026457858 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.010 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 244, 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.200573626, 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000822023 2023-09-23T05:14:02.011 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.012 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2023-09-23T05:14:02.012 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.012 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.013 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.013 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.013 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2023-09-23T05:14:02.013 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.014 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 17768, 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 52.584337500, 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002959496 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.015 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.016 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 2, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 1, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 2, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2023-09-23T05:14:02.017 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 17370, 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 8685, 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 1601, 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:14:02.018 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:14:02.019 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2023-09-23T05:14:02.020 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:14:02.021 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:14:02.022 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2023-09-23T05:14:02.022 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:14:02.022 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:14:02.022 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:14:02.022 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:14:02.023 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 345858, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 781, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 528, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 6, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:14:02.024 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 10016, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 195, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:14:02.025 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 4310314, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 34294, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2023-09-23T05:14:02.026 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 64047, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 324553882, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 33, 2023-09-23T05:14:02.027 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 64047, 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 64047, 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 747.769740669, 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.011675328 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.028 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 64047, 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 64556 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 64047, 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 934, 2023-09-23T05:14:02.029 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 63113, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 0, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 21, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 838, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 26709, 2023-09-23T05:14:02.030 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 482, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 3, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 35628, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2023-09-23T05:14:02.031 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 90, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 42, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2023-09-23T05:14:02.032 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2023-09-23T05:14:02.033 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2023-09-23T05:14:02.034 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2023-09-23T05:14:02.034 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2023-09-23T05:14:02.034 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2023-09-23T05:14:02.035 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2023-09-23T05:14:02.035 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2023-09-23T05:14:02.035 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2023-09-23T05:14:02.035 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2023-09-23T05:14:02.035 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2023-09-23T05:14:02.036 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2023-09-23T05:14:02.036 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2023-09-23T05:14:02.036 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2023-09-23T05:14:02.036 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2023-09-23T05:14:02.036 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2023-09-23T05:14:02.037 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2023-09-23T05:14:02.037 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 132, 2023-09-23T05:14:02.037 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2023-09-23T05:14:02.037 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 45, 2023-09-23T05:14:02.038 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 12, 2023-09-23T05:14:02.038 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 12, 2023-09-23T05:14:02.038 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2023-09-23T05:14:02.038 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2023-09-23T05:14:02.038 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 517, 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 9, 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 217 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 1, 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:14:02.039 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 2 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2023-09-23T05:14:02.040 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 5, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 17768, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.041 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 5259908, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 462746299, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:02.042 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 5259909, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 5259908, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 462746299, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.043 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:02.044 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 64047, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 326492610, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 64047, 2023-09-23T05:14:02.045 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 326492610, 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.046 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:02.047 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 64047, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 64047, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 64047, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 64047, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.048 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 17768, 2023-09-23T05:14:02.049 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 1938864, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 17768, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 838, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 1938864, 2023-09-23T05:14:02.050 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x55bcc6a06640": { 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:02.051 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 100628, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 320925611, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 100628, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:02.052 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 25871, 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 320925611, 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.053 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:02.054 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-09-23T05:14:02.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:14:02.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:14:02.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:01 smithi005 ceph-mon[97638]: pgmap v875: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:14:02.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.242 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-23T05:14:02.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-23T05:14:02.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' 2023-09-23T05:14:02.243 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:01 smithi069 ceph-mon[133931]: pgmap v875: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-23T05:14:02.423 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2023-09-23T05:14:02.424 INFO:tasks.check_counter:Daemon mds.c mds.imported=10 2023-09-23T05:14:02.424 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2023-09-23T05:14:02.424 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:14:02.424 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:14:02.424 DEBUG:tasks.check_counter:Getting stats from f 2023-09-23T05:14:02.424 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-09-23T05:14:03.142 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T05:14:03.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:03 smithi005 ceph-mon[97638]: pgmap v876: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:14:03.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:03 smithi118 ceph-mon[134297]: pgmap v876: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:14:03.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:03 smithi069 ceph-mon[133931]: pgmap v876: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-23T05:14:05.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:05 smithi005 ceph-mon[97638]: pgmap v877: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:14:05.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:05 smithi118 ceph-mon[134297]: pgmap v877: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:14:05.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:05 smithi069 ceph-mon[133931]: pgmap v877: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-09-23T05:14:07.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:07 smithi005 ceph-mon[97638]: pgmap v878: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:14:07.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:07 smithi118 ceph-mon[134297]: pgmap v878: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:14:07.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:07 smithi069 ceph-mon[133931]: pgmap v878: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-23T05:14:09.085 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-09-23T05:14:09.085 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:14:09.085 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 150572, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 143132, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 53655729, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 164327693, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 22, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 8, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 8.582403493, 2023-09-23T05:14:09.086 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 3.400879584, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 5.698840349, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.654388176, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 143130, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 25.349814218, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000177110 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.087 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 257300, 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.046824539, 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000181 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 4880, 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 32560 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.088 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 67413, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 83995, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 20132447, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 154323610, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 21, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 6, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 4.564345994, 2023-09-23T05:14:09.089 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 1.690284001, 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 2.442879518, 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.693696631, 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 83995, 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 21.630095231, 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000257516 2023-09-23T05:14:09.090 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 105943, 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.022118840, 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000208 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 2088512, 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 315536 2023-09-23T05:14:09.091 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 4737626, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 4805268, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 782034783, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 1113960920, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 16, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 7, 2023-09-23T05:14:09.092 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 161.224758966, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 70.014137871, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 110.855390430, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.843837040, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 4805265, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 118.985397025, 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000024761 2023-09-23T05:14:09.093 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 9289026, 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.807498383, 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000086 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 1232, 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 811504 2023-09-23T05:14:09.094 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:09.095 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 44482, 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 56.857272635, 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.001278208 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:09.096 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 22304, 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 4.427516237, 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000198507 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.097 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 99382, 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 95742, 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 95742, 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 610.514327280, 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.006376661 2023-09-23T05:14:09.098 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 3639, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 3, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 38, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 3, 2023-09-23T05:14:09.099 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 1769, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 282, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 972, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 515, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 1367, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 473, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 59, 2023-09-23T05:14:09.100 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 59, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 48, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 164531, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 101710, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 3626, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 89, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2023-09-23T05:14:09.101 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 1488, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 11, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 305, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 22, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 1036, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2023-09-23T05:14:09.102 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 105, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 749, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 227408842, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 7, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 105, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2023-09-23T05:14:09.103 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 105, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 35, 2023-09-23T05:14:09.104 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 20666, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 51649, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 273, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 39636, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 10905, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 2257, 2023-09-23T05:14:09.105 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 72291 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 131, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 193, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 116, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 226, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 174, 2023-09-23T05:14:09.106 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 11964, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 11964, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2023-09-23T05:14:09.107 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 14, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 3, 2023-09-23T05:14:09.108 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 83225, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 0, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 0, 2023-09-23T05:14:09.109 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 83225, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 0, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 123, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 0, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 0, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 123, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 8, 2023-09-23T05:14:09.110 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 0, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 4194304, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 425090896, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 0, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 25543, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 300.200852848, 2023-09-23T05:14:09.111 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.011752764 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 0 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 1521, 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 13760, 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 12239, 2023-09-23T05:14:09.112 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 134, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 243, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 109, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 1769, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 23411, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 21642, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 59, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 14664, 2023-09-23T05:14:09.113 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 14605, 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 73548, 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 198960 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 151242, 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 7851, 2023-09-23T05:14:09.114 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 99382, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 966, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 21521, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 11958, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 76.372831811, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.006386756 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.115 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 11010, 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 255.220037654, 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.023180748 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.116 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2023-09-23T05:14:09.117 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 20301, 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 51.360150859, 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002529932 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.118 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.119 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.120 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.121 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.122 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.123 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.001260895, 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.001260895 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2023-09-23T05:14:09.124 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 38193, 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 156.722803020, 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.004103443 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2073, 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 11.045314658, 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.005328178 2023-09-23T05:14:09.125 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.126 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.127 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.128 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 66, 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 1.246202331, 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.018881853 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2023-09-23T05:14:09.129 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 176, 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.449584560, 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002554457 2023-09-23T05:14:09.130 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.131 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.132 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 11964, 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 58.096141492, 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.004855912 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2023-09-23T05:14:09.133 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 2, 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2023-09-23T05:14:09.134 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 1, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 2, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 19001, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 9500, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 1575, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.135 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:14:09.136 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2023-09-23T05:14:09.137 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:14:09.138 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2023-09-23T05:14:09.139 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 334318, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 1143, 2023-09-23T05:14:09.140 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 264, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 3, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:14:09.141 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 10016, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 195, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2023-09-23T05:14:09.142 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 5436787, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 48426, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2023-09-23T05:14:09.143 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 51539, 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 423568198, 2023-09-23T05:14:09.144 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 21, 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 51539, 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 51539, 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 776.464130791, 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.015065564 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.145 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 51539, 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 51977 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 51539, 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 719, 2023-09-23T05:14:09.146 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 50820, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 0, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 10, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 609, 2023-09-23T05:14:09.147 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 26152, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 372, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 3, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 24040, 2023-09-23T05:14:09.148 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 105, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 20, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:14:09.149 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2023-09-23T05:14:09.150 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2023-09-23T05:14:09.151 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2023-09-23T05:14:09.152 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 132, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 45, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 12, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 12, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2023-09-23T05:14:09.153 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 468, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 8, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 190 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 1, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2023-09-23T05:14:09.154 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 0 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T05:14:09.155 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 5, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 11964, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 68, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:09.156 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 4955610, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 412143160, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 4955610, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 4955609, 2023-09-23T05:14:09.157 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 412143092, 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2023-09-23T05:14:09.158 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:09.159 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 51539, 2023-09-23T05:14:09.160 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 424875064, 2023-09-23T05:14:09.160 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 51539, 2023-09-23T05:14:09.160 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 424875064, 2023-09-23T05:14:09.160 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:09.160 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2023-09-23T05:14:09.161 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 51539, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 51539, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 51539, 2023-09-23T05:14:09.162 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 51539, 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:14:09.163 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 11964, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 1306972, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 11964, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:09.164 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 609, 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 1306972, 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.165 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x563005194640": { 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 83225, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 420896592, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:09.166 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 83225, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 25543, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 420896592, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:09.167 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:09.168 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.168 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:09.168 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-09-23T05:14:09.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:09 smithi118 ceph-mon[134297]: pgmap v879: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:14:09.526 INFO:tasks.check_counter:Daemon mds.f mds.exported=11 2023-09-23T05:14:09.526 INFO:tasks.check_counter:Daemon mds.f mds.imported=22 2023-09-23T05:14:09.526 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2023-09-23T05:14:09.526 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:14:09.526 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:14:09.526 DEBUG:tasks.check_counter:Getting stats from i 2023-09-23T05:14:09.527 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-09-23T05:14:09.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:09 smithi005 ceph-mon[97638]: pgmap v879: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:14:09.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:09 smithi069 ceph-mon[133931]: pgmap v879: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2023-09-23T05:14:10.176 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T05:14:11.497 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-09-23T05:14:11.498 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:14:11.498 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 75926, 2023-09-23T05:14:11.498 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 74416, 2023-09-23T05:14:11.498 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 24072627, 2023-09-23T05:14:11.498 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 235387059, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 40, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 7, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 5.421530890, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 2.195159611, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 2.458506011, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 1.052857465, 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:11.499 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 74410, 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 13.222652395, 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000177699 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 83280, 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.017161005, 2023-09-23T05:14:11.500 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000206 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 6352, 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 4736 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 1957771, 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 1931320, 2023-09-23T05:14:11.501 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 350103569, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 437482125, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 10, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 5, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 53.394251956, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 20.054488000, 2023-09-23T05:14:11.502 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 36.835440239, 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.813812441, 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1931322, 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 33.799799574, 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000017500 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.503 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 3828514, 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.326774284, 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000085 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 810320 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.504 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 244086, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 363578, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 82399178, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 370904160, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 11, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 7, 2023-09-23T05:14:11.505 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 13.855166822, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 6.270361061, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 8.698300246, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 1.048981379, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 363577, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 65.410693430, 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000179908 2023-09-23T05:14:11.506 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 276573, 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.092048987, 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000332 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 2104048, 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 321344 2023-09-23T05:14:11.507 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:11.508 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 67710, 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 67.608152545, 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000998495 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:11.509 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 39128, 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 4.831366252, 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000123475 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 126243, 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 120909, 2023-09-23T05:14:11.510 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 120909, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 517.041733097, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.004276288 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 5334, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 1, 2023-09-23T05:14:11.511 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 21, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 1, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 1139, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 130, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 922, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 87, 2023-09-23T05:14:11.512 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 952, 2023-09-23T05:14:11.513 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 343, 2023-09-23T05:14:11.513 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 127, 2023-09-23T05:14:11.513 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 127, 2023-09-23T05:14:11.513 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 23, 2023-09-23T05:14:11.513 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 208176, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 124031, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 5324, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 106, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 2863, 2023-09-23T05:14:11.514 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 4, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 136, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 9, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 449, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 118, 2023-09-23T05:14:11.515 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 749, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 227408842, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 7, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 118, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:14:11.516 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 118, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 27, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 21811, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 85730, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:14:11.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 627, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 51759, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 22433, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 3350, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 91738 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2023-09-23T05:14:11.518 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 13, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 207, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 114, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 299, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 218, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:14:11.519 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 21001, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 21001, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2023-09-23T05:14:11.520 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 8, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 1, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.521 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 111833, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 26629, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 24581, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 87252, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 2048, 2023-09-23T05:14:11.522 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 159, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 31, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 29, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 130, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 8, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 2, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 42528701, 2023-09-23T05:14:11.523 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 457599603, 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 0, 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 36658, 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 282.012669626, 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.007693072 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 0 2023-09-23T05:14:11.524 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 1018, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 22252, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 21234, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 114, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 238, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 124, 2023-09-23T05:14:11.525 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 1139, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 36211, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 35072, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 127, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 22161, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 22034, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 73228, 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 198960 2023-09-23T05:14:11.526 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 189533, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 2964, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 126243, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 4374, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 13754, 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2023-09-23T05:14:11.527 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 21125, 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 76.825095078, 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003636690 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 3256, 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 63.505208345, 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.019504056 2023-09-23T05:14:11.528 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.529 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 28880, 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 49.398555655, 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.001710476 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.530 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1, 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000101523, 2023-09-23T05:14:11.531 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000101523 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2023-09-23T05:14:11.532 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.533 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2023-09-23T05:14:11.534 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.535 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.536 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 39896, 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 243.152566696, 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.006094660 2023-09-23T05:14:11.537 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 5264, 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 17.909683490, 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003402295 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.538 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.539 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.540 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2023-09-23T05:14:11.541 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 162, 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 2.315241503, 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.014291614 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.542 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1324, 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.916961282, 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000692568 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.543 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.544 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 21001, 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 63.018319525, 2023-09-23T05:14:11.545 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.003000729 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.546 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 1, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 3, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 2, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 1, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2023-09-23T05:14:11.547 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 17015, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 17015, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 1667, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2023-09-23T05:14:11.548 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:14:11.549 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2023-09-23T05:14:11.550 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2023-09-23T05:14:11.551 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:14:11.552 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 339291, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 694, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 528, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 6, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2023-09-23T05:14:11.553 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 10016, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 195, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:14:11.554 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 3653170, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 31277, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2023-09-23T05:14:11.555 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 81454, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 457309469, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 27, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 81454, 2023-09-23T05:14:11.556 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 81454, 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 973.886843380, 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.011956280 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 81454, 2023-09-23T05:14:11.557 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 81971 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 81454, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 1002, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 80452, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 0, 2023-09-23T05:14:11.558 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 21, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 883, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 37541, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 499, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 1, 2023-09-23T05:14:11.559 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 42131, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 118, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 42, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2023-09-23T05:14:11.560 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2023-09-23T05:14:11.561 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2023-09-23T05:14:11.562 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2023-09-23T05:14:11.563 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 133, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 46, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 12, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 12, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2023-09-23T05:14:11.564 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 499, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 2, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 234 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 1, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2023-09-23T05:14:11.565 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 1 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 10, 2023-09-23T05:14:11.566 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 5, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 21001, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:11.567 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 2277779, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 254560291, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 2277779, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 2277779, 2023-09-23T05:14:11.568 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 254560291, 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2023-09-23T05:14:11.569 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:11.570 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 81454, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 459601602, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 81454, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 459601602, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.571 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:11.572 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 81454, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 81454, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 81454, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 81454, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.573 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:11.574 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 21001, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 2292149, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 21001, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 883, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 2292149, 2023-09-23T05:14:11.575 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x56293ab76640": { 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:11.576 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 111833, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 453405299, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 111833, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:11.577 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 36658, 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 453405299, 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:11.578 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-09-23T05:14:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:11 smithi005 ceph-mon[97638]: osdmap e133: 12 total, 12 up, 12 in 2023-09-23T05:14:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:11 smithi005 ceph-mon[97638]: pgmap v881: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:11 smithi118 ceph-mon[134297]: osdmap e133: 12 total, 12 up, 12 in 2023-09-23T05:14:11.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:11 smithi118 ceph-mon[134297]: pgmap v881: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:11.934 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2023-09-23T05:14:11.935 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2023-09-23T05:14:11.935 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2023-09-23T05:14:11.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:14:11.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:14:11.935 DEBUG:tasks.check_counter:Getting stats from l 2023-09-23T05:14:11.936 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:862663913a74a99e1c52d50e6a7e59dd6705749a shell --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-09-23T05:14:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:11 smithi069 ceph-mon[133931]: osdmap e133: 12 total, 12 up, 12 in 2023-09-23T05:14:11.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:11 smithi069 ceph-mon[133931]: pgmap v881: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:12.551 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/mon.c/config 2023-09-23T05:14:13.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:13 smithi118 ceph-mon[134297]: pgmap v882: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:13.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:13 smithi005 ceph-mon[97638]: pgmap v882: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:13.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:13 smithi069 ceph-mon[133931]: pgmap v882: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-23T05:14:13.861 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2023-09-23T05:14:13.861 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-0": { 2023-09-23T05:14:13.861 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 1515, 2023-09-23T05:14:13.861 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 1517, 2023-09-23T05:14:13.862 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 210108465, 2023-09-23T05:14:13.862 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 471229, 2023-09-23T05:14:13.862 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 8, 2023-09-23T05:14:13.862 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 4, 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.232863132, 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.029247674, 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.149610384, 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.042794081, 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:13.863 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 1515, 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.155112274, 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000102384 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.864 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 0 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.865 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-1": { 2023-09-23T05:14:13.866 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 2252, 2023-09-23T05:14:13.866 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 2183, 2023-09-23T05:14:13.866 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 191248505, 2023-09-23T05:14:13.866 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 816408, 2023-09-23T05:14:13.867 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 6, 2023-09-23T05:14:13.867 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 5, 2023-09-23T05:14:13.867 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.283485762, 2023-09-23T05:14:13.867 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.058241191, 2023-09-23T05:14:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.167764068, 2023-09-23T05:14:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.056253871, 2023-09-23T05:14:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 2183, 2023-09-23T05:14:13.868 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.159198447, 2023-09-23T05:14:13.869 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000072926 2023-09-23T05:14:13.869 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.869 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:13.869 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.869 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.870 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.870 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.870 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 2082864, 2023-09-23T05:14:13.870 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 307296 2023-09-23T05:14:13.871 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.871 INFO:teuthology.orchestra.run.smithi118.stdout: "AsyncMessenger::Worker-2": { 2023-09-23T05:14:13.871 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_messages": 3480, 2023-09-23T05:14:13.871 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages": 3855, 2023-09-23T05:14:13.872 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_bytes": 97317427, 2023-09-23T05:14:13.872 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_bytes": 1971651, 2023-09-23T05:14:13.872 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_created_connections": 9, 2023-09-23T05:14:13.872 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_active_connections": 6, 2023-09-23T05:14:13.873 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_total_time": 0.305321862, 2023-09-23T05:14:13.873 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_send_time": 0.072260784, 2023-09-23T05:14:13.873 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_recv_time": 0.115362282, 2023-09-23T05:14:13.873 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_running_fast_dispatch_time": 0.095070955, 2023-09-23T05:14:13.874 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_messages_queue_lat": { 2023-09-23T05:14:13.874 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 3852, 2023-09-23T05:14:13.874 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.212219305, 2023-09-23T05:14:13.875 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000055093 2023-09-23T05:14:13.875 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.875 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_handle_ack_lat": { 2023-09-23T05:14:13.875 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.876 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.876 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.876 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.876 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_recv_encrypted_bytes": 480, 2023-09-23T05:14:13.877 INFO:teuthology.orchestra.run.smithi118.stdout: "msgr_send_encrypted_bytes": 890304 2023-09-23T05:14:13.877 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.877 INFO:teuthology.orchestra.run.smithi118.stdout: "cct": { 2023-09-23T05:14:13.878 INFO:teuthology.orchestra.run.smithi118.stdout: "total_workers": 1, 2023-09-23T05:14:13.878 INFO:teuthology.orchestra.run.smithi118.stdout: "unhealthy_workers": 0 2023-09-23T05:14:13.878 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.878 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-MDSRank": { 2023-09-23T05:14:13.879 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:13.879 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:13.879 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 10976, 2023-09-23T05:14:13.879 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.313301500, 2023-09-23T05:14:13.880 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000028544 2023-09-23T05:14:13.880 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.880 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.880 INFO:teuthology.orchestra.run.smithi118.stdout: "finisher-PurgeQueue": { 2023-09-23T05:14:13.881 INFO:teuthology.orchestra.run.smithi118.stdout: "queue_len": 0, 2023-09-23T05:14:13.881 INFO:teuthology.orchestra.run.smithi118.stdout: "complete_latency": { 2023-09-23T05:14:13.881 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.881 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.882 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.882 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.882 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.882 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2023-09-23T05:14:13.883 INFO:teuthology.orchestra.run.smithi118.stdout: "request": 0, 2023-09-23T05:14:13.883 INFO:teuthology.orchestra.run.smithi118.stdout: "reply": 0, 2023-09-23T05:14:13.883 INFO:teuthology.orchestra.run.smithi118.stdout: "reply_latency": { 2023-09-23T05:14:13.883 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.884 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.884 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.884 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.884 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_reply": 0, 2023-09-23T05:14:13.885 INFO:teuthology.orchestra.run.smithi118.stdout: "forward": 0, 2023-09-23T05:14:13.885 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_complete": 0, 2023-09-23T05:14:13.885 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_fetch_keys": 0, 2023-09-23T05:14:13.885 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_commit": 0, 2023-09-23T05:14:13.886 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_split": 0, 2023-09-23T05:14:13.886 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_merge": 0, 2023-09-23T05:14:13.886 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes": 19915, 2023-09-23T05:14:13.887 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_top": 0, 2023-09-23T05:14:13.887 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_bottom": 0, 2023-09-23T05:14:13.887 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pin_tail": 19915, 2023-09-23T05:14:13.887 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_pinned": 19915, 2023-09-23T05:14:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_expired": 3783, 2023-09-23T05:14:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "inodes_with_caps": 0, 2023-09-23T05:14:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "caps": 0, 2023-09-23T05:14:13.888 INFO:teuthology.orchestra.run.smithi118.stdout: "subtrees": 25, 2023-09-23T05:14:13.889 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse": 0, 2023-09-23T05:14:13.889 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_hit": 0, 2023-09-23T05:14:13.889 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_forward": 0, 2023-09-23T05:14:13.889 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_discover": 0, 2023-09-23T05:14:13.890 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_dir_fetch": 0, 2023-09-23T05:14:13.890 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_remote_ino": 0, 2023-09-23T05:14:13.890 INFO:teuthology.orchestra.run.smithi118.stdout: "traverse_lock": 0, 2023-09-23T05:14:13.891 INFO:teuthology.orchestra.run.smithi118.stdout: "load_cent": 0, 2023-09-23T05:14:13.893 INFO:teuthology.orchestra.run.smithi118.stdout: "q": 0, 2023-09-23T05:14:13.893 INFO:teuthology.orchestra.run.smithi118.stdout: "exported": 0, 2023-09-23T05:14:13.893 INFO:teuthology.orchestra.run.smithi118.stdout: "exported_inodes": 0, 2023-09-23T05:14:13.894 INFO:teuthology.orchestra.run.smithi118.stdout: "imported": 0, 2023-09-23T05:14:13.894 INFO:teuthology.orchestra.run.smithi118.stdout: "imported_inodes": 0, 2023-09-23T05:14:13.894 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_dir_fetch": 0, 2023-09-23T05:14:13.894 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_backtrace_fetch": 0, 2023-09-23T05:14:13.895 INFO:teuthology.orchestra.run.smithi118.stdout: "openino_peer_discover": 0, 2023-09-23T05:14:13.895 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rfiles": 0, 2023-09-23T05:14:13.895 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rbytes": 0, 2023-09-23T05:14:13.895 INFO:teuthology.orchestra.run.smithi118.stdout: "root_rsnaps": 0, 2023-09-23T05:14:13.895 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_fetch": 0, 2023-09-23T05:14:13.896 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_set_tag": 0, 2023-09-23T05:14:13.896 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_backtrace_repaired": 0, 2023-09-23T05:14:13.896 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_inotable_repaired": 0, 2023-09-23T05:14:13.896 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_inodes": 0, 2023-09-23T05:14:13.896 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dir_base_inodes": 0, 2023-09-23T05:14:13.897 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_dirfrag_rstats": 0, 2023-09-23T05:14:13.897 INFO:teuthology.orchestra.run.smithi118.stdout: "scrub_file_inodes": 0, 2023-09-23T05:14:13.897 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_inode_file_caps": 0, 2023-09-23T05:14:13.897 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_revoke": 0, 2023-09-23T05:14:13.898 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_grant": 0, 2023-09-23T05:14:13.898 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_trunc": 0, 2023-09-23T05:14:13.898 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-23T05:14:13.898 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-23T05:14:13.898 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps": 0, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_caps_dirty": 0, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_cap_release": 0, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "process_request_cap_release": 0 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_cache": { 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update": 0, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_update_receipt": 0, 2023-09-23T05:14:13.899 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_try_discover": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_send_discover": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "dir_handle_discover": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_delayed": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "num_strays_enqueuing": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_created": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_enqueued": 0, 2023-09-23T05:14:13.900 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_reintegrated": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "strays_migrated": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_processing": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_enqueued": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "num_recovering_prioritized": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_started": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "recovery_completed": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_enqueue_scrub": 0, 2023-09-23T05:14:13.901 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_exportdir": 0, 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_flush": 0, 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragmentdir": 0, 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_fragstats": 0, 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "ireq_inodestats": 0 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_log": { 2023-09-23T05:14:13.902 INFO:teuthology.orchestra.run.smithi118.stdout: "evlrg": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "evadd": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "evex": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "evtrm": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "ev": 117793, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "evexg": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "evexd": 0, 2023-09-23T05:14:13.903 INFO:teuthology.orchestra.run.smithi118.stdout: "segadd": 0, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "segex": 0, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "segtrm": 0, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "seg": 131, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "segmjr": 11, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "segexg": 0, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "segexd": 0, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "expos": 46688004, 2023-09-23T05:14:13.904 INFO:teuthology.orchestra.run.smithi118.stdout: "wrpos": 498942523, 2023-09-23T05:14:13.905 INFO:teuthology.orchestra.run.smithi118.stdout: "rdpos": 498941949, 2023-09-23T05:14:13.905 INFO:teuthology.orchestra.run.smithi118.stdout: "jlat": { 2023-09-23T05:14:13.905 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.905 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.905 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.906 INFO:teuthology.orchestra.run.smithi118.stdout: "replayed": 117793 2023-09-23T05:14:13.906 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.906 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_mem": { 2023-09-23T05:14:13.906 INFO:teuthology.orchestra.run.smithi118.stdout: "ino": 659, 2023-09-23T05:14:13.907 INFO:teuthology.orchestra.run.smithi118.stdout: "ino+": 24111, 2023-09-23T05:14:13.907 INFO:teuthology.orchestra.run.smithi118.stdout: "ino-": 23452, 2023-09-23T05:14:13.907 INFO:teuthology.orchestra.run.smithi118.stdout: "dir": 66, 2023-09-23T05:14:13.907 INFO:teuthology.orchestra.run.smithi118.stdout: "dir+": 107, 2023-09-23T05:14:13.908 INFO:teuthology.orchestra.run.smithi118.stdout: "dir-": 41, 2023-09-23T05:14:13.908 INFO:teuthology.orchestra.run.smithi118.stdout: "dn": 19915, 2023-09-23T05:14:13.908 INFO:teuthology.orchestra.run.smithi118.stdout: "dn+": 24472, 2023-09-23T05:14:13.908 INFO:teuthology.orchestra.run.smithi118.stdout: "dn-": 4557, 2023-09-23T05:14:13.908 INFO:teuthology.orchestra.run.smithi118.stdout: "cap": 0, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "cap+": 0, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "cap-": 0, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "rss": 72452, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "heap": 207152 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_server": { 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_client_request": 0, 2023-09-23T05:14:13.909 INFO:teuthology.orchestra.run.smithi118.stdout: "dispatch_server_request": 0, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_request": 0, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_client_session": 0, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "handle_peer_request": 0, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "req_create_latency": { 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.910 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getattr_latency": { 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.911 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getfilelock_latency": { 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "req_link_latency": { 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.912 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookup_latency": { 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.913 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookuphash_latency": { 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupino_latency": { 2023-09-23T05:14:13.914 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupname_latency": { 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.915 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupparent_latency": { 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.916 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lookupsnap_latency": { 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "req_lssnap_latency": { 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.917 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mkdir_latency": { 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.918 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mknod_latency": { 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "req_mksnap_latency": { 2023-09-23T05:14:13.919 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "req_open_latency": { 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.920 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: "req_readdir_latency": { 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.921 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rename_latency": { 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "req_renamesnap_latency": { 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.922 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: "req_snapdiff_latency": { 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.923 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.924 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmdir_latency": { 2023-09-23T05:14:13.924 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.924 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.924 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.925 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.925 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmsnap_latency": { 2023-09-23T05:14:13.925 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.925 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.925 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.926 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.926 INFO:teuthology.orchestra.run.smithi118.stdout: "req_rmxattr_latency": { 2023-09-23T05:14:13.926 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.926 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.927 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.927 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.927 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setattr_latency": { 2023-09-23T05:14:13.927 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.927 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.928 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.928 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.928 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setdirlayout_latency": { 2023-09-23T05:14:13.928 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.929 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.929 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.929 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.929 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setfilelock_latency": { 2023-09-23T05:14:13.930 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.930 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.930 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.930 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.931 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setlayout_latency": { 2023-09-23T05:14:13.931 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.931 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.931 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.932 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.932 INFO:teuthology.orchestra.run.smithi118.stdout: "req_setxattr_latency": { 2023-09-23T05:14:13.932 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.932 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.933 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.933 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.933 INFO:teuthology.orchestra.run.smithi118.stdout: "req_symlink_latency": { 2023-09-23T05:14:13.934 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.934 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.934 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.934 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.934 INFO:teuthology.orchestra.run.smithi118.stdout: "req_unlink_latency": { 2023-09-23T05:14:13.935 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.935 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.935 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.935 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.936 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_revoke_eviction": 0, 2023-09-23T05:14:13.936 INFO:teuthology.orchestra.run.smithi118.stdout: "cap_acquisition_throttle": 0, 2023-09-23T05:14:13.936 INFO:teuthology.orchestra.run.smithi118.stdout: "req_getvxattr_latency": { 2023-09-23T05:14:13.936 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.937 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.937 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.937 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.937 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.937 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_sessions": { 2023-09-23T05:14:13.938 INFO:teuthology.orchestra.run.smithi118.stdout: "session_count": 2, 2023-09-23T05:14:13.938 INFO:teuthology.orchestra.run.smithi118.stdout: "session_add": 4, 2023-09-23T05:14:13.938 INFO:teuthology.orchestra.run.smithi118.stdout: "session_remove": 2, 2023-09-23T05:14:13.938 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_open": 3, 2023-09-23T05:14:13.939 INFO:teuthology.orchestra.run.smithi118.stdout: "sessions_stale": 0, 2023-09-23T05:14:13.939 INFO:teuthology.orchestra.run.smithi118.stdout: "total_load": 0, 2023-09-23T05:14:13.939 INFO:teuthology.orchestra.run.smithi118.stdout: "average_load": 0, 2023-09-23T05:14:13.939 INFO:teuthology.orchestra.run.smithi118.stdout: "avg_session_uptime": 4391, 2023-09-23T05:14:13.939 INFO:teuthology.orchestra.run.smithi118.stdout: "mdthresh_evicted": 0 2023-09-23T05:14:13.940 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.940 INFO:teuthology.orchestra.run.smithi118.stdout: "mempool": { 2023-09-23T05:14:13.940 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_bytes": 0, 2023-09-23T05:14:13.940 INFO:teuthology.orchestra.run.smithi118.stdout: "bloom_filter_items": 0, 2023-09-23T05:14:13.941 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_bytes": 0, 2023-09-23T05:14:13.941 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_alloc_items": 0, 2023-09-23T05:14:13.941 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_bytes": 0, 2023-09-23T05:14:13.941 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_data_items": 0, 2023-09-23T05:14:13.941 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-23T05:14:13.942 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_onode_items": 0, 2023-09-23T05:14:13.942 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-23T05:14:13.942 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_meta_items": 0, 2023-09-23T05:14:13.942 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_bytes": 0, 2023-09-23T05:14:13.943 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_other_items": 0, 2023-09-23T05:14:13.943 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-23T05:14:13.943 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_cache_buffer_items": 0, 2023-09-23T05:14:13.943 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_bytes": 0, 2023-09-23T05:14:13.943 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_extent_items": 0, 2023-09-23T05:14:13.944 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_bytes": 0, 2023-09-23T05:14:13.944 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_blob_items": 0, 2023-09-23T05:14:13.944 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-23T05:14:13.944 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_shared_blob_items": 0, 2023-09-23T05:14:13.945 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-23T05:14:13.945 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_inline_bl_items": 0, 2023-09-23T05:14:13.945 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_bytes": 0, 2023-09-23T05:14:13.945 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_fsck_items": 0, 2023-09-23T05:14:13.946 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_bytes": 0, 2023-09-23T05:14:13.946 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_txc_items": 0, 2023-09-23T05:14:13.946 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-23T05:14:13.946 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_deferred_items": 0, 2023-09-23T05:14:13.946 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_bytes": 0, 2023-09-23T05:14:13.947 INFO:teuthology.orchestra.run.smithi118.stdout: "bluestore_writing_items": 0, 2023-09-23T05:14:13.947 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_bytes": 0, 2023-09-23T05:14:13.947 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_items": 0, 2023-09-23T05:14:13.947 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_bytes": 0, 2023-09-23T05:14:13.948 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_reader_items": 0, 2023-09-23T05:14:13.948 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_bytes": 0, 2023-09-23T05:14:13.948 INFO:teuthology.orchestra.run.smithi118.stdout: "bluefs_file_writer_items": 0, 2023-09-23T05:14:13.948 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_bytes": 271493, 2023-09-23T05:14:13.948 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_anon_items": 959, 2023-09-23T05:14:13.949 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_bytes": 0, 2023-09-23T05:14:13.949 INFO:teuthology.orchestra.run.smithi118.stdout: "buffer_meta_items": 0, 2023-09-23T05:14:13.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_bytes": 0, 2023-09-23T05:14:13.949 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_items": 0, 2023-09-23T05:14:13.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_bytes": 0, 2023-09-23T05:14:13.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_mapbl_items": 0, 2023-09-23T05:14:13.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_bytes": 0, 2023-09-23T05:14:13.950 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_pglog_items": 0, 2023-09-23T05:14:13.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_bytes": 10256, 2023-09-23T05:14:13.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_items": 201, 2023-09-23T05:14:13.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_bytes": 0, 2023-09-23T05:14:13.951 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_mapping_items": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_bytes": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "pgmap_items": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_bytes": 15794094, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "mds_co_items": 99978, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_bytes": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_1_items": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_bytes": 0, 2023-09-23T05:14:13.952 INFO:teuthology.orchestra.run.smithi118.stdout: "unittest_2_items": 0 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "objecter": { 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_active": 0, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_laggy": 0, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send": 6629, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_send_bytes": 21, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_resend": 3, 2023-09-23T05:14:13.953 INFO:teuthology.orchestra.run.smithi118.stdout: "op_reply": 6629, 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "op_latency": { 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 6629, 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 19.114583775, 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.002883479 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "op_inflight": 0, 2023-09-23T05:14:13.954 INFO:teuthology.orchestra.run.smithi118.stdout: "oplen_avg": { 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 6629, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 6630 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "op": 6629, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "op_r": 6629, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "op_w": 0, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "op_rmw": 0, 2023-09-23T05:14:13.955 INFO:teuthology.orchestra.run.smithi118.stdout: "op_pg": 0, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_stat": 3822, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_create": 0, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_read": 2805, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_write": 0, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writefull": 0, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_writesame": 0, 2023-09-23T05:14:13.956 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_append": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_zero": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_truncate": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_delete": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_mapext": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_sparse_read": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_clonerange": 0, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_getxattr": 1, 2023-09-23T05:14:13.957 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_setxattr": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_cmpxattr": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_rmxattr": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_resetxattrs": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_call": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_watch": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_notify": 0, 2023-09-23T05:14:13.958 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_src_cmpxattr": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_pgls_filter": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "osdop_other": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_active": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_send": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_resend": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "linger_ping": 0, 2023-09-23T05:14:13.959 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_active": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_send": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "poolop_resend": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_active": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_send": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "poolstat_resend": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_active": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_send": 0, 2023-09-23T05:14:13.960 INFO:teuthology.orchestra.run.smithi118.stdout: "statfs_resend": 0, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "command_active": 0, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "command_send": 0, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "command_resend": 0, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "map_epoch": 133, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "map_full": 0, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "map_inc": 46, 2023-09-23T05:14:13.961 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_sessions": 11, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_open": 11, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_session_close": 0, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "osd_laggy": 0, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_wr": 0, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_rd": 2, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_del": 0 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.962 INFO:teuthology.orchestra.run.smithi118.stdout: "oft": { 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_objs": 0, 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_kv_pairs": 0, 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_updates": 0, 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "omap_total_removes": 0 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "osd-slow-ops": { 2023-09-23T05:14:13.963 INFO:teuthology.orchestra.run.smithi118.stdout: "slow_ops_count": 0 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "purge_queue": { 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops": 0, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_ops_high_water": 0, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing": 0, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executing_high_water": 0, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_executed": 0, 2023-09-23T05:14:13.964 INFO:teuthology.orchestra.run.smithi118.stdout: "pq_item_in_journal": 0 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 7243, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 497988796, 2023-09-23T05:14:13.965 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 7243, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 7243, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 497988796, 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:13.966 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_bytes": { 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:13.967 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 104857600, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 6629, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 494760677, 2023-09-23T05:14:13.968 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 6629, 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 494760677, 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.969 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-objecter_ops": { 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 1024, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:13.970 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 6629, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 6629, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 6629, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 6629, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.971 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle": { 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:13.972 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 0, 2023-09-23T05:14:13.973 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: "throttle-write_buf_throttle-0x55ab8282c960": { 2023-09-23T05:14:13.974 INFO:teuthology.orchestra.run.smithi118.stdout: "val": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "max": 3758096384, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "get_started": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "get": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "get_sum": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_fail": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "get_or_fail_success": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "take": 0, 2023-09-23T05:14:13.975 INFO:teuthology.orchestra.run.smithi118.stdout: "take_sum": 0, 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "put": 0, 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "put_sum": 0, 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "wait": { 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "avgcount": 0, 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "sum": 0.000000000, 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: "avgtime": 0.000000000 2023-09-23T05:14:13.976 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.977 INFO:teuthology.orchestra.run.smithi118.stdout: } 2023-09-23T05:14:13.977 INFO:teuthology.orchestra.run.smithi118.stdout:} 2023-09-23T05:14:15.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:15 smithi005 ceph-mon[97638]: pgmap v883: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:14:15.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:15 smithi118 ceph-mon[134297]: pgmap v883: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:14:15.741 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:15 smithi069 ceph-mon[133931]: pgmap v883: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-23T05:14:16.525 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-09-23T05:14:16.525 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-09-23T05:14:16.525 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-09-23T05:14:16.525 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-09-23T05:14:16.526 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-09-23T05:14:16.526 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-23T05:14:16.538 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-23T05:14:16.539 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T05:14:16.539 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T05:14:16.605 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T05:14:16.605 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T05:14:16.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T05:14:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 -- 172.21.15.5:0/1076289440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d00690a0 msgr2=0x7f80d0069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1076289440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d00690a0 0x7f80d0069480 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009990 tx=0x7f80c4009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 -- 172.21.15.5:0/1076289440 shutdown_connections 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1076289440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 0x7f80d010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1076289440 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d00690a0 0x7f80d0069480 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1076289440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d0069a50 0x7f80d0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 -- 172.21.15.5:0/1076289440 >> 172.21.15.5:0/1076289440 conn(0x7f80d0077ca0 msgr2=0x7f80d00780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 -- 172.21.15.5:0/1076289440 shutdown_connections 2023-09-23T05:14:16.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.903+0000 7f80d71ff700 1 -- 172.21.15.5:0/1076289440 wait complete. 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.904+0000 7f80d71ff700 1 Processor -- start 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.904+0000 7f80d71ff700 1 -- start start 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 0x7f80d0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 0x7f80d01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d019e5a0 con 0x7f80d00690a0 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d019e710 con 0x7f80d0106200 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d71ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d019e8d0 con 0x7f80d019db20 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d4f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 0x7f80d0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39688/0 (socket says 172.21.15.5:39688) 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 -- 172.21.15.5:0/1663555383 learned_addr learned my addr 172.21.15.5:0/1663555383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80cffff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 0x7f80d01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 -- 172.21.15.5:0/1663555383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 msgr2=0x7f80d01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 0x7f80d01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 -- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 msgr2=0x7f80d0199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 0x7f80d0199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d579c700 1 -- 172.21.15.5:0/1663555383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009520 con 0x7f80d019db20 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.905+0000 7f80d4f9b700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 0x7f80d0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:16.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80d579c700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80c8007f10 tx=0x7f80c800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017070 con 0x7f80d019db20 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c40095f0 con 0x7f80d019db20 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c800f040 con 0x7f80d019db20 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80054b0 con 0x7f80d019db20 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.906+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d01a6130 con 0x7f80d019db20 2023-09-23T05:14:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.907+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d0067410 con 0x7f80d019db20 2023-09-23T05:14:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.909+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f80c8004ad0 con 0x7f80d019db20 2023-09-23T05:14:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.910+0000 7f80cdffb700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 0x7f80c00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.911+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f80c80cce30 con 0x7f80d019db20 2023-09-23T05:14:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.911+0000 7f80d4f9b700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 0x7f80c00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.911+0000 7f80d4f9b700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 0x7f80c00a3bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80c400b2e0 tx=0x7f80c4005fb0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:16.914+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f80c8095bb0 con 0x7f80d019db20 2023-09-23T05:14:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.127+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80d0069b90 con 0x7f80d019db20 2023-09-23T05:14:17.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.130+0000 7f80cdffb700 1 -- 172.21.15.5:0/1663555383 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+25333 (secure 0 0 0) 0x7f80c80991d0 con 0x7f80d019db20 2023-09-23T05:14:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:14:17.131 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":134,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T05:14:16.097130+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3587586220":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2853321891":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1767282859":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/734408633":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":37,"length":1},{"begin":50,"length":1}]},{"pool":4,"snaps":[{"begin":37,"length":1},{"begin":50,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T05:14:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 msgr2=0x7f80c00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 0x7f80c00a3bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80c400b2e0 tx=0x7f80c4005fb0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 msgr2=0x7f80d019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80c8007f10 tx=0x7f80c800d3b0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 shutdown_connections 2023-09-23T05:14:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f80c00a1710 0x7f80c00a3bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f80d019db20 0x7f80d019dfa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d00690a0 0x7f80d0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 --2- 172.21.15.5:0/1663555383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d0106200 0x7f80d01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 >> 172.21.15.5:0/1663555383 conn(0x7f80d0077ca0 msgr2=0x7f80d010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 shutdown_connections 2023-09-23T05:14:17.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.132+0000 7f80d71ff700 1 -- 172.21.15.5:0/1663555383 wait complete. 2023-09-23T05:14:17.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T04:39:38.158798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T04:43:45.094222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T04:43:46.564900+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T04:43:50.207152+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T05:14:17.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T05:14:17.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 -- 172.21.15.5:0/3065355155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581061f0 msgr2=0x7f7a5810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/3065355155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581061f0 0x7f7a5810dd70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000b0a0 tx=0x7f7a5000b3b0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 -- 172.21.15.5:0/3065355155 shutdown_connections 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/3065355155 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581061f0 0x7f7a5810dd70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/3065355155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a581010d0 0x7f7a581014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/3065355155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a58101a80 0x7f7a58105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 -- 172.21.15.5:0/3065355155 >> 172.21.15.5:0/3065355155 conn(0x7f7a580fc990 msgr2=0x7f7a580fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:17.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 -- 172.21.15.5:0/3065355155 shutdown_connections 2023-09-23T05:14:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.361+0000 7f7a5fdff700 1 -- 172.21.15.5:0/3065355155 wait complete. 2023-09-23T05:14:17.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.362+0000 7f7a5fdff700 1 Processor -- start 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.362+0000 7f7a5fdff700 1 -- start start 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.362+0000 7f7a5fdff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.362+0000 7f7a5fdff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 0x7f7a581997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5819dc50 0x7f7a5819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819e6d0 con 0x7f7a5819dc50 2023-09-23T05:14:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5fdff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819e840 con 0x7f7a581061f0 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5fdff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5819ea00 con 0x7f7a581010d0 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39698/0 (socket says 172.21.15.5:39698) 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 -- 172.21.15.5:0/1905099583 learned_addr learned my addr 172.21.15.5:0/1905099583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5e39c700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5819dc50 0x7f7a5819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5d39a700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 0x7f7a581997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 -- 172.21.15.5:0/1905099583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 msgr2=0x7f7a581997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 0x7f7a581997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 -- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5819dc50 msgr2=0x7f7a5819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5819dc50 0x7f7a5819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 -- 172.21.15.5:0/1905099583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54009520 con 0x7f7a581010d0 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5d39a700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 0x7f7a581997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.363+0000 7f7a5db9b700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400eb30 tx=0x7f7a5400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.364+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54009880 con 0x7f7a581010d0 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.364+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a540044e0 con 0x7f7a581010d0 2023-09-23T05:14:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.364+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54003e90 con 0x7f7a581010d0 2023-09-23T05:14:17.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.364+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a50009d00 con 0x7f7a581010d0 2023-09-23T05:14:17.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.364+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a581a6200 con 0x7f7a581010d0 2023-09-23T05:14:17.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.365+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a58067ca0 con 0x7f7a581010d0 2023-09-23T05:14:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.367+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7a54004000 con 0x7f7a581010d0 2023-09-23T05:14:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.368+0000 7f7a4effd700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 0x7f7a440a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.368+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f7a54014070 con 0x7f7a581010d0 2023-09-23T05:14:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.368+0000 7f7a5d39a700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 0x7f7a440a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.369+0000 7f7a5d39a700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 0x7f7a440a3bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a5819a8c0 tx=0x7f7a48009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.370+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7a54096a00 con 0x7f7a581010d0 2023-09-23T05:14:17.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:17 smithi005 ceph-mon[97638]: osdmap e134: 12 total, 12 up, 12 in 2023-09-23T05:14:17.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:17 smithi118 ceph-mon[134297]: osdmap e134: 12 total, 12 up, 12 in 2023-09-23T05:14:17.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:17 smithi069 ceph-mon[133931]: osdmap e134: 12 total, 12 up, 12 in 2023-09-23T05:14:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.580+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a58101c00 con 0x7f7a581010d0 2023-09-23T05:14:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.580+0000 7f7a4effd700 1 -- 172.21.15.5:0/1905099583 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a5409a020 con 0x7f7a581010d0 2023-09-23T05:14:17.581 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.582+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 msgr2=0x7f7a440a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.582+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 0x7f7a440a3bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a5819a8c0 tx=0x7f7a48009260 comp rx=0 tx=0).stop 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.582+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 msgr2=0x7f7a581992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.582+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400eb30 tx=0x7f7a5400ee40 comp rx=0 tx=0).stop 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 shutdown_connections 2023-09-23T05:14:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f7a440a1710 0x7f7a440a3bd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5819dc50 0x7f7a5819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7a581010d0 0x7f7a581992b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 --2- 172.21.15.5:0/1905099583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a581061f0 0x7f7a581997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 >> 172.21.15.5:0/1905099583 conn(0x7f7a580fc990 msgr2=0x7f7a580fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 shutdown_connections 2023-09-23T05:14:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.583+0000 7f7a5fdff700 1 -- 172.21.15.5:0/1905099583 wait complete. 2023-09-23T05:14:17.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T05:14:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 -- 172.21.15.5:0/2264813104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c1010d0 msgr2=0x7fe87c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 --2- 172.21.15.5:0/2264813104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c1010d0 0x7fe87c1014b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009990 tx=0x7fe86c009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 -- 172.21.15.5:0/2264813104 shutdown_connections 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 --2- 172.21.15.5:0/2264813104 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 --2- 172.21.15.5:0/2264813104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c1010d0 0x7fe87c1014b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 --2- 172.21.15.5:0/2264813104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c101a80 0x7fe87c105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 -- 172.21.15.5:0/2264813104 >> 172.21.15.5:0/2264813104 conn(0x7fe87c0fc990 msgr2=0x7fe87c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 -- 172.21.15.5:0/2264813104 shutdown_connections 2023-09-23T05:14:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.809+0000 7fe8819ff700 1 -- 172.21.15.5:0/2264813104 wait complete. 2023-09-23T05:14:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 Processor -- start 2023-09-23T05:14:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 -- start start 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c1010d0 0x7fe87c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c19db60 0x7fe87c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c19e5e0 con 0x7fe87c19db60 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c19e750 con 0x7fe87c1010d0 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.810+0000 7fe8819ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c19e910 con 0x7fe87c1061f0 2023-09-23T05:14:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe87b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c19db60 0x7fe87c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39710/0 (socket says 172.21.15.5:39710) 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 -- 172.21.15.5:0/1343941154 learned_addr learned my addr 172.21.15.5:0/1343941154 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe87affd700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c1010d0 0x7fe87c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 -- 172.21.15.5:0/1343941154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c1010d0 msgr2=0x7fe87c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c1010d0 0x7fe87c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 -- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c19db60 msgr2=0x7fe87c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c19db60 0x7fe87c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 -- 172.21.15.5:0/1343941154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0095f0 con 0x7fe87c1061f0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe873fff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d980 tx=0x7fe86400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.811+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400f890 con 0x7fe87c1061f0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.812+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe864010460 con 0x7fe87c1061f0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.812+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c1a5e70 con 0x7fe87c1061f0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.812+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400e610 con 0x7fe87c1061f0 2023-09-23T05:14:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.812+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c1a6340 con 0x7fe87c1061f0 2023-09-23T05:14:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.813+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c067ca0 con 0x7fe87c1061f0 2023-09-23T05:14:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.815+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fe8640105d0 con 0x7fe87c1061f0 2023-09-23T05:14:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.817+0000 7fe8809fd700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 0x7fe8680a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.817+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fe8640cd0f0 con 0x7fe87c1061f0 2023-09-23T05:14:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.817+0000 7fe87affd700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 0x7fe8680a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.817+0000 7fe87affd700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 0x7fe8680a3c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00b2e0 tx=0x7fe86c005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:17.821+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fe864095f10 con 0x7fe87c1061f0 2023-09-23T05:14:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.034+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe87c101cd0 con 0x7fe87c1061f0 2023-09-23T05:14:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.035+0000 7fe8809fd700 1 -- 172.21.15.5:0/1343941154 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe87c101cd0 con 0x7fe87c1061f0 2023-09-23T05:14:18.035 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T05:14:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 msgr2=0x7fe8680a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 0x7fe8680a3c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00b2e0 tx=0x7fe86c005fd0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 msgr2=0x7fe87c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d980 tx=0x7fe86400dc90 comp rx=0 tx=0).stop 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 shutdown_connections 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fe8680a1790 0x7fe8680a3c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87c19db60 0x7fe87c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe87c1010d0 0x7fe87c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 --2- 172.21.15.5:0/1343941154 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fe87c1061f0 0x7fe87c199700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 >> 172.21.15.5:0/1343941154 conn(0x7fe87c0fc990 msgr2=0x7fe87c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 shutdown_connections 2023-09-23T05:14:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.037+0000 7fe8819ff700 1 -- 172.21.15.5:0/1343941154 wait complete. 2023-09-23T05:14:18.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T05:14:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 -- 172.21.15.5:0/992630888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd774101a80 msgr2=0x7fd774105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 --2- 172.21.15.5:0/992630888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd774101a80 0x7fd774105ac0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd768009990 tx=0x7fd768009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 -- 172.21.15.5:0/992630888 shutdown_connections 2023-09-23T05:14:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 --2- 172.21.15.5:0/992630888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd77410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 --2- 172.21.15.5:0/992630888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7741010d0 0x7fd7741014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 --2- 172.21.15.5:0/992630888 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd774101a80 0x7fd774105ac0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 -- 172.21.15.5:0/992630888 >> 172.21.15.5:0/992630888 conn(0x7fd7740fc990 msgr2=0x7fd7740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 -- 172.21.15.5:0/992630888 shutdown_connections 2023-09-23T05:14:18.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.266+0000 7fd7795ff700 1 -- 172.21.15.5:0/992630888 wait complete. 2023-09-23T05:14:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 Processor -- start 2023-09-23T05:14:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 -- start start 2023-09-23T05:14:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd774197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd77419b9e0 0x7fd77419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77419c460 con 0x7fd7741061f0 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77419c5d0 con 0x7fd77419b9e0 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.267+0000 7fd7795ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77419c790 con 0x7fd7741010d0 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39728/0 (socket says 172.21.15.5:39728) 2023-09-23T05:14:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd7727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd774197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 -- 172.21.15.5:0/1318344741 learned_addr learned my addr 172.21.15.5:0/1318344741 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd7737fe700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd77419b9e0 0x7fd77419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 -- 172.21.15.5:0/1318344741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd77419b9e0 msgr2=0x7fd77419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd77419b9e0 0x7fd77419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 -- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 msgr2=0x7fd774197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd774197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 -- 172.21.15.5:0/1318344741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c009c40 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd7727fc700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd774197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd772ffd700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9a0 tx=0x7fd76400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.268+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764009790 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.269+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76400ce80 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.269+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7680095f0 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.269+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764018680 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.269+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7741a4090 con 0x7fd7741010d0 2023-09-23T05:14:18.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.270+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774067ca0 con 0x7fd7741010d0 2023-09-23T05:14:18.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.272+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fd764010430 con 0x7fd7741010d0 2023-09-23T05:14:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.273+0000 7fd75bfff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 0x7fd75c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.273+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fd764014070 con 0x7fd7741010d0 2023-09-23T05:14:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.273+0000 7fd7727fc700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 0x7fd75c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.274+0000 7fd7727fc700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 0x7fd75c0a3c50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd774198650 tx=0x7fd768005fb0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.275+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fd764096aa0 con 0x7fd7741010d0 2023-09-23T05:14:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1663555383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:18 smithi005 ceph-mon[97638]: pgmap v885: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:14:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1905099583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:18 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1343941154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1663555383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:18.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:18 smithi118 ceph-mon[134297]: pgmap v885: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:14:18.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1905099583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:18 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1343941154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.488+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd774101a80 con 0x7fd7741010d0 2023-09-23T05:14:18.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.488+0000 7fd75bfff700 1 -- 172.21.15.5:0/1318344741 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd7640260a0 con 0x7fd7741010d0 2023-09-23T05:14:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-09-23T05:14:18.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 msgr2=0x7fd75c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 0x7fd75c0a3c50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd774198650 tx=0x7fd768005fb0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 msgr2=0x7fd774197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9a0 tx=0x7fd76400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 shutdown_connections 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fd75c0a1790 0x7fd75c0a3c50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd77419b9e0 0x7fd77419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fd7741010d0 0x7fd774197040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 --2- 172.21.15.5:0/1318344741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7741061f0 0x7fd774197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 >> 172.21.15.5:0/1318344741 conn(0x7fd7740fc990 msgr2=0x7fd7740fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 shutdown_connections 2023-09-23T05:14:18.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.491+0000 7fd7795ff700 1 -- 172.21.15.5:0/1318344741 wait complete. 2023-09-23T05:14:18.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1663555383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:18.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:18 smithi069 ceph-mon[133931]: pgmap v885: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-23T05:14:18.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1905099583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:18 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1343941154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-23T05:14:18.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T05:14:18.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 -- 172.21.15.5:0/1388273700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb40731f0 msgr2=0x7fbbb40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/1388273700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb40731f0 0x7fbbb40735d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009990 tx=0x7fbbb0009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 -- 172.21.15.5:0/1388273700 shutdown_connections 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/1388273700 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb410d700 0x7fbbb410fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/1388273700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb40731f0 0x7fbbb40735d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/1388273700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb4073b10 0x7fbbb410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 -- 172.21.15.5:0/1388273700 >> 172.21.15.5:0/1388273700 conn(0x7fbbb40fc990 msgr2=0x7fbbb40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 -- 172.21.15.5:0/1388273700 shutdown_connections 2023-09-23T05:14:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.718+0000 7fbbbbbff700 1 -- 172.21.15.5:0/1388273700 wait complete. 2023-09-23T05:14:18.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.719+0000 7fbbbbbff700 1 Processor -- start 2023-09-23T05:14:18.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.719+0000 7fbbbbbff700 1 -- start start 2023-09-23T05:14:18.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb4073b10 0x7fbbb41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb410d700 0x7fbbb419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419e240 con 0x7fbbb410d700 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419e3b0 con 0x7fbbb4073b10 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbbbbff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419e570 con 0x7fbbb40731f0 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39756/0 (socket says 172.21.15.5:39756) 2023-09-23T05:14:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 -- 172.21.15.5:0/2253378368 learned_addr learned my addr 172.21.15.5:0/2253378368 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbba19c700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb410d700 0x7fbbb419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb919a700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb4073b10 0x7fbbb41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 -- 172.21.15.5:0/2253378368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb4073b10 msgr2=0x7fbbb41997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb4073b10 0x7fbbb41997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 -- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb410d700 msgr2=0x7fbbb419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.720+0000 7fbbb999b700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb410d700 0x7fbbb419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbb999b700 1 -- 172.21.15.5:0/2253378368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00095f0 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbb999b700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0005910 tx=0x7fbbb0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb001d070 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb419e7f0 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbb0022470 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0017650 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.721+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41a6090 con 0x7fbbb40731f0 2023-09-23T05:14:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.722+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb4108910 con 0x7fbbb40731f0 2023-09-23T05:14:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.724+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fbbb00225e0 con 0x7fbbb40731f0 2023-09-23T05:14:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.725+0000 7fbbaaffd700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 0x7fbba00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.726+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fbbb00cf120 con 0x7fbbb40731f0 2023-09-23T05:14:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.726+0000 7fbbb919a700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 0x7fbba00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.726+0000 7fbbb919a700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 0x7fbba00a3c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbbb419a870 tx=0x7fbba4009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:18.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.728+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbbb0097fd0 con 0x7fbbb40731f0 2023-09-23T05:14:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.941+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbbb419a650 con 0x7fbbb40731f0 2023-09-23T05:14:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.942+0000 7fbbaaffd700 1 -- 172.21.15.5:0/2253378368 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v134) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbbb00270a0 con 0x7fbbb40731f0 2023-09-23T05:14:18.942 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 msgr2=0x7fbba00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 0x7fbba00a3c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbbb419a870 tx=0x7fbba4009260 comp rx=0 tx=0).stop 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 msgr2=0x7fbbb4199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0005910 tx=0x7fbbb0005720 comp rx=0 tx=0).stop 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 shutdown_connections 2023-09-23T05:14:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbba00a1790 0x7fbba00a3c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb410d700 0x7fbbb419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbbb40731f0 0x7fbbb4199260 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 --2- 172.21.15.5:0/2253378368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbb4073b10 0x7fbbb41997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.944+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 >> 172.21.15.5:0/2253378368 conn(0x7fbbb40fc990 msgr2=0x7fbbb410f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.945+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 shutdown_connections 2023-09-23T05:14:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:18.945+0000 7fbbbbbff700 1 -- 172.21.15.5:0/2253378368 wait complete. 2023-09-23T05:14:18.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T05:14:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 -- 172.21.15.5:0/228944835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76640690a0 msgr2=0x7f7664069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 --2- 172.21.15.5:0/228944835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76640690a0 0x7f7664069480 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f76540098f0 tx=0x7f7654009c00 comp rx=0 tx=0).stop 2023-09-23T05:14:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 -- 172.21.15.5:0/228944835 shutdown_connections 2023-09-23T05:14:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 --2- 172.21.15.5:0/228944835 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7664106210 0x7f766410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 --2- 172.21.15.5:0/228944835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76640690a0 0x7f7664069480 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 --2- 172.21.15.5:0/228944835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7664069a50 0x7f7664105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.169+0000 7f76697ff700 1 -- 172.21.15.5:0/228944835 >> 172.21.15.5:0/228944835 conn(0x7f7664077ca0 msgr2=0x7f76640780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.170+0000 7f76697ff700 1 -- 172.21.15.5:0/228944835 shutdown_connections 2023-09-23T05:14:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.170+0000 7f76697ff700 1 -- 172.21.15.5:0/228944835 wait complete. 2023-09-23T05:14:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.170+0000 7f76697ff700 1 Processor -- start 2023-09-23T05:14:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 -- start start 2023-09-23T05:14:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7664106210 0x7f7664199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f766419db60 0x7f766419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766419e5e0 con 0x7f7664106210 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766419e750 con 0x7f766419db60 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76697ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766419e910 con 0x7f76640690a0 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f7662ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f7662ffd700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39760/0 (socket says 172.21.15.5:39760) 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f7662ffd700 1 -- 172.21.15.5:0/2751853810 learned_addr learned my addr 172.21.15.5:0/2751853810 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76627fc700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7664106210 0x7f7664199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.171+0000 7f76637fe700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f766419db60 0x7f766419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 -- 172.21.15.5:0/2751853810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f766419db60 msgr2=0x7f766419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f766419db60 0x7f766419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 -- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7664106210 msgr2=0x7f7664199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7664106210 0x7f7664199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 -- 172.21.15.5:0/2751853810 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654009520 con 0x7f76640690a0 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f7662ffd700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7654009230 tx=0x7f765400bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765401c070 con 0x7f76640690a0 2023-09-23T05:14:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76540058e0 con 0x7f76640690a0 2023-09-23T05:14:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76540038e0 con 0x7f76640690a0 2023-09-23T05:14:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76641a5e10 con 0x7f76640690a0 2023-09-23T05:14:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.172+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76641a6230 con 0x7f76640690a0 2023-09-23T05:14:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.173+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7664067410 con 0x7f76640690a0 2023-09-23T05:14:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.175+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7654020030 con 0x7f76640690a0 2023-09-23T05:14:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.177+0000 7f764bfff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 0x7f764c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.177+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f76540ce540 con 0x7f76640690a0 2023-09-23T05:14:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.177+0000 7f76627fc700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 0x7f764c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.177+0000 7f76627fc700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 0x7f764c0a3c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f766419a7d0 tx=0x7f7658009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:19.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.180+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7654097450 con 0x7f76640690a0 2023-09-23T05:14:19.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:19 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1318344741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:19 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2253378368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.407+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f766419a510 con 0x7f76640690a0 2023-09-23T05:14:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.407+0000 7f764bfff700 1 -- 172.21.15.5:0/2751853810 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v134) v1 ==== 81+0+927 (secure 0 0 0) 0x7f765409aa70 con 0x7f76640690a0 2023-09-23T05:14:19.407 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/4010853589 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3587586220 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2853321891 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3461224331 2023-09-24T04:37:23.553088+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1973270664 2023-09-24T04:37:23.553088+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1525019998 2023-09-24T04:37:23.553088+0000 2023-09-23T05:14:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2022747688 2023-09-24T04:36:56.816377+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1767282859 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2627772518 2023-09-24T04:36:56.816377+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3431143820 2023-09-24T04:36:56.816377+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/774964976 2023-09-24T04:36:56.816377+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/4010853589 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/734408633 2023-09-24T04:46:16.587035+0000 2023-09-23T05:14:19.409 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1934869339 2023-09-24T04:37:23.553088+0000 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.409+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 msgr2=0x7f764c0a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.409+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 0x7f764c0a3c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f766419a7d0 tx=0x7f7658009260 comp rx=0 tx=0).stop 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 msgr2=0x7f76641991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7654009230 tx=0x7f765400bf20 comp rx=0 tx=0).stop 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 shutdown_connections 2023-09-23T05:14:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f764c0a1790 0x7f764c0a3c50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f766419db60 0x7f766419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76640690a0 0x7f76641991c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 --2- 172.21.15.5:0/2751853810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7664106210 0x7f7664199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 >> 172.21.15.5:0/2751853810 conn(0x7f7664077ca0 msgr2=0x7f766410c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 shutdown_connections 2023-09-23T05:14:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:19.410+0000 7f76697ff700 1 -- 172.21.15.5:0/2751853810 wait complete. 2023-09-23T05:14:19.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:19 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1318344741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:19 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2253378368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2023-09-23T05:14:19.429 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-23T05:14:19.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:19.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-09-23T05:14:19.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1318344741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:19 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2253378368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-23T05:14:19.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T05:14:19.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:19.691 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T05:14:19.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:19.870 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-09-23T05:14:20.002 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T05:14:20.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:20.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T05:14:20.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:20.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-09-23T05:14:20.204 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-23T05:14:20.204 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-09-23T05:14:20.205 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T05:14:20.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348692k,nr_inodes=4087173,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b9b12ed9a9675756398c01c798a6a045b210c5bc96a25f14ee5094d6775b4a04/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/b9b12ed9a9675756398c01c798a6a045b210c5bc96a25f14ee5094d6775b4a04/diff,workdir=/var/lib/containers/storage/overlay/b9b12ed9a9675756398c01c798a6a045b210c5bc96a25f14ee5094d6775b4a04/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/24f25012bc38e573cdb8845c578de1db32d9d29ad3a5c289ff6387d62067fba8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/24f25012bc38e573cdb8845c578de1db32d9d29ad3a5c289ff6387d62067fba8/diff,workdir=/var/lib/containers/storage/overlay/24f25012bc38e573cdb8845c578de1db32d9d29ad3a5c289ff6387d62067fba8/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/832e5e69fb0ede7891a43bed46ca18d241229af141dce7e0be9cc8a928cc3d58/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/832e5e69fb0ede7891a43bed46ca18d241229af141dce7e0be9cc8a928cc3d58/diff,workdir=/var/lib/containers/storage/overlay/832e5e69fb0ede7891a43bed46ca18d241229af141dce7e0be9cc8a928cc3d58/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f8522220dd0625668b683ef7fe0ae97d169166af79afdc5d8d0a63df7c2694c9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/f8522220dd0625668b683ef7fe0ae97d169166af79afdc5d8d0a63df7c2694c9/diff,workdir=/var/lib/containers/storage/overlay/f8522220dd0625668b683ef7fe0ae97d169166af79afdc5d8d0a63df7c2694c9/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ef4639c4f68c15d01a53462586b846fbcaad933e39f0467c8cab33f9947db678/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/ef4639c4f68c15d01a53462586b846fbcaad933e39f0467c8cab33f9947db678/diff,workdir=/var/lib/containers/storage/overlay/ef4639c4f68c15d01a53462586b846fbcaad933e39f0467c8cab33f9947db678/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4600bab91f22accac71b625898083ebeb4956d0a78c736d6e21e185625325044/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/4600bab91f22accac71b625898083ebeb4956d0a78c736d6e21e185625325044/diff,workdir=/var/lib/containers/storage/overlay/4600bab91f22accac71b625898083ebeb4956d0a78c736d6e21e185625325044/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/56328f78487dafec3fc7754d4a7b6dfee95af6ae64a120487ea9bbcfb92a9b3b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/56328f78487dafec3fc7754d4a7b6dfee95af6ae64a120487ea9bbcfb92a9b3b/diff,workdir=/var/lib/containers/storage/overlay/56328f78487dafec3fc7754d4a7b6dfee95af6ae64a120487ea9bbcfb92a9b3b/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/efa8c0ae26e31a4cb4d4ba7889418b5b6c7e7d4d88544efa68136a05b751b092/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/efa8c0ae26e31a4cb4d4ba7889418b5b6c7e7d4d88544efa68136a05b751b092/diff,workdir=/var/lib/containers/storage/overlay/efa8c0ae26e31a4cb4d4ba7889418b5b6c7e7d4d88544efa68136a05b751b092/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/557655226d71ff5d11ad8f1dc304d2db23fafc59a46feb4c0c1e6277fd762e9d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/557655226d71ff5d11ad8f1dc304d2db23fafc59a46feb4c0c1e6277fd762e9d/diff,workdir=/var/lib/containers/storage/overlay/557655226d71ff5d11ad8f1dc304d2db23fafc59a46feb4c0c1e6277fd762e9d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/83c5ba2a76e0c732da705c63df967bc6c58223aa1bda8c4ef36b924fa80b8be7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/U5LOWSZ2RWUIVVP7CJHSUFAXD4:/var/lib/containers/storage/overlay/l/PK5C4VPX3I6653B6W4OAARRQSF,upperdir=/var/lib/containers/storage/overlay/83c5ba2a76e0c732da705c63df967bc6c58223aa1bda8c4ef36b924fa80b8be7/diff,workdir=/var/lib/containers/storage/overlay/83c5ba2a76e0c732da705c63df967bc6c58223aa1bda8c4ef36b924fa80b8be7/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-23T05:14:20.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:20.252 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-09-23T05:14:20.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T05:14:20.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-09-23T05:14:20.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:20 smithi005 ceph-mon[97638]: pgmap v886: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:14:20.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:20 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2751853810' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T05:14:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:20 smithi118 ceph-mon[134297]: pgmap v886: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:14:20.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:20 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2751853810' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T05:14:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:20 smithi069 ceph-mon[133931]: pgmap v886: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 119 B/s wr, 9 op/s 2023-09-23T05:14:20.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:20 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2751853810' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-23T05:14:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2023-09-23T05:14:20.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:14:20.549 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-23T05:14:20.563 INFO:tasks.cephadm:Teardown begin 2023-09-23T05:14:20.564 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T05:14:20.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T05:14:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.794+0000 7f107e9ff700 1 -- 172.21.15.5:0/2046063655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 msgr2=0x7f10780735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.794+0000 7f107e9ff700 1 --2- 172.21.15.5:0/2046063655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10780735d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1068009990 tx=0x7f1068009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 -- 172.21.15.5:0/2046063655 shutdown_connections 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 --2- 172.21.15.5:0/2046063655 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f107810d710 0x7f107810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 --2- 172.21.15.5:0/2046063655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10780735d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 --2- 172.21.15.5:0/2046063655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1078073b10 0x7f107810d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 -- 172.21.15.5:0/2046063655 >> 172.21.15.5:0/2046063655 conn(0x7f10780fc980 msgr2=0x7f10780feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 -- 172.21.15.5:0/2046063655 shutdown_connections 2023-09-23T05:14:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.795+0000 7f107e9ff700 1 -- 172.21.15.5:0/2046063655 wait complete. 2023-09-23T05:14:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 Processor -- start 2023-09-23T05:14:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 -- start start 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10781991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107810d710 0x7f107819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107819e1d0 con 0x7f10780731f0 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107819e340 con 0x7f107810d710 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.796+0000 7f107e9ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107819e500 con 0x7f1078073b10 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:20.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:39778/0 (socket says 172.21.15.5:39778) 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f1077fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10781991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 -- 172.21.15.5:0/4248979051 learned_addr learned my addr 172.21.15.5:0/4248979051 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f107cf9c700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107810d710 0x7f107819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 -- 172.21.15.5:0/4248979051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107810d710 msgr2=0x7f107819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107810d710 0x7f107819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 -- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 msgr2=0x7f10781991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10781991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 -- 172.21.15.5:0/4248979051 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680095f0 con 0x7f1078073b10 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f1077fff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10781991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.797+0000 7f10777fe700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.798+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c009940 con 0x7f1078073b10 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.798+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f106c010460 con 0x7f1078073b10 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.798+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00f620 con 0x7f1078073b10 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.798+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107819e7e0 con 0x7f1078073b10 2023-09-23T05:14:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.798+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10781a6020 con 0x7f1078073b10 2023-09-23T05:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.799+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1078108920 con 0x7f1078073b10 2023-09-23T05:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.801+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f106c0108d0 con 0x7f1078073b10 2023-09-23T05:14:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.802+0000 7f10757fa700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 0x7f10600a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.802+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f106c0cd590 con 0x7f1078073b10 2023-09-23T05:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.802+0000 7f1077fff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 0x7f10600a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.803+0000 7f1077fff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 0x7f10600a3c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f106800b2e0 tx=0x7f1068005fd0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:20.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:20.807+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f106c0963b0 con 0x7f1078073b10 2023-09-23T05:14:21.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.065+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1078067ca0 con 0x7f1078073b10 2023-09-23T05:14:21.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.066+0000 7f10757fa700 1 -- 172.21.15.5:0/4248979051 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f106c0999d0 con 0x7f1078073b10 2023-09-23T05:14:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:14:21.067 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T05:09:05.556984+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T05:14:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 msgr2=0x7f10600a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 0x7f10600a3c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f106800b2e0 tx=0x7f1068005fd0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 msgr2=0x7f1078199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00dc60 comp rx=0 tx=0).stop 2023-09-23T05:14:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 shutdown_connections 2023-09-23T05:14:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f10600a1790 0x7f10600a3c50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f107810d710 0x7f107819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10780731f0 0x7f10781991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 --2- 172.21.15.5:0/4248979051 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f1078073b10 0x7f1078199730 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 >> 172.21.15.5:0/4248979051 conn(0x7f10780fc980 msgr2=0x7f107810f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 shutdown_connections 2023-09-23T05:14:21.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.069+0000 7f107e9ff700 1 -- 172.21.15.5:0/4248979051 wait complete. 2023-09-23T05:14:21.072 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2023-09-23T05:14:21.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T05:14:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.293+0000 7f8fb65ff700 1 -- 172.21.15.5:0/224826203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb01010b0 msgr2=0x7f8fb0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.293+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/224826203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb01010b0 0x7f8fb0101490 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00098f0 tx=0x7f8fa0009c00 comp rx=0 tx=0).stop 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 -- 172.21.15.5:0/224826203 shutdown_connections 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/224826203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01061d0 0x7f8fb010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/224826203 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb01010b0 0x7f8fb0101490 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/224826203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0101a60 0x7f8fb0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 -- 172.21.15.5:0/224826203 >> 172.21.15.5:0/224826203 conn(0x7f8fb00fc930 msgr2=0x7f8fb00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 -- 172.21.15.5:0/224826203 shutdown_connections 2023-09-23T05:14:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.294+0000 7f8fb65ff700 1 -- 172.21.15.5:0/224826203 wait complete. 2023-09-23T05:14:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 Processor -- start 2023-09-23T05:14:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 -- start start 2023-09-23T05:14:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 0x7f8fb0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb019e620 con 0x7f8fb01061d0 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb019e790 con 0x7f8fb01010b0 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb65ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb019e950 con 0x7f8fb019dba0 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8faf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb4b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8faffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 0x7f8fb0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb4b9c700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54758/0 (socket says 172.21.15.5:54758) 2023-09-23T05:14:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8fb4b9c700 1 -- 172.21.15.5:0/3865675918 learned_addr learned my addr 172.21.15.5:0/3865675918 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.295+0000 7f8faf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37060/0 (socket says 172.21.15.5:37060) 2023-09-23T05:14:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 -- 172.21.15.5:0/3865675918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 msgr2=0x7f8fb0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 0x7f8fb0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 -- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 msgr2=0x7f8fb0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 -- 172.21.15.5:0/3865675918 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0009520 con 0x7f8fb019dba0 2023-09-23T05:14:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8faffff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 0x7f8fb0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8faf7fe700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb4b9c700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ef30 tx=0x7f8fa800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8018070 con 0x7f8fb019dba0 2023-09-23T05:14:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa800f040 con 0x7f8fb019dba0 2023-09-23T05:14:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb01a5e50 con 0x7f8fb019dba0 2023-09-23T05:14:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.296+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800ccf0 con 0x7f8fb019dba0 2023-09-23T05:14:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.297+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01a62e0 con 0x7f8fb019dba0 2023-09-23T05:14:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.297+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0067ca0 con 0x7f8fb019dba0 2023-09-23T05:14:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.299+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f8fa8003a00 con 0x7f8fb019dba0 2023-09-23T05:14:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.301+0000 7f8fad7fa700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 0x7f8f980a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.301+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f8fa80cd3b0 con 0x7f8fb019dba0 2023-09-23T05:14:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.301+0000 7f8faffff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 0x7f8f980a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.302+0000 7f8faffff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 0x7f8f980a3bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005e90 tx=0x7f8fa0019040 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.303+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f8fa8096260 con 0x7f8fb019dba0 2023-09-23T05:14:21.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:21 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/4248979051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:21.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:21 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/4248979051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:21.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:21 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/4248979051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:21.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.559+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb019a550 con 0x7f8fb019dba0 2023-09-23T05:14:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.560+0000 7f8fad7fa700 1 -- 172.21.15.5:0/3865675918 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f8fa8099880 con 0x7f8fb019dba0 2023-09-23T05:14:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:14:21.561 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3475689025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3475689025},{"type":"v1","addr":"172.21.15.5:6835","nonce":3475689025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3951415099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3951415099},{"type":"v1","addr":"172.21.15.5:6837","nonce":3951415099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-23T04:43:47.148877+0000","modified":"2023-09-23T05:09:05.556984+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24577,"mds_1":24559,"mds_2":24554,"mds_3":24535,"mds_4":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/1890134322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1890134322},{"type":"v1","addr":"172.21.15.5:6841","nonce":1890134322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24443":{"gid":24443,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6833/827181928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":827181928},{"type":"v1","addr":"172.21.15.69:6833","nonce":827181928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.118:6833/3707604246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3707604246},{"type":"v1","addr":"172.21.15.118:6833","nonce":3707604246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6835/3966429650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3966429650},{"type":"v1","addr":"172.21.15.69:6835","nonce":3966429650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6837/2958173168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2958173168},{"type":"v1","addr":"172.21.15.69:6837","nonce":2958173168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24529":{"gid":24529,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.118:6835/3772048959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3772048959},{"type":"v1","addr":"172.21.15.118:6835","nonce":3772048959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24535":{"gid":24535,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.118:6837/704534117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6836","nonce":704534117},{"type":"v1","addr":"172.21.15.118:6837","nonce":704534117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24554":{"gid":24554,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6839/1851157393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1851157393},{"type":"v1","addr":"172.21.15.69:6839","nonce":1851157393}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.118:6839/430957039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6838","nonce":430957039},{"type":"v1","addr":"172.21.15.118:6839","nonce":430957039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24577":{"gid":24577,"name":"g","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.5:6839/3661239153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3661239153},{"type":"v1","addr":"172.21.15.5:6839","nonce":3661239153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 msgr2=0x7f8f980a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 0x7f8f980a3bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005e90 tx=0x7f8fa0019040 comp rx=0 tx=0).stop 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 msgr2=0x7f8fb019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800ef30 tx=0x7f8fa800c3a0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 shutdown_connections 2023-09-23T05:14:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f8f980a1710 0x7f8f980a3bd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f8fb019dba0 0x7f8fb019e020 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fb01010b0 0x7f8fb0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 --2- 172.21.15.5:0/3865675918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb01061d0 0x7f8fb0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.562+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 >> 172.21.15.5:0/3865675918 conn(0x7f8fb00fc930 msgr2=0x7f8fb010d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.563+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 shutdown_connections 2023-09-23T05:14:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.563+0000 7f8fb65ff700 1 -- 172.21.15.5:0/3865675918 wait complete. 2023-09-23T05:14:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2023-09-23T05:14:21.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T05:14:21.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 -- 172.21.15.5:0/3098956038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 msgr2=0x7fbab4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3098956038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4101490 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40098f0 tx=0x7fbaa4009c00 comp rx=0 tx=0).stop 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 -- 172.21.15.5:0/3098956038 shutdown_connections 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3098956038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3098956038 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4101490 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3098956038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4101a60 0x7fbab4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 -- 172.21.15.5:0/3098956038 >> 172.21.15.5:0/3098956038 conn(0x7fbab40fc930 msgr2=0x7fbab40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:21.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 -- 172.21.15.5:0/3098956038 shutdown_connections 2023-09-23T05:14:21.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.786+0000 7fbab9bff700 1 -- 172.21.15.5:0/3098956038 wait complete. 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.787+0000 7fbab9bff700 1 Processor -- start 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.787+0000 7fbab9bff700 1 -- start start 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4073ba0 0x7fbab41a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41a5910 con 0x7fbab41061d0 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41a5a80 con 0x7fbab4073ba0 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab9bff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41a5c40 con 0x7fbab41010b0 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbaabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54780/0 (socket says 172.21.15.5:54780) 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 -- 172.21.15.5:0/3304574353 learned_addr learned my addr 172.21.15.5:0/3304574353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbaabfff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37076/0 (socket says 172.21.15.5:37076) 2023-09-23T05:14:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab3fff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4073ba0 0x7fbab41a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 -- 172.21.15.5:0/3304574353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4073ba0 msgr2=0x7fbab41a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4073ba0 0x7fbab41a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 -- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 msgr2=0x7fbab4073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbab37fe700 1 -- 172.21.15.5:0/3304574353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4009520 con 0x7fbab41010b0 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.788+0000 7fbaabfff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T05:14:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab37fe700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009230 tx=0x7fbaa400bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa401c070 con 0x7fbab41010b0 2023-09-23T05:14:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaa40058e0 con 0x7fbab41010b0 2023-09-23T05:14:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40038e0 con 0x7fbab41010b0 2023-09-23T05:14:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab41a5ec0 con 0x7fbab41010b0 2023-09-23T05:14:21.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.789+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab41a6290 con 0x7fbab41010b0 2023-09-23T05:14:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.790+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4067ca0 con 0x7fbab41010b0 2023-09-23T05:14:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.792+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fbaa4020030 con 0x7fbab41010b0 2023-09-23T05:14:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.793+0000 7fbab17fa700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 0x7fbaa00a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.794+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7fbaa40ce700 con 0x7fbab41010b0 2023-09-23T05:14:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.794+0000 7fbaabfff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 0x7fbaa00a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:21.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.794+0000 7fbaabfff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 0x7fbaa00a3bd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbab4074910 tx=0x7fba9c009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:21.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:21.795+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fbaa40975b0 con 0x7fbab41010b0 2023-09-23T05:14:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.006+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab4101a60 con 0x7fbab41010b0 2023-09-23T05:14:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.008+0000 7fbab17fa700 1 -- 172.21.15.5:0/3304574353 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+25333 (secure 0 0 0) 0x7fbaa409abd0 con 0x7fbab41010b0 2023-09-23T05:14:22.009 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-09-23T05:14:22.009 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":134,"fsid":"a18d5e54-59ca-11ee-8db4-212e2dc638e7","created":"2023-09-23T04:36:25.203666+0000","modified":"2023-09-23T05:14:16.097130+0000","last_up_change":"2023-09-23T04:42:59.457632+0000","last_in_change":"2023-09-23T04:42:41.873377+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T04:39:38.158798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T04:43:45.094222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T04:43:46.564900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T04:43:50.207152+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":50,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"35700daf-4df6-4103-ade1-12999750ad33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6803","nonce":2285726296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6805","nonce":2285726296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6809","nonce":2285726296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2285726296},{"type":"v1","addr":"172.21.15.5:6807","nonce":2285726296}]},"public_addr":"172.21.15.5:6803/2285726296","cluster_addr":"172.21.15.5:6805/2285726296","heartbeat_back_addr":"172.21.15.5:6809/2285726296","heartbeat_front_addr":"172.21.15.5:6807/2285726296","state":["exists","up"]},{"osd":1,"uuid":"d0d9aada-e5c2-4198-8a8e-a9db453fd897","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6801","nonce":286933964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6803","nonce":286933964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6807","nonce":286933964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":286933964},{"type":"v1","addr":"172.21.15.69:6805","nonce":286933964}]},"public_addr":"172.21.15.69:6801/286933964","cluster_addr":"172.21.15.69:6803/286933964","heartbeat_back_addr":"172.21.15.69:6807/286933964","heartbeat_front_addr":"172.21.15.69:6805/286933964","state":["exists","up"]},{"osd":2,"uuid":"101be3f4-34d5-43eb-a131-2144d5a0f309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6801","nonce":275897405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6803","nonce":275897405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6807","nonce":275897405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":275897405},{"type":"v1","addr":"172.21.15.118:6805","nonce":275897405}]},"public_addr":"172.21.15.118:6801/275897405","cluster_addr":"172.21.15.118:6803/275897405","heartbeat_back_addr":"172.21.15.118:6807/275897405","heartbeat_front_addr":"172.21.15.118:6805/275897405","state":["exists","up"]},{"osd":3,"uuid":"b08d036a-cfc0-424d-963b-fc5fbeb8af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6811","nonce":4155254124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6813","nonce":4155254124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6817","nonce":4155254124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4155254124},{"type":"v1","addr":"172.21.15.5:6815","nonce":4155254124}]},"public_addr":"172.21.15.5:6811/4155254124","cluster_addr":"172.21.15.5:6813/4155254124","heartbeat_back_addr":"172.21.15.5:6817/4155254124","heartbeat_front_addr":"172.21.15.5:6815/4155254124","state":["exists","up"]},{"osd":4,"uuid":"03c71626-c985-43df-92a1-aa005836c3cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6809","nonce":3327136642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6811","nonce":3327136642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6815","nonce":3327136642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3327136642},{"type":"v1","addr":"172.21.15.69:6813","nonce":3327136642}]},"public_addr":"172.21.15.69:6809/3327136642","cluster_addr":"172.21.15.69:6811/3327136642","heartbeat_back_addr":"172.21.15.69:6815/3327136642","heartbeat_front_addr":"172.21.15.69:6813/3327136642","state":["exists","up"]},{"osd":5,"uuid":"35af4018-9f6b-4950-85d4-4af32e09ef2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6809","nonce":2643723046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6811","nonce":2643723046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6815","nonce":2643723046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":2643723046},{"type":"v1","addr":"172.21.15.118:6813","nonce":2643723046}]},"public_addr":"172.21.15.118:6809/2643723046","cluster_addr":"172.21.15.118:6811/2643723046","heartbeat_back_addr":"172.21.15.118:6815/2643723046","heartbeat_front_addr":"172.21.15.118:6813/2643723046","state":["exists","up"]},{"osd":6,"uuid":"3a9d6647-95c6-4cf0-ac70-52cb2639c1b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6819","nonce":1080291272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6821","nonce":1080291272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6825","nonce":1080291272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1080291272},{"type":"v1","addr":"172.21.15.5:6823","nonce":1080291272}]},"public_addr":"172.21.15.5:6819/1080291272","cluster_addr":"172.21.15.5:6821/1080291272","heartbeat_back_addr":"172.21.15.5:6825/1080291272","heartbeat_front_addr":"172.21.15.5:6823/1080291272","state":["exists","up"]},{"osd":7,"uuid":"496c61c7-6762-44b0-8361-bc62e51fb6cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6817","nonce":1999230066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6819","nonce":1999230066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6823","nonce":1999230066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1999230066},{"type":"v1","addr":"172.21.15.69:6821","nonce":1999230066}]},"public_addr":"172.21.15.69:6817/1999230066","cluster_addr":"172.21.15.69:6819/1999230066","heartbeat_back_addr":"172.21.15.69:6823/1999230066","heartbeat_front_addr":"172.21.15.69:6821/1999230066","state":["exists","up"]},{"osd":8,"uuid":"3aea8c15-9de3-4519-9f86-93b891d108e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6817","nonce":1078105960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6819","nonce":1078105960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6823","nonce":1078105960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1078105960},{"type":"v1","addr":"172.21.15.118:6821","nonce":1078105960}]},"public_addr":"172.21.15.118:6817/1078105960","cluster_addr":"172.21.15.118:6819/1078105960","heartbeat_back_addr":"172.21.15.118:6823/1078105960","heartbeat_front_addr":"172.21.15.118:6821/1078105960","state":["exists","up"]},{"osd":9,"uuid":"463a1b28-e1f2-485e-a11c-16f788e17414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6827","nonce":2457675866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6829","nonce":2457675866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6833","nonce":2457675866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2457675866},{"type":"v1","addr":"172.21.15.5:6831","nonce":2457675866}]},"public_addr":"172.21.15.5:6827/2457675866","cluster_addr":"172.21.15.5:6829/2457675866","heartbeat_back_addr":"172.21.15.5:6833/2457675866","heartbeat_front_addr":"172.21.15.5:6831/2457675866","state":["exists","up"]},{"osd":10,"uuid":"33ab3d09-beb2-4736-99b1-554ad9d15218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6825","nonce":4220981201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6827","nonce":4220981201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6831","nonce":4220981201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":4220981201},{"type":"v1","addr":"172.21.15.69:6829","nonce":4220981201}]},"public_addr":"172.21.15.69:6825/4220981201","cluster_addr":"172.21.15.69:6827/4220981201","heartbeat_back_addr":"172.21.15.69:6831/4220981201","heartbeat_front_addr":"172.21.15.69:6829/4220981201","state":["exists","up"]},{"osd":11,"uuid":"6354fc3d-d3ae-491d-8592-1b9bb4336d15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6825","nonce":1838374502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6827","nonce":1838374502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6831","nonce":1838374502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":1838374502},{"type":"v1","addr":"172.21.15.118:6829","nonce":1838374502}]},"public_addr":"172.21.15.118:6825/1838374502","cluster_addr":"172.21.15.118:6827/1838374502","heartbeat_back_addr":"172.21.15.118:6831/1838374502","heartbeat_front_addr":"172.21.15.118:6829/1838374502","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:38:53.974772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:15.194049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:39:35.697165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:02.106261+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:23.212820+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:40:43.593795+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:09.028452+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:30.276774+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:41:50.705584+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:16.122131+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:37.145665+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T04:42:57.623658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":3,"to":7},{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":1,"to":2}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3587586220":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2853321891":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/3461224331":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6801/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/1973270664":"2023-09-24T04:37:23.553088+0000","172.21.15.5:0/1525019998":"2023-09-24T04:37:23.553088+0000","172.21.15.5:6800/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/2022747688":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/1767282859":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/2627772518":"2023-09-24T04:36:56.816377+0000","172.21.15.5:0/3431143820":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/774964976":"2023-09-24T04:36:56.816377+0000","172.21.15.5:6801/4010853589":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/734408633":"2023-09-24T04:46:16.587035+0000","172.21.15.5:0/1934869339":"2023-09-24T04:37:23.553088+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":37,"length":1},{"begin":50,"length":1}]},{"pool":4,"snaps":[{"begin":37,"length":1},{"begin":50,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T05:14:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 msgr2=0x7fbaa00a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 0x7fbaa00a3bd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbab4074910 tx=0x7fba9c009260 comp rx=0 tx=0).stop 2023-09-23T05:14:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 msgr2=0x7fbab4073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009230 tx=0x7fbaa400bf20 comp rx=0 tx=0).stop 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 shutdown_connections 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fbaa00a1710 0x7fbaa00a3bd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbab4073ba0 0x7fbab41a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fbab41010b0 0x7fbab4073120 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 --2- 172.21.15.5:0/3304574353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab41061d0 0x7fbab4073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 >> 172.21.15.5:0/3304574353 conn(0x7fbab40fc930 msgr2=0x7fbab4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 shutdown_connections 2023-09-23T05:14:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.011+0000 7fbab9bff700 1 -- 172.21.15.5:0/3304574353 wait complete. 2023-09-23T05:14:22.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T05:14:22.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 -- 172.21.15.5:0/1629342000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581010d0 msgr2=0x7f76581014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 --2- 172.21.15.5:0/1629342000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581010d0 0x7f76581014b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f76480098f0 tx=0x7f7648009c00 comp rx=0 tx=0).stop 2023-09-23T05:14:22.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 -- 172.21.15.5:0/1629342000 shutdown_connections 2023-09-23T05:14:22.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 --2- 172.21.15.5:0/1629342000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f765810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 --2- 172.21.15.5:0/1629342000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581010d0 0x7f76581014b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 --2- 172.21.15.5:0/1629342000 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f7658101a80 0x7f7658105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 -- 172.21.15.5:0/1629342000 >> 172.21.15.5:0/1629342000 conn(0x7f76580fc990 msgr2=0x7f76580fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 -- 172.21.15.5:0/1629342000 shutdown_connections 2023-09-23T05:14:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.237+0000 7f765e1ff700 1 -- 172.21.15.5:0/1629342000 wait complete. 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.238+0000 7f765e1ff700 1 Processor -- start 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.238+0000 7f765e1ff700 1 -- start start 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f7658073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658073ba0 0x7f76581a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5900 con 0x7f7658073ba0 2023-09-23T05:14:22.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5a70 con 0x7f76581061f0 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f765e1ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581a5c30 con 0x7f76581010d0 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54798/0 (socket says 172.21.15.5:54798) 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 -- 172.21.15.5:0/2376108671 learned_addr learned my addr 172.21.15.5:0/2376108671 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f7657fff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658073ba0 0x7f76581a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f764ffff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f7658073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 -- 172.21.15.5:0/2376108671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 msgr2=0x7f7658073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f7658073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 -- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658073ba0 msgr2=0x7f76581a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658073ba0 0x7f76581a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f76577fe700 1 -- 172.21.15.5:0/2376108671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400095f0 con 0x7f76581010d0 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.239+0000 7f764ffff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f7658073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f76577fe700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7648009810 tx=0x7f764800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764801c070 con 0x7f76581010d0 2023-09-23T05:14:22.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764800bb40 con 0x7f76581010d0 2023-09-23T05:14:22.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7648009520 con 0x7f76581010d0 2023-09-23T05:14:22.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480049c0 con 0x7f76581010d0 2023-09-23T05:14:22.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.240+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581a6140 con 0x7f76581010d0 2023-09-23T05:14:22.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.241+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658067ca0 con 0x7f76581010d0 2023-09-23T05:14:22.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.243+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f7648020070 con 0x7f76581010d0 2023-09-23T05:14:22.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.244+0000 7f76557fa700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 0x7f76440a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.245+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 9593+0+0 (secure 0 0 0) 0x7f76480cea90 con 0x7f76581010d0 2023-09-23T05:14:22.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.245+0000 7f764ffff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 0x7f76440a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.245+0000 7f764ffff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 0x7f76440a3bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7658074910 tx=0x7f7640009310 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.249+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f7648097940 con 0x7f76581010d0 2023-09-23T05:14:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-mon[97638]: pgmap v887: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-23T05:14:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3865675918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3304574353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mon[134297]: pgmap v887: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-23T05:14:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3865675918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3304574353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mon[133931]: pgmap v887: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-23T05:14:22.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3865675918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3304574353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-23T05:14:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.510+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7658101b20 con 0x7f76581010d0 2023-09-23T05:14:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.518+0000 7f76557fa700 1 -- 172.21.15.5:0/2376108671 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f764809af60 con 0x7f76581010d0 2023-09-23T05:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 msgr2=0x7f76440a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 0x7f76440a3bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7658074910 tx=0x7f7640009310 comp rx=0 tx=0).stop 2023-09-23T05:14:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 msgr2=0x7f7658073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7648009810 tx=0x7f764800fa00 comp rx=0 tx=0).stop 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 shutdown_connections 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f76440a1710 0x7f76440a3bd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658073ba0 0x7f76581a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f76581010d0 0x7f7658073120 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 --2- 172.21.15.5:0/2376108671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76581061f0 0x7f7658073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 >> 172.21.15.5:0/2376108671 conn(0x7f76580fc990 msgr2=0x7f76580fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 shutdown_connections 2023-09-23T05:14:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.522+0000 7f765e1ff700 1 -- 172.21.15.5:0/2376108671 wait complete. 2023-09-23T05:14:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T05:14:22.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T05:14:22.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 -- 172.21.15.5:0/3259664783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa030073ae0 msgr2=0x7fa030111930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 --2- 172.21.15.5:0/3259664783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa030073ae0 0x7fa030111930 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009990 tx=0x7fa01c009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 -- 172.21.15.5:0/3259664783 shutdown_connections 2023-09-23T05:14:22.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 --2- 172.21.15.5:0/3259664783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030111e70 0x7fa030114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 --2- 172.21.15.5:0/3259664783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0300731c0 0x7fa0300735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 --2- 172.21.15.5:0/3259664783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa030073ae0 0x7fa030111930 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 -- 172.21.15.5:0/3259664783 >> 172.21.15.5:0/3259664783 conn(0x7fa0300fc990 msgr2=0x7fa0300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:22.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 -- 172.21.15.5:0/3259664783 shutdown_connections 2023-09-23T05:14:22.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.770+0000 7fa036fff700 1 -- 172.21.15.5:0/3259664783 wait complete. 2023-09-23T05:14:22.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.771+0000 7fa036fff700 1 Processor -- start 2023-09-23T05:14:22.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 -- start start 2023-09-23T05:14:22.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030073ae0 0x7fa03019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa030111e70 0x7fa0301a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a2530 con 0x7fa0300731c0 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a26a0 con 0x7fa030111e70 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa036fff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301a2860 con 0x7fa030073ae0 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa034d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa034d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37134/0 (socket says 172.21.15.5:37134) 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa034d9b700 1 -- 172.21.15.5:0/3353672584 learned_addr learned my addr 172.21.15.5:0/3353672584 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:22.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa02ffff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030073ae0 0x7fa03019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.772+0000 7fa03559c700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa030111e70 0x7fa0301a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 -- 172.21.15.5:0/3353672584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030073ae0 msgr2=0x7fa03019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030073ae0 0x7fa03019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 -- 172.21.15.5:0/3353672584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa030111e70 msgr2=0x7fa0301a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa030111e70 0x7fa0301a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 -- 172.21.15.5:0/3353672584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0095f0 con 0x7fa0300731c0 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa034d9b700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9a0 tx=0x7fa02400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:22.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024009790 con 0x7fa0300731c0 2023-09-23T05:14:22.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301a2b40 con 0x7fa0300731c0 2023-09-23T05:14:22.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa024010430 con 0x7fa0300731c0 2023-09-23T05:14:22.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024018680 con 0x7fa0300731c0 2023-09-23T05:14:22.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.773+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301aa380 con 0x7fa0300731c0 2023-09-23T05:14:22.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.774+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa030104390 con 0x7fa0300731c0 2023-09-23T05:14:22.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.776+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7fa0240105a0 con 0x7fa0300731c0 2023-09-23T05:14:22.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.777+0000 7fa02dffb700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 0x7fa0200a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:22.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.778+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 9935+0+0 (secure 0 0 0) 0x7fa024014070 con 0x7fa0300731c0 2023-09-23T05:14:22.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.778+0000 7fa02ffff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 0x7fa0200a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:22.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.778+0000 7fa02ffff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 0x7fa0200a3bd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b2e0 tx=0x7fa01c005960 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:22.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:22.782+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7fa024051ec0 con 0x7fa0300731c0 2023-09-23T05:14:22.905 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.905 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.905 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-mds[149577]: -- 172.21.15.5:0/2256966162 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eac52bf1e0 con 0x55eac5294c00 2023-09-23T05:14:22.905 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: starting mds.a at 2023-09-23T05:14:22.906 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.906 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.906 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-mds[149186]: -- 172.21.15.5:0/2516961207 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623831bd1e0 con 0x562383193000 2023-09-23T05:14:22.906 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: starting mds.g at 2023-09-23T05:14:22.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a[97614]: 2023-09-23T05:14:22.511+0000 7f3276992700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T05:14:22.911 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.911 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.911 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mds[171064]: -- 172.21.15.118:0/3651456157 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d609ad1e0 con 0x555d60982c00 2023-09-23T05:14:22.912 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: starting mds.l at 2023-09-23T05:14:22.912 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.912 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.912 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mds[171754]: -- 172.21.15.118:0/1320774823 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603469371e0 con 0x56034690d000 2023-09-23T05:14:22.912 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: starting mds.c at 2023-09-23T05:14:22.913 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.913 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.913 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mds[171413]: -- 172.21.15.118:0/4192133247 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4beeeb1e0 con 0x55c4beec0800 2023-09-23T05:14:22.913 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: starting mds.f at 2023-09-23T05:14:22.913 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.914 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.914 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-mds[172092]: -- 172.21.15.118:0/4159672346 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0d5a051e0 con 0x55d0d59db000 2023-09-23T05:14:22.914 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:14:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: starting mds.i at 2023-09-23T05:14:22.991 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.991 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.991 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mds[170587]: -- 172.21.15.69:0/1756772762 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fac098b1e0 con 0x55fac0960800 2023-09-23T05:14:22.992 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: starting mds.b at 2023-09-23T05:14:22.992 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.992 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.993 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mds[169906]: -- 172.21.15.69:0/450536909 <== mon.1 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba10f991e0 con 0x55ba10f6f000 2023-09-23T05:14:22.993 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: starting mds.e at 2023-09-23T05:14:22.993 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.993 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.993 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mds[170248]: -- 172.21.15.69:0/2585745615 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bef5c6b1e0 con 0x55bef5c41000 2023-09-23T05:14:22.994 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: starting mds.h at 2023-09-23T05:14:22.994 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: ignoring --setuser ceph since I am not root 2023-09-23T05:14:22.994 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: ignoring --setgroup ceph since I am not root 2023-09-23T05:14:22.994 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-mds[169565]: -- 172.21.15.69:0/2805651637 <== mon.2 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561bec0691e0 con 0x561bec03f000 2023-09-23T05:14:22.994 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:14:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: starting mds.k at 2023-09-23T05:14:23.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.045+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa03019e9e0 con 0x7fa0300731c0 2023-09-23T05:14:23.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/2376108671' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T05:14:23.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T05:14:23.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T05:14:23.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: osdmap e135: 12 total, 12 up, 12 in 2023-09-23T05:14:23.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-23T05:14:23.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:23 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T05:14:23.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/2376108671' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T05:14:23.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T05:14:23.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T05:14:23.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: osdmap e135: 12 total, 12 up, 12 in 2023-09-23T05:14:23.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-23T05:14:23.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:23 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T05:14:23.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/2376108671' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: osdmap e135: 12 total, 12 up, 12 in 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-23T05:14:23.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:23 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-23T05:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.515+0000 7fa02dffb700 1 -- 172.21.15.5:0/3353672584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa024096780 con 0x7fa0300731c0 2023-09-23T05:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.518+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 msgr2=0x7fa0200a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.518+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 0x7fa0200a3bd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00b2e0 tx=0x7fa01c005960 comp rx=0 tx=0).stop 2023-09-23T05:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.518+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 msgr2=0x7fa03019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.518+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e9a0 tx=0x7fa02400ecb0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 shutdown_connections 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7fa0200a1710 0x7fa0200a3bd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa030111e70 0x7fa0301a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0300731c0 0x7fa03019d550 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 --2- 172.21.15.5:0/3353672584 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7fa030073ae0 0x7fa03019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 >> 172.21.15.5:0/3353672584 conn(0x7fa0300fc990 msgr2=0x7fa0300fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 shutdown_connections 2023-09-23T05:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.519+0000 7fa036fff700 1 -- 172.21.15.5:0/3353672584 wait complete. 2023-09-23T05:14:23.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T05:14:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 -- 172.21.15.5:0/4189624783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e4101a50 msgr2=0x7f04e4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 --2- 172.21.15.5:0/4189624783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e4101a50 0x7f04e4105a90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009990 tx=0x7f04cc009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 -- 172.21.15.5:0/4189624783 shutdown_connections 2023-09-23T05:14:23.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 --2- 172.21.15.5:0/4189624783 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 0x7f04e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 --2- 172.21.15.5:0/4189624783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e41010a0 0x7f04e4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 --2- 172.21.15.5:0/4189624783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e4101a50 0x7f04e4105a90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 -- 172.21.15.5:0/4189624783 >> 172.21.15.5:0/4189624783 conn(0x7f04e40fc920 msgr2=0x7f04e40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 -- 172.21.15.5:0/4189624783 shutdown_connections 2023-09-23T05:14:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.752+0000 7f04e99ff700 1 -- 172.21.15.5:0/4189624783 wait complete. 2023-09-23T05:14:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.753+0000 7f04e99ff700 1 Processor -- start 2023-09-23T05:14:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 -- start start 2023-09-23T05:14:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 0x7f04e41975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:23.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e419ba10 0x7f04e419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e419c490 con 0x7f04e41010a0 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e419c600 con 0x7f04e419ba10 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e99ff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e419c7c0 con 0x7f04e41061c0 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37154/0 (socket says 172.21.15.5:37154) 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e2ffd700 1 -- 172.21.15.5:0/946955244 learned_addr learned my addr 172.21.15.5:0/946955244 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04dbfff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 0x7f04e41975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:23.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.754+0000 7f04e37fe700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e419ba10 0x7f04e419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 -- 172.21.15.5:0/946955244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 msgr2=0x7f04e41975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 0x7f04e41975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 -- 172.21.15.5:0/946955244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e419ba10 msgr2=0x7f04e419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e419ba10 0x7f04e419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 -- 172.21.15.5:0/946955244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4009520 con 0x7f04e41010a0 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e2ffd700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f04d400eb30 tx=0x7f04d400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4009880 con 0x7f04e41010a0 2023-09-23T05:14:23.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc0095f0 con 0x7f04e41010a0 2023-09-23T05:14:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d40044e0 con 0x7f04e41010a0 2023-09-23T05:14:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003d80 con 0x7f04e41010a0 2023-09-23T05:14:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.755+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e41a40c0 con 0x7f04e41010a0 2023-09-23T05:14:23.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.756+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e40680f0 con 0x7f04e41010a0 2023-09-23T05:14:23.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.758+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f04d4003f20 con 0x7f04e41010a0 2023-09-23T05:14:23.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.760+0000 7f04e89fd700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 0x7f04d00a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:23.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.760+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 9935+0+0 (secure 0 0 0) 0x7f04d4014070 con 0x7f04e41010a0 2023-09-23T05:14:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.760+0000 7f04dbfff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 0x7f04d00a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.761+0000 7f04dbfff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 0x7f04d00a3c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f04e4198680 tx=0x7f04cc005fb0 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:23.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.763+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f04d4096ab0 con 0x7f04e41010a0 2023-09-23T05:14:23.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:23.983+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04e4101af0 con 0x7f04e41010a0 2023-09-23T05:14:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: pgmap v889: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-23T05:14:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T05:14:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T05:14:24.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: Cluster is now healthy 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6832/2836474782,v1:172.21.15.69:6833/2836474782] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6836/1784019605,v1:172.21.15.69:6838/1784019605] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6837/4204761889,v1:172.21.15.69:6839/4204761889] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.69:6834/3714386816,v1:172.21.15.69:6835/3714386816] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6837/2890347960,v1:172.21.15.118:6839/2890347960] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6838/3142165150,v1:172.21.15.5:6839/3142165150] up:boot 2023-09-23T05:14:24.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6833/4205382370,v1:172.21.15.118:6835/4205382370] up:boot 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6832/1507073060,v1:172.21.15.118:6834/1507073060] up:boot 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.5:6840/2650000822,v1:172.21.15.5:6841/2650000822] up:boot 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: mds.? [v2:172.21.15.118:6836/3113563486,v1:172.21.15.118:6838/3113563486] up:boot 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: fsmap 12 up:standby 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T05:14:24.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T05:14:24.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:24 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: pgmap v889: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: Cluster is now healthy 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T05:14:24.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6832/2836474782,v1:172.21.15.69:6833/2836474782] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6836/1784019605,v1:172.21.15.69:6838/1784019605] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6837/4204761889,v1:172.21.15.69:6839/4204761889] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.69:6834/3714386816,v1:172.21.15.69:6835/3714386816] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6837/2890347960,v1:172.21.15.118:6839/2890347960] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6838/3142165150,v1:172.21.15.5:6839/3142165150] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6833/4205382370,v1:172.21.15.118:6835/4205382370] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6832/1507073060,v1:172.21.15.118:6834/1507073060] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.5:6840/2650000822,v1:172.21.15.5:6841/2650000822] up:boot 2023-09-23T05:14:24.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: mds.? [v2:172.21.15.118:6836/3113563486,v1:172.21.15.118:6838/3113563486] up:boot 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: fsmap 12 up:standby 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T05:14:24.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T05:14:24.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T05:14:24.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T05:14:24.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T05:14:24.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:24 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:24.491 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: pgmap v889: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: Cluster is now healthy 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3353672584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6832/2836474782,v1:172.21.15.69:6833/2836474782] up:boot 2023-09-23T05:14:24.492 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6836/1784019605,v1:172.21.15.69:6838/1784019605] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6837/4204761889,v1:172.21.15.69:6839/4204761889] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.69:6834/3714386816,v1:172.21.15.69:6835/3714386816] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6837/2890347960,v1:172.21.15.118:6839/2890347960] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6838/3142165150,v1:172.21.15.5:6839/3142165150] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6833/4205382370,v1:172.21.15.118:6835/4205382370] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6832/1507073060,v1:172.21.15.118:6834/1507073060] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.5:6840/2650000822,v1:172.21.15.5:6841/2650000822] up:boot 2023-09-23T05:14:24.493 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: mds.? [v2:172.21.15.118:6836/3113563486,v1:172.21.15.118:6838/3113563486] up:boot 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: fsmap 12 up:standby 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-23T05:14:24.494 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-23T05:14:24.495 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-23T05:14:24.495 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-23T05:14:24.495 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='mgr.15594 172.21.15.5:0/467991127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-23T05:14:24.495 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:24 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:24.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.517+0000 7f04e89fd700 1 -- 172.21.15.5:0/946955244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7f04d409a0d0 con 0x7f04e41010a0 2023-09-23T05:14:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 msgr2=0x7f04d00a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 0x7f04d00a3c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f04e4198680 tx=0x7f04cc005fb0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 msgr2=0x7f04e4197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f04d400eb30 tx=0x7f04d400ee40 comp rx=0 tx=0).stop 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 shutdown_connections 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f04d00a1790 0x7f04d00a3c50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e419ba10 0x7f04e419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41010a0 0x7f04e4197070 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 --2- 172.21.15.5:0/946955244 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f04e41061c0 0x7f04e41975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 >> 172.21.15.5:0/946955244 conn(0x7f04e40fc920 msgr2=0x7f04e40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 shutdown_connections 2023-09-23T05:14:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.520+0000 7f04e99ff700 1 -- 172.21.15.5:0/946955244 wait complete. 2023-09-23T05:14:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2023-09-23T05:14:24.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T05:14:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 -- 172.21.15.5:0/3429968531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48201061f0 msgr2=0x7f482010dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 --2- 172.21.15.5:0/3429968531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48201061f0 0x7f482010dd70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f481800b0a0 tx=0x7f481800b3b0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 -- 172.21.15.5:0/3429968531 shutdown_connections 2023-09-23T05:14:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 --2- 172.21.15.5:0/3429968531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48201061f0 0x7f482010dd70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 --2- 172.21.15.5:0/3429968531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 --2- 172.21.15.5:0/3429968531 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f4820101a80 0x7f4820105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 -- 172.21.15.5:0/3429968531 >> 172.21.15.5:0/3429968531 conn(0x7f48200fc990 msgr2=0x7f48200fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 -- 172.21.15.5:0/3429968531 shutdown_connections 2023-09-23T05:14:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.748+0000 7f4827dff700 1 -- 172.21.15.5:0/3429968531 wait complete. 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.749+0000 7f4827dff700 1 Processor -- start 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 -- start start 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48201061f0 0x7f48201997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f482019dc50 0x7f482019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482019e6d0 con 0x7f482019dc50 2023-09-23T05:14:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482019e840 con 0x7f48201010d0 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4827dff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482019ea00 con 0x7f48201061f0 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4825b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f482539a700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48201061f0 0x7f48201997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4825b9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.5:37608/0 (socket says 172.21.15.5:37608) 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f4825b9b700 1 -- 172.21.15.5:0/1542459876 learned_addr learned my addr 172.21.15.5:0/1542459876 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.750+0000 7f482639c700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f482019dc50 0x7f482019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 -- 172.21.15.5:0/1542459876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48201061f0 msgr2=0x7f48201997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48201061f0 0x7f48201997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 -- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f482019dc50 msgr2=0x7f482019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f482019dc50 0x7f482019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 -- 172.21.15.5:0/1542459876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009d00 con 0x7f48201010d0 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4825b9b700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9a0 tx=0x7f481c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c009790 con 0x7f48201010d0 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c010430 con 0x7f48201010d0 2023-09-23T05:14:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48201a5f60 con 0x7f48201010d0 2023-09-23T05:14:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.751+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c018680 con 0x7f48201010d0 2023-09-23T05:14:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.752+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48201a63e0 con 0x7f48201010d0 2023-09-23T05:14:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.752+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820067ca0 con 0x7f48201010d0 2023-09-23T05:14:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.755+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f481c0105a0 con 0x7f48201010d0 2023-09-23T05:14:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.756+0000 7f4816ffd700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 0x7f480c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.756+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 9352+0+0 (secure 0 0 0) 0x7f481c014070 con 0x7f48201010d0 2023-09-23T05:14:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.756+0000 7f482539a700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 0x7f480c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.757+0000 7f482539a700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 0x7f480c0a3bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f482019a8c0 tx=0x7f4810009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.760+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f481c096930 con 0x7f48201010d0 2023-09-23T05:14:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:24.976+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4820101bc0 con 0x7f48201010d0 2023-09-23T05:14:25.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.530+0000 7f4816ffd700 1 -- 172.21.15.5:0/1542459876 <== mon.2 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) v1 ==== 169+0+0 (secure 0 0 0) 0x7f481c099f50 con 0x7f48201010d0 2023-09-23T05:14:25.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.536+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 msgr2=0x7f480c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:25.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.536+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 0x7f480c0a3bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f482019a8c0 tx=0x7f4810009260 comp rx=0 tx=0).stop 2023-09-23T05:14:25.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.536+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 msgr2=0x7f48201992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:25.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.536+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9a0 tx=0x7f481c00ecb0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 shutdown_connections 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f480c0a1710 0x7f480c0a3bd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f482019dc50 0x7f482019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48201010d0 0x7f48201992b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 --2- 172.21.15.5:0/1542459876 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f48201061f0 0x7f48201997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.540+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 >> 172.21.15.5:0/1542459876 conn(0x7f48200fc990 msgr2=0x7f48200fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:25.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.543+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 shutdown_connections 2023-09-23T05:14:25.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.546+0000 7f4827dff700 1 -- 172.21.15.5:0/1542459876 wait complete. 2023-09-23T05:14:25.562 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2023-09-23T05:14:25.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-23T05:14:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:25 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:25 smithi005 ceph-mon[97638]: osdmap e136: 12 total, 12 up, 12 in 2023-09-23T05:14:25.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:25 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/1542459876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:25 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:25 smithi005 ceph-mon[97638]: pgmap v891: 97 pgs: 97 active+clean; 238 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T05:14:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:25 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:25 smithi118 ceph-mon[134297]: osdmap e136: 12 total, 12 up, 12 in 2023-09-23T05:14:25.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:25 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/1542459876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:25 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.912 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:25 smithi118 ceph-mon[134297]: pgmap v891: 97 pgs: 97 active+clean; 238 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T05:14:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.975+0000 7f134adff700 1 -- 172.21.15.5:0/2661154567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13441005f0 msgr2=0x7f134410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.975+0000 7f134adff700 1 --2- 172.21.15.5:0/2661154567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13441005f0 0x7f134410d2b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1330009990 tx=0x7f1330009ca0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 -- 172.21.15.5:0/2661154567 shutdown_connections 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 --2- 172.21.15.5:0/2661154567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f134410d7f0 0x7f134410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 --2- 172.21.15.5:0/2661154567 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13440ffc40 0x7f1344100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 --2- 172.21.15.5:0/2661154567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13441005f0 0x7f134410d2b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 -- 172.21.15.5:0/2661154567 >> 172.21.15.5:0/2661154567 conn(0x7f1344076f30 msgr2=0x7f1344077340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 -- 172.21.15.5:0/2661154567 shutdown_connections 2023-09-23T05:14:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.976+0000 7f134adff700 1 -- 172.21.15.5:0/2661154567 wait complete. 2023-09-23T05:14:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.977+0000 7f134adff700 1 Processor -- start 2023-09-23T05:14:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.978+0000 7f134adff700 1 -- start start 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 0x7f134410b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 0x7f13441a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134410c470 con 0x7f13440ffc40 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13441a9ac0 con 0x7f134410d7f0 2023-09-23T05:14:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134adff700 1 -- --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13441a9c00 con 0x7f13441005f0 2023-09-23T05:14:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f1348b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 0x7f134410b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f1343fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f1343fff700 1 --2- >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.118:3300/0 says I am v2:172.21.15.5:54886/0 (socket says 172.21.15.5:54886) 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f1343fff700 1 -- 172.21.15.5:0/3216089759 learned_addr learned my addr 172.21.15.5:0/3216089759 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f1348b9b700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 0x7f134410b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37192/0 (socket says 172.21.15.5:37192) 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.979+0000 7f134939c700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 0x7f13441a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f1343fff700 1 -- 172.21.15.5:0/3216089759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 msgr2=0x7f13441a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f1343fff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 0x7f13441a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f1343fff700 1 -- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 msgr2=0x7f134410b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f1343fff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 0x7f134410b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f1343fff700 1 -- 172.21.15.5:0/3216089759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300095f0 con 0x7f13441005f0 2023-09-23T05:14:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.980+0000 7f134939c700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 0x7f13441a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T05:14:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f1343fff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1330005910 tx=0x7f1330005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133001d070 con 0x7f13441005f0 2023-09-23T05:14:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1330022470 con 0x7f13441005f0 2023-09-23T05:14:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330017650 con 0x7f13441005f0 2023-09-23T05:14:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13441a9e00 con 0x7f13441005f0 2023-09-23T05:14:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.981+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13441aa1d0 con 0x7f13441005f0 2023-09-23T05:14:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.983+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1344067ca0 con 0x7f13441005f0 2023-09-23T05:14:25.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.986+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 4 ==== mgrmap(e 26) v1 ==== 136322+0+0 (secure 0 0 0) 0x7f1330022850 con 0x7f13441005f0 2023-09-23T05:14:25.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.988+0000 7f1341ffb700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 0x7f13340a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T05:14:25.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.988+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8682+0+0 (secure 0 0 0) 0x7f13300ce7b0 con 0x7f13441005f0 2023-09-23T05:14:25.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.989+0000 7f1348b9b700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 0x7f13340a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T05:14:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.990+0000 7f1348b9b700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 0x7f13340a3c50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1338009c60 tx=0x7f1338009260 comp rx=0 tx=0).ready entity=mgr.15594 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T05:14:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:25 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/946955244' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:25 smithi069 ceph-mon[133931]: osdmap e136: 12 total, 12 up, 12 in 2023-09-23T05:14:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:25 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/1542459876' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:25 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:25.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:25 smithi069 ceph-mon[133931]: pgmap v891: 97 pgs: 97 active+clean; 238 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-23T05:14:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:25.992+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179873 (secure 0 0 0) 0x7f13300979f0 con 0x7f13441005f0 2023-09-23T05:14:26.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:26.233+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 --> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f134410cc10 con 0x7f13441005f0 2023-09-23T05:14:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:26 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:26 smithi005 ceph-mon[97638]: osdmap e137: 12 total, 12 up, 12 in 2023-09-23T05:14:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:26 smithi005 ceph-mon[97638]: from='client.? 172.21.15.5:0/3216089759' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:26.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:26 smithi005 ceph-mon[97638]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:26.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:26 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:26.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:26 smithi118 ceph-mon[134297]: osdmap e137: 12 total, 12 up, 12 in 2023-09-23T05:14:26.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:26 smithi118 ceph-mon[134297]: from='client.? 172.21.15.5:0/3216089759' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:26.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:26 smithi118 ceph-mon[134297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:26.991 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:26 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-23T05:14:26.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:26 smithi069 ceph-mon[133931]: osdmap e137: 12 total, 12 up, 12 in 2023-09-23T05:14:26.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:26 smithi069 ceph-mon[133931]: from='client.? 172.21.15.5:0/3216089759' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:26.992 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:26 smithi069 ceph-mon[133931]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-23T05:14:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.107+0000 7f1341ffb700 1 -- 172.21.15.5:0/3216089759 <== mon.1 v2:172.21.15.118:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v138) v1 ==== 178+0+0 (secure 0 0 0) 0x7f133009b010 con 0x7f13441005f0 2023-09-23T05:14:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 msgr2=0x7f13340a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 0x7f13340a3c50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1338009c60 tx=0x7f1338009260 comp rx=0 tx=0).stop 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 msgr2=0x7f134410bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1330005910 tx=0x7f1330005740 comp rx=0 tx=0).stop 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 shutdown_connections 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:6800/2064728089,v1:172.21.15.5:6801/2064728089] conn(0x7f13340a1790 0x7f13340a3c50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134410d7f0 0x7f13441a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13440ffc40 0x7f134410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 --2- 172.21.15.5:0/3216089759 >> [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] conn(0x7f13441005f0 0x7f134410bc20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T05:14:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 >> 172.21.15.5:0/3216089759 conn(0x7f1344076f30 msgr2=0x7f134410ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T05:14:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 shutdown_connections 2023-09-23T05:14:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-09-23T05:14:27.110+0000 7f134adff700 1 -- 172.21.15.5:0/3216089759 wait complete. 2023-09-23T05:14:27.112 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2023-09-23T05:14:27.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:14:27.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:14:27.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:14:27.340 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-23T05:14:27.341 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T05:14:27.385 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T05:14:27.402 DEBUG:teuthology.orchestra.run.smithi118:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-23T05:14:27.424 INFO:tasks.cephadm:Stopping all daemons... 2023-09-23T05:14:27.425 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-23T05:14:27.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a 2023-09-23T05:14:27.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:27 smithi005 systemd[1]: Stopping Ceph mon.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a[97614]: 2023-09-23T05:14:27.944+0000 7f327c19d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:28.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a[97614]: 2023-09-23T05:14:27.944+0000 7f327c19d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-23T05:14:28.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 23 05:14:28 smithi005 bash[190427]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-a 2023-09-23T05:14:28.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.a.service' 2023-09-23T05:14:28.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:28.947 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-23T05:14:28.948 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-23T05:14:28.948 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.b 2023-09-23T05:14:29.241 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:29 smithi069 systemd[1]: Stopping Ceph mon.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:29.808 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-b[133907]: 2023-09-23T05:14:29.490+0000 7f2be12c2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:29.809 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-b[133907]: 2023-09-23T05:14:29.490+0000 7f2be12c2700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-23T05:14:30.067 INFO:journalctl@ceph.mon.b.smithi069.stdout:Sep 23 05:14:29 smithi069 bash[185525]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-b 2023-09-23T05:14:30.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.b.service' 2023-09-23T05:14:30.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:30.139 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-23T05:14:30.139 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-23T05:14:30.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.c 2023-09-23T05:14:30.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:30 smithi118 systemd[1]: Stopping Ceph mon.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:30.980 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-c[134273]: 2023-09-23T05:14:30.727+0000 7f26a378f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:30.980 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-c[134273]: 2023-09-23T05:14:30.727+0000 7f26a378f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-23T05:14:31.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Sep 23 05:14:31 smithi118 bash[185925]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mon-c 2023-09-23T05:14:31.423 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mon.c.service' 2023-09-23T05:14:31.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:31.484 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-23T05:14:31.485 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-23T05:14:31.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x 2023-09-23T05:14:31.905 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 05:14:31 smithi005 systemd[1]: Stopping Ceph mgr.x for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:32.270 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 05:14:31 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x[97888]: 2023-09-23T05:14:31.985+0000 7f68a54f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:32.655 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 23 05:14:32 smithi005 bash[190570]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-x 2023-09-23T05:14:32.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.x.service' 2023-09-23T05:14:32.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:32.894 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-23T05:14:32.894 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-23T05:14:32.894 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.y 2023-09-23T05:14:33.241 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 05:14:32 smithi069 systemd[1]: Stopping Ceph mgr.y for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:33.600 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 05:14:33 smithi069 bash[185668]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-y 2023-09-23T05:14:33.600 INFO:journalctl@ceph.mgr.y.smithi069.stdout:Sep 23 05:14:33 smithi069 systemd[1]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-23T05:14:33.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.y.service' 2023-09-23T05:14:33.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:33.819 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-23T05:14:33.819 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-23T05:14:33.819 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.z 2023-09-23T05:14:34.161 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 05:14:33 smithi118 systemd[1]: Stopping Ceph mgr.z for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:34.807 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 05:14:34 smithi118 bash[186074]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mgr-z 2023-09-23T05:14:34.808 INFO:journalctl@ceph.mgr.z.smithi118.stdout:Sep 23 05:14:34 smithi118 systemd[1]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-23T05:14:34.814 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mgr.z.service' 2023-09-23T05:14:34.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:34.850 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-23T05:14:34.851 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-23T05:14:34.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.0 2023-09-23T05:14:35.369 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 05:14:34 smithi005 systemd[1]: Stopping Ceph osd.0 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:35.655 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 05:14:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T05:14:35.368+0000 7f04b4623700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:35.655 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 05:14:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T05:14:35.368+0000 7f04b4623700 -1 osd.0 138 *** Got signal Terminated *** 2023-09-23T05:14:35.655 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 05:14:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0[116640]: 2023-09-23T05:14:35.368+0000 7f04b4623700 -1 osd.0 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:14:41.367 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 23 05:14:41 smithi005 bash[190716]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-0 2023-09-23T05:14:43.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.0.service' 2023-09-23T05:14:43.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:43.196 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-23T05:14:43.196 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-23T05:14:43.196 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.1 2023-09-23T05:14:43.491 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 05:14:43 smithi069 systemd[1]: Stopping Ceph osd.1 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:43.991 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 05:14:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T05:14:43.567+0000 7fd287fdd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:43.992 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 05:14:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T05:14:43.567+0000 7fd287fdd700 -1 osd.1 138 *** Got signal Terminated *** 2023-09-23T05:14:43.992 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 05:14:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1[138984]: 2023-09-23T05:14:43.567+0000 7fd287fdd700 -1 osd.1 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:14:49.991 INFO:journalctl@ceph.osd.1.smithi069.stdout:Sep 23 05:14:49 smithi069 bash[185817]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-1 2023-09-23T05:14:51.172 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.1.service' 2023-09-23T05:14:51.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:51.213 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-23T05:14:51.213 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-23T05:14:51.213 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.2 2023-09-23T05:14:51.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 05:14:51 smithi118 systemd[1]: Stopping Ceph osd.2 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:14:52.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 05:14:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T05:14:51.888+0000 7f0047259700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:14:52.162 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 05:14:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T05:14:51.888+0000 7f0047259700 -1 osd.2 138 *** Got signal Terminated *** 2023-09-23T05:14:52.162 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 05:14:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2[140085]: 2023-09-23T05:14:51.888+0000 7f0047259700 -1 osd.2 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:14:57.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Sep 23 05:14:57 smithi118 bash[186223]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-2 2023-09-23T05:14:59.313 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.2.service' 2023-09-23T05:14:59.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:14:59.354 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-23T05:14:59.354 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-23T05:14:59.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.3 2023-09-23T05:14:59.867 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:14:59 smithi005 systemd[1]: Stopping Ceph osd.3 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:00.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:14:59 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:14:59.866+0000 7fba13548700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:00.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:14:59 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:14:59.866+0000 7fba13548700 -1 osd.3 138 *** Got signal Terminated *** 2023-09-23T05:15:00.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:14:59 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:14:59.866+0000 7fba13548700 -1 osd.3 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:02.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:15:01 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:15:01.692+0000 7fba0bb36700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:37.259207+0000 front 2023-09-23T05:14:37.259164+0000 (oldest deadline 2023-09-23T05:15:01.260260+0000) 2023-09-23T05:15:02.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:02.110+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:02.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:02 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:02.281+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:03.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:15:02 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:15:02.693+0000 7fba0bb36700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:37.259207+0000 front 2023-09-23T05:14:37.259164+0000 (oldest deadline 2023-09-23T05:15:01.260260+0000) 2023-09-23T05:15:03.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:03.093+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:03.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:03 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:03.274+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:04.154 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:15:03 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:15:03.666+0000 7fba0bb36700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:37.259207+0000 front 2023-09-23T05:14:37.259164+0000 (oldest deadline 2023-09-23T05:15:01.260260+0000) 2023-09-23T05:15:04.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:04.085+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:04.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:04 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:04.266+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:04.911 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:04.428+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:05.155 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:15:04 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3[124454]: 2023-09-23T05:15:04.712+0000 7fba0bb36700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:37.259207+0000 front 2023-09-23T05:14:37.259164+0000 (oldest deadline 2023-09-23T05:15:01.260260+0000) 2023-09-23T05:15:05.388 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:05 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:05.082+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:05.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:05 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:05.387+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:05.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:05 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:05.248+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:05.905 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 23 05:15:05 smithi005 bash[190987]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-3 2023-09-23T05:15:06.380 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:06.119+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:06.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:06.378+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:06.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:06 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:06.245+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:07.376 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:07.079+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:07.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.3.service' 2023-09-23T05:15:07.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:07 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:07.375+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:07.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:07.684 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-23T05:15:07.685 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-23T05:15:07.685 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.4 2023-09-23T05:15:07.715 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:07 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:07.250+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:07.991 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:07 smithi069 systemd[1]: Stopping Ceph osd.4 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:08.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:08.104+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:08.412 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:08 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:08.343+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:08.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:08.211+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:08.492 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T05:15:08.045+0000 7f90be0d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:08.492 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T05:15:08.045+0000 7f90be0d2700 -1 osd.4 138 *** Got signal Terminated *** 2023-09-23T05:15:08.492 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:08 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T05:15:08.045+0000 7f90be0d2700 -1 osd.4 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:09.387 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:09.070+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:09.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:09.386+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:09.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:09 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:09.386+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:09.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:09 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:09.260+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:10.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:09.881+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:10.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:09 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:09.776+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:10.338 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:10 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:10.062+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:10.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:10 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:10.338+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:10.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:10 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:10.338+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:10.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:10 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:10.256+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:10.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:10 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:10.256+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:11.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:10 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:10.857+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:11.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:10 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:10.752+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:11.382 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:11.032+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:11.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:11.382+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:11.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:11 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:11.382+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:11.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:11.285+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:11.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:11.285+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:12.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:11 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:11.764+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:12.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:11 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:11.906+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:12.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:11.785+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:12.241 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:11 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T05:15:11.756+0000 7f90ba6c8700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.327818+0000 front 2023-09-23T05:14:47.327772+0000 (oldest deadline 2023-09-23T05:15:10.828012+0000) 2023-09-23T05:15:12.353 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:12.047+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:12.353 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:12.047+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:12.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:12.352+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:12.662 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:12.352+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:12.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:12.333+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:12.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:12.333+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:13.068 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:12 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:12.667+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:13.156 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:12 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:12.761+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:13.156 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:12 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:12.868+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:13.241 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4[147013]: 2023-09-23T05:15:12.779+0000 7f90ba6c8700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.327818+0000 front 2023-09-23T05:14:47.327772+0000 (oldest deadline 2023-09-23T05:15:10.828012+0000) 2023-09-23T05:15:13.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:12 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:12.746+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:13.396 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:13.067+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:13.396 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:13.067+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:13.661 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:13.631+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:13.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:13.395+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:13.662 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:13 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:13.395+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:13.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:13 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:13.327+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:13.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:13 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:13.327+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:14.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:13 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:13.727+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:14.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:13 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:13.841+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:14.242 INFO:journalctl@ceph.osd.4.smithi069.stdout:Sep 23 05:15:13 smithi069 bash[186094]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-4 2023-09-23T05:15:14.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:13 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:13.740+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:14.383 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:14.115+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:14.384 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:14.115+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:14.645 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:14.383+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:14.645 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:14.383+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:14.738 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:14.313+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:14.738 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:14.314+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:14.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:14 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:14.645+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:14.993 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:14 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:14.755+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:15.082 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.4.service' 2023-09-23T05:15:15.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:14.683+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:15.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:14 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:14.863+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:15.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:15.163 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-23T05:15:15.163 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-23T05:15:15.163 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.5 2023-09-23T05:15:15.411 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 systemd[1]: Stopping Ceph osd.5 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:15.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:15.150+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:15.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:15.150+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:15.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:15 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:15.282+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:15.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:15 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:15.282+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:15.688 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:15.425+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:15.689 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:15.425+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:15.689 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:15.517+0000 7f35e356d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:15.689 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:15.517+0000 7f35e356d700 -1 osd.5 138 *** Got signal Terminated *** 2023-09-23T05:15:15.689 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:15.517+0000 7f35e356d700 -1 osd.5 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:15.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:15.637+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:15.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:15.816+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:16.146 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:15 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:15.688+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:16.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:15 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:15.772+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:16.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:16 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:16.145+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:16.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:16 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:16.145+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:16.691 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:16 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:16.440+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:16.691 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:16 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:16.440+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:16.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:16 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:16.256+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:16.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:16 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:16.256+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:17.124 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:16 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:16.690+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:17.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:16.673+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:17.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:16 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:16.844+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:17.240 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:16 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:16.759+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:17.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:17.123+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:17.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:17.123+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:17.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:17 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:17.239+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:17.491 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:17 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:17.239+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:17.697 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:17.429+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:17.698 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:17.429+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:18.104 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:17.697+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:18.104 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:17 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:17.697+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:18.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:17 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:17.670+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:18.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:17 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:17.867+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:18.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:17 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:17.804+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:18.385 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:18.103+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:18.386 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:18.103+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:18.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:18.385+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:18.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:18.385+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:18.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:18.385+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.923832+0000 front 2023-09-23T05:14:54.923859+0000 (oldest deadline 2023-09-23T05:15:17.823960+0000) 2023-09-23T05:15:18.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:18 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:18.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:18.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:18 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:18.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:19.059 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:18.695+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:19.060 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:18 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:18.695+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:19.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:18.656+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:19.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:18.656+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:19.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:18.874+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:19.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:18 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:18.824+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:19.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:19.059+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:19.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:19.059+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:19.737 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:19.416+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:19.737 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:19.416+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:19.738 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:19.416+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.923832+0000 front 2023-09-23T05:14:54.923859+0000 (oldest deadline 2023-09-23T05:15:17.823960+0000) 2023-09-23T05:15:19.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:19 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:19.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:19.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:19 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:19.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:19.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:19 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:19.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:20.099 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:19.736+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:20.099 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:19 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:19.736+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:20.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:19.672+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:20.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:19.672+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:20.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:19.885+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:20.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:19 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:19.885+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:20.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:19 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:19.841+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:20.395 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:20.098+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:20.395 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:20.098+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:20.395 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:20.098+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:20.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:20.394+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:39.320643+0000 front 2023-09-23T05:14:39.320709+0000 (oldest deadline 2023-09-23T05:15:04.020510+0000) 2023-09-23T05:15:20.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:20.394+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.321706+0000 front 2023-09-23T05:14:46.321859+0000 (oldest deadline 2023-09-23T05:15:09.221611+0000) 2023-09-23T05:15:20.662 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5[148767]: 2023-09-23T05:15:20.394+0000 7f35dfb63700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.923832+0000 front 2023-09-23T05:14:54.923859+0000 (oldest deadline 2023-09-23T05:15:17.823960+0000) 2023-09-23T05:15:20.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:20 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:20.309+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:20.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:20 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:20.309+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:20.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:20 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:20.309+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:21.107 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:20.754+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:21.108 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:20 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:20.754+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:21.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:20.670+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:21.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:20.670+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:21.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:20.859+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:21.156 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:20 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:20.859+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:21.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:20 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:20.846+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:21.400 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:21 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:21.106+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:21.400 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:21 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:21.106+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:21.400 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:21 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:21.106+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:21.661 INFO:journalctl@ceph.osd.5.smithi118.stdout:Sep 23 05:15:21 smithi118 bash[186495]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-5 2023-09-23T05:15:21.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:21 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:21.313+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:21.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:21 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:21.313+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:21.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:21 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:21.313+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:21.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:21.633+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:21.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:21.633+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:21.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:21.857+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:21.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:21 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:21.857+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:22.062 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:21 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:21.787+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:22.063 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:21 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:21.787+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:22.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:21 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:21.855+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:22.326 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:22.062+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:22.326 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:22.062+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:22.327 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:22.062+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:22.663 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.5.service' 2023-09-23T05:15:22.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:22.703 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-23T05:15:22.703 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-23T05:15:22.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.6 2023-09-23T05:15:22.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:22.331+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:22.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:22.331+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:22.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:22.331+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:22.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:22.818+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:22.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:22.818+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:22.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:22.647+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:22.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:22 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:22.647+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:22.906 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:22 smithi005 systemd[1]: Stopping Ceph osd.6 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:22.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:22.795+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:22.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:22 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:22.795+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:23.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:22 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:22.869+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:23.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.028+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:23.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.028+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:23.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.028+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:23.655 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:23.219+0000 7fb8b0181700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:23.655 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:23.219+0000 7fb8b0181700 -1 osd.6 138 *** Got signal Terminated *** 2023-09-23T05:15:23.655 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:23.220+0000 7fb8b0181700 -1 osd.6 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:23.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:23.326+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:23.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:23.326+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:23.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:23.326+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:24.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:23.662+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:24.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:23.662+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:24.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:23.836+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:24.156 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:23.836+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:24.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:23.755+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:24.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:23.755+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:24.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:24.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:24.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:23 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:23.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:24.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:23.859+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:24.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:23 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:23.859+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:24.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:24 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:24.363+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:24.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:24 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:24.363+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:24.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:24 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:24.363+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:24.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:24 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:24.633+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:24.905 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:24 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:24.633+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:24.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:24 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:24.827+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:24.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:24 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:24.827+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:24.977 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:24 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:24.712+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:24.977 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:24 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:24.712+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:25.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:24 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:24.897+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:25.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:24 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:24.897+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:25.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:24 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:24.977+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:25.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:24 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:24.977+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:25.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:24 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:24.977+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:25.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:25 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:25.343+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:25.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:25 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:25.343+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:25.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:25 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:25.343+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:26.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:25 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:25.671+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:26.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:25 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:25.671+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:26.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:25 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:25.798+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:26.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:25 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:25.798+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:26.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:25.715+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:26.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:25.715+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:26.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:25.965+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:26.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:25.965+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:26.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:25 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:25.965+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:26.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:25 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:25.849+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:26.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:25 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:25.849+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:26.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:26 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:26.323+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:26.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:26 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:26.323+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:26.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:26 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:26.323+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:27.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:26 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:26.696+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:27.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:26 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:26.696+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:27.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:26 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:26.779+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:27.156 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:26 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:26.779+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:27.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:26.758+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:27.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:26.758+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:27.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:26.950+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:27.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:26.950+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:27.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:26 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:26.950+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:27.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:26 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:26.835+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:27.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:26 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:26.835+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:27.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:27.347+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:27.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:27.347+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:27.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:27.347+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:28.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:27.723+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.081874+0000 front 2023-09-23T05:14:45.081839+0000 (oldest deadline 2023-09-23T05:15:10.981464+0000) 2023-09-23T05:15:28.155 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6[132431]: 2023-09-23T05:15:27.723+0000 7fb8aa773700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.383112+0000 front 2023-09-23T05:14:54.383081+0000 (oldest deadline 2023-09-23T05:15:18.483358+0000) 2023-09-23T05:15:28.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:27.750+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:28.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:27 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:27.750+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:28.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:27.739+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:28.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:27.739+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:28.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:27.739+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:28.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:27.931+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:28.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:27.931+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:28.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:27 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:27.931+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:28.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:27.802+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:28.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:27.802+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:28.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:27.802+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:28.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:28.376+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:28.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:28.376+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:28.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:28.376+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:29.017 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 23 05:15:28 smithi005 bash[191264]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-6 2023-09-23T05:15:29.017 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:28 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:28.745+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:29.018 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:28 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:28.745+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:29.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:28.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:29.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:28.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:29.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:28.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:29.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:28.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:29.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:28.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:29.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:28 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:28.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:29.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:28.795+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:29.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:28.795+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:29.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:28 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:28.795+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:29.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:29.348+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:29.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:29.348+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:29.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:29.348+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:30.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:29 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:29.735+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:30.157 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:29 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:29.735+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:30.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:29.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:30.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:29.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:30.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:29.720+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:30.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:29.901+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:30.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:29.901+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:30.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:29 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:29.901+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:30.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:29.746+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:30.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:29.746+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:30.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:29 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:29.746+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:30.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:30.393+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:30.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:30.393+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:30.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:30.393+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:30.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:30.393+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:30.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.6.service' 2023-09-23T05:15:30.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:30.922 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-23T05:15:30.923 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-23T05:15:30.923 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.7 2023-09-23T05:15:31.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:30 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:30.767+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:31.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:30 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:30.767+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:31.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:30 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:30.767+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:31.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:30.694+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:31.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:30.694+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:31.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:30.694+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:31.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:30.912+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:31.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:30.912+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:31.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:30 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:30.912+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:31.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:30.770+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:31.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:30.770+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:31.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:30 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:30.770+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:31.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:30 smithi069 systemd[1]: Stopping Ceph osd.7 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:31.722 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.285+0000 7f4553ea5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:31.722 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.285+0000 7f4553ea5700 -1 osd.7 138 *** Got signal Terminated *** 2023-09-23T05:15:31.722 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.285+0000 7f4553ea5700 -1 osd.7 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:31.722 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.721+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:31.722 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:31.353+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:31.723 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:31.353+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:31.723 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:31.353+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:31.723 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:31.353+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:31.991 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.721+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:31.991 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:31.721+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:32.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:31 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:31.780+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:32.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:31 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:31.780+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:32.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:31 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:31.780+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:32.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:31.670+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:32.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:31.670+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:32.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:31.670+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:32.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:31.875+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:32.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:31.875+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:32.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:31 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:31.875+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:32.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:32.317+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:32.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:32.317+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:32.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:32.317+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:32.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:32.317+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:32.909 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:32.651+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:32.909 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:32.651+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:32.910 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:32.651+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:33.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:32 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:32.731+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:33.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:32 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:32.731+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:33.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:32 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:32.731+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:33.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:32.908+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:33.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:32.908+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:33.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:32 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:32.908+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:33.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:32.767+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:33.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:32.767+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:33.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:32 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:32.767+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:33.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:33.325+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:33.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:33.325+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:33.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:33.325+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:33.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:33.325+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:33.899 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:33.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:33.899 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:33.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:33.900 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:33.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:34.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:33.690+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:34.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:33.690+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:34.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:33 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:33.690+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:34.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:33.898+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:34.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:33.898+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:34.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:33 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:33.898+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:34.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:33.797+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:34.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:33.797+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:34.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:33 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:33.797+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:34.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:34.298+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:34.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:34.298+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:34.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:34.298+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:34.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:34.298+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:34.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:34.623+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:34.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:34.623+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:34.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:34.623+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:34.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:34.852+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:34.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:34.852+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:34.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:34 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:34.852+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:35.154 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:34.680+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:35.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:34.680+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:35.155 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:34 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:34.680+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:35.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:34.779+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:35.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:34.779+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:35.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:34 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:34.779+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:35.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:35.276+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:35.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:35.276+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:35.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:35.276+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:35.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:35.276+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:35.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:35.276+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:35.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:35.641+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:35.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:35.641+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:35.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:35.641+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:35.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:35 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:35.641+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:35.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:35.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:35.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:35.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:35.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:35.612+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:35.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:35.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:35.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:35.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:35.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:35 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:35.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:36.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:35.807+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:44.919362+0000 front 2023-09-23T05:14:44.919461+0000 (oldest deadline 2023-09-23T05:15:09.619448+0000) 2023-09-23T05:15:36.241 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:35.807+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.521667+0000 front 2023-09-23T05:14:53.521707+0000 (oldest deadline 2023-09-23T05:15:22.922253+0000) 2023-09-23T05:15:36.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:35.807+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.922699+0000 front 2023-09-23T05:15:02.922716+0000 (oldest deadline 2023-09-23T05:15:27.022738+0000) 2023-09-23T05:15:36.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7[155552]: 2023-09-23T05:15:35.807+0000 7f454fc9a700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.824267+0000 front 2023-09-23T05:15:12.824290+0000 (oldest deadline 2023-09-23T05:15:35.724620+0000) 2023-09-23T05:15:36.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:36.287+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:36.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:36.287+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:36.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:36.287+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:36.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:36.287+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:36.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:36 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:36.287+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:36.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:36.609+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:36.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:36.609+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:36.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:36.609+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:36.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:36 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:36.609+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:36.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:36.647+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:36.647+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:36.647+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:36.804+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:36.804+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:36.804+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:36.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:36 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:36.804+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:37.242 INFO:journalctl@ceph.osd.7.smithi069.stdout:Sep 23 05:15:37 smithi069 bash[186360]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-7 2023-09-23T05:15:37.628 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:37.296+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:37.628 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:37.296+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:37.629 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:37.296+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:37.629 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:37.296+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:37.629 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:37 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:37.296+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:37.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:37.610+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:37.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:37.610+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:37.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:37.610+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:37.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:37 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:37.610+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:37.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:37.644+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:37.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:37.644+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:37.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:37.644+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:37.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:37.791+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:37.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:37.791+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:37.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:37.792+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:37.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:37 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:37.792+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:38.324 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.7.service' 2023-09-23T05:15:38.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:38.364 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-23T05:15:38.365 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-23T05:15:38.365 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.8 2023-09-23T05:15:38.647 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 systemd[1]: Stopping Ceph osd.8 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:38.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:38 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:38.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:38.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:38 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:38.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:38.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:38 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:38.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:38.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:38 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:38.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:38.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:38 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:38.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:38.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:38.573+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:38.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:38.573+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:38.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:38.573+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:38.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:38 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:38.573+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:38.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.645+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.645+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.645+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.645+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.722+0000 7fd3c298e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.722+0000 7fd3c298e700 -1 osd.8 138 *** Got signal Terminated *** 2023-09-23T05:15:38.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:38.722+0000 7fd3c298e700 -1 osd.8 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:38.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:38.793+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:38.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:38.793+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:38.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:38.793+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:38.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:38 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:38.793+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:39.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:39.284+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:39.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:39.284+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:39.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:39.284+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:39.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:39.284+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:39.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:39.284+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:39.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:39 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:39.580+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:39.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:39 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:39.580+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:39.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:39 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:39.580+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:39.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:39 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:39.580+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:39.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:39.621+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:39.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:39.621+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:39.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:39.621+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:39.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:39.621+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:39.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:39.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:39.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:39.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:39.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:39.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:39.913 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:39 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:39.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:40.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:40.334+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:40.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:40.334+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:40.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:40.334+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:40.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:40.334+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:40.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:40 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:40.334+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:40.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:40.582+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:40.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:40.582+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:40.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:40.582+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:40.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:40 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:40.582+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:40.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:40.631+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:40.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:40.631+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:40.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:40.631+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:40.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:40.631+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:40.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:40.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:40.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:40.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:40.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:40.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:40.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:40 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:40.812+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:41.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:41 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:41.300+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:41.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:41 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:41.300+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:41.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:41 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:41.300+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:41.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:41 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:41.300+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:41.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:41 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:41.300+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:41.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:41 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:41.556+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:41.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:41 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:41.556+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:41.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:41 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:41.556+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:41.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:41 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:41.556+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:41.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:41.638+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:41.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:41.638+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:41.911 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:41.638+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:41.638+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:41.638+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:18.642493+0000 front 2023-09-23T05:15:18.642542+0000 (oldest deadline 2023-09-23T05:15:41.542613+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:41.844+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:41.844+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:41.844+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:41.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:41 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:41.844+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:42.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:42 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:42.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:42.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:42 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:42.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:42.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:42 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:42.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:42.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:42 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:42.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:42.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:42 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:42.320+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:42.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:42 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:42.554+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:42.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:42 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:42.554+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:42.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:42 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:42.554+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:42.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:42 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:42.554+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:43.161 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:42.675+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:43.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:42.675+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:43.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:42.675+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:43.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:42.675+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:43.162 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:42.675+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:18.642493+0000 front 2023-09-23T05:15:18.642542+0000 (oldest deadline 2023-09-23T05:15:41.542613+0000) 2023-09-23T05:15:43.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:42.841+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:43.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:42.841+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:43.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:42.841+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:43.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:42.841+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:43.163 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:42 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:42.841+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:43.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:43.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:43.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:43.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:43.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:43.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:43 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:43.356+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:43.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:43 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:43.534+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:43.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:43 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:43.534+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:43.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:43 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:43.534+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:43.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:43 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:43.534+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:43.911 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:43.857+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:43.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:43.857+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:43.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:43.857+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:43.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:43.857+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:43.912 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:43.857+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:43.913 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:43.652+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:45.337670+0000 front 2023-09-23T05:14:45.337832+0000 (oldest deadline 2023-09-23T05:15:11.738445+0000) 2023-09-23T05:15:43.913 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:43.652+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.039239+0000 front 2023-09-23T05:14:54.039460+0000 (oldest deadline 2023-09-23T05:15:17.539370+0000) 2023-09-23T05:15:43.913 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:43.652+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:02.840575+0000 front 2023-09-23T05:15:02.840725+0000 (oldest deadline 2023-09-23T05:15:27.540522+0000) 2023-09-23T05:15:43.913 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:43.652+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:10.441755+0000 front 2023-09-23T05:15:10.441899+0000 (oldest deadline 2023-09-23T05:15:38.642167+0000) 2023-09-23T05:15:43.914 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8[156696]: 2023-09-23T05:15:43.652+0000 7fd3baf7c700 -1 osd.8 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:18.642493+0000 front 2023-09-23T05:15:18.642542+0000 (oldest deadline 2023-09-23T05:15:41.542613+0000) 2023-09-23T05:15:44.606 INFO:journalctl@ceph.osd.8.smithi118.stdout:Sep 23 05:15:44 smithi118 bash[186767]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-8 2023-09-23T05:15:44.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:44.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:44.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:44.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:44.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:44.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:44 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:44.355+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:44.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:44 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:44.544+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:44.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:44 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:44.544+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:44.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:44 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:44.544+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:44.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:44 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:44.544+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:45.151 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:44.867+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:45.151 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:44.867+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:45.151 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:44.867+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:45.151 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:44.867+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:45.152 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:44 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:44.867+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:45.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:45.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:45.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:45.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:45.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:45.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:45 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:45.365+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:45.867 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.8.service' 2023-09-23T05:15:45.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:45 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:45.550+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:45.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:45 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:45.550+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:45.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:45 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:45.550+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:45.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:45 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:45.550+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:45.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:45.908 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-23T05:15:45.908 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-23T05:15:45.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.9 2023-09-23T05:15:46.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:45 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:45.890+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:46.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:45 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:45.890+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:46.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:45 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:45.890+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:46.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:45 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:45.890+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:46.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:45 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:45.890+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:46.405 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 systemd[1]: Stopping Ceph osd.9 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:46.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:46.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:46.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:46.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:46.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:46.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:46 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:46.383+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:46.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.417+0000 7f11ea4c1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.417+0000 7f11ea4c1700 -1 osd.9 138 *** Got signal Terminated *** 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.417+0000 7f11ea4c1700 -1 osd.9 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.559+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.559+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.559+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:46.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:46 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:46.559+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:47.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:46 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:46.885+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:47.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:46 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:46.885+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:47.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:46 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:46.885+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:47.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:46 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:46.885+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:47.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:46 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:46.885+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:47.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:47.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:47.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:47.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:47.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:47.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:47 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:47.412+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:47.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:47 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:47.542+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:47.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:47 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:47.542+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:47.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:47 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:47.542+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:47.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:47 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:47.542+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:47.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:47 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:47.542+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.461256+0000 front 2023-09-23T05:15:17.461149+0000 (oldest deadline 2023-09-23T05:15:46.861895+0000) 2023-09-23T05:15:48.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:47.918+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:48.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:47.918+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:48.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:47.918+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:48.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:47.918+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:48.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:47.918+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:48.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:48.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:48.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:48.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:48.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:48.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:48 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:48.364+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:48.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:48.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:48.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:48.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:48.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:48.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:48.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:48.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:48.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:48 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:48.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.461256+0000 front 2023-09-23T05:15:17.461149+0000 (oldest deadline 2023-09-23T05:15:46.861895+0000) 2023-09-23T05:15:49.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:48 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:48.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:49.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:48 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:48.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:49.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:48 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:48.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:49.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:48 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:48.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:49.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:48 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:48.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:49.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:49.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:49.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:49.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:49.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:49.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:49 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:49.369+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:49.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:49.547+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:49.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:49.547+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:49.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:49.547+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:49.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:49.547+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:49.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:49 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:49.547+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.461256+0000 front 2023-09-23T05:15:17.461149+0000 (oldest deadline 2023-09-23T05:15:46.861895+0000) 2023-09-23T05:15:50.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:50.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:50.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:50.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:50.414 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:50.414 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:49 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:49.928+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:50.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:50.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:50.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:50.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:50.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:50.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:50.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:50 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:50.358+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:50.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:43.456467+0000 front 2023-09-23T05:14:43.456556+0000 (oldest deadline 2023-09-23T05:15:09.356362+0000) 2023-09-23T05:15:50.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:54.057813+0000 front 2023-09-23T05:14:54.057615+0000 (oldest deadline 2023-09-23T05:15:19.357488+0000) 2023-09-23T05:15:50.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.658789+0000 front 2023-09-23T05:15:04.658756+0000 (oldest deadline 2023-09-23T05:15:30.558961+0000) 2023-09-23T05:15:50.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.860074+0000 front 2023-09-23T05:15:12.860109+0000 (oldest deadline 2023-09-23T05:15:35.160022+0000) 2023-09-23T05:15:50.905 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.461256+0000 front 2023-09-23T05:15:17.461149+0000 (oldest deadline 2023-09-23T05:15:46.861895+0000) 2023-09-23T05:15:50.906 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:50 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9[141040]: 2023-09-23T05:15:50.539+0000 7f11e6ab7700 -1 osd.9 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:27.362901+0000 front 2023-09-23T05:15:27.362981+0000 (oldest deadline 2023-09-23T05:15:50.263009+0000) 2023-09-23T05:15:51.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:51.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:51.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:51.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:51.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:51.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:50 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:50.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:51.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:51.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:51.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:51.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:51.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:51.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:51.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:51 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:51.310+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:52.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:52.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:52.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:52.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:52.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:52.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:51.976+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:52.502 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 23 05:15:52 smithi005 bash[191536]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-9 2023-09-23T05:15:52.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:52.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:52.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:52.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:52.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:52.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:52.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:52 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:52.349+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:53.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:53.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:53.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:53.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:53.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:53.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:53 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:53.023+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:53.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:53.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:53.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:53.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:53.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:53.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:53.743 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:53 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:53.306+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:54.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.9.service' 2023-09-23T05:15:54.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:15:54.342 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-23T05:15:54.342 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-23T05:15:54.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.10 2023-09-23T05:15:54.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:54.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:54.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:54.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:54.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:54.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.038+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:54.698 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:54.699 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:54.699 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:54.699 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:54.699 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:54.699 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:54.700 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:54.700 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 systemd[1]: Stopping Ceph osd.10 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:15:54.991 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.696+0000 7f097dd8d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:15:54.991 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.696+0000 7f097dd8d700 -1 osd.10 138 *** Got signal Terminated *** 2023-09-23T05:15:54.991 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:54 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:54.696+0000 7f097dd8d700 -1 osd.10 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:15:55.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:55.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:55.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:55.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:55.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:55.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:54 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:54.995+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:55.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:55.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.272+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:55.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.273+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:55.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.273+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:55.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.273+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:55.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.273+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:55.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:55 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:55.273+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:56.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:56.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:56.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:56.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:56.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:56.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:55.947+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:56.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:56.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:56 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:56.280+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:57.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:57.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:57.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:57.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:57.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:57.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:56 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:56.942+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:57.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:57.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:57.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:57.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:57.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:57.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:57.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:57 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:57.292+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:58.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:58.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:58.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:58.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:58.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:58.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:57 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:57.927+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:58.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:58.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:58.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:58.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:58.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:58.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:58.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:58 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:58.329+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:15:59.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:15:59.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:15:59.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:15:59.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:15:59.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:15:59.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:58 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:58.888+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:15:59.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:35.518047+0000 front 2023-09-23T05:14:35.518271+0000 (oldest deadline 2023-09-23T05:15:01.418115+0000) 2023-09-23T05:15:59.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:46.719058+0000 front 2023-09-23T05:14:46.719029+0000 (oldest deadline 2023-09-23T05:15:09.619319+0000) 2023-09-23T05:15:59.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:53.620983+0000 front 2023-09-23T05:14:53.620915+0000 (oldest deadline 2023-09-23T05:15:18.821599+0000) 2023-09-23T05:15:59.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-09-23T05:15:04.424185+0000 front 2023-09-23T05:15:04.424292+0000 (oldest deadline 2023-09-23T05:15:30.324292+0000) 2023-09-23T05:15:59.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:11.325628+0000 front 2023-09-23T05:15:11.325601+0000 (oldest deadline 2023-09-23T05:15:34.825724+0000) 2023-09-23T05:15:59.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:17.726986+0000 front 2023-09-23T05:15:17.726842+0000 (oldest deadline 2023-09-23T05:15:42.426661+0000) 2023-09-23T05:15:59.742 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:15:59 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10[163537]: 2023-09-23T05:15:59.283+0000 7f097837f700 -1 osd.10 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:22.427323+0000 front 2023-09-23T05:15:22.427539+0000 (oldest deadline 2023-09-23T05:15:49.427518+0000) 2023-09-23T05:16:00.161 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:00.162 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:15:59 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:15:59.907+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:00.741 INFO:journalctl@ceph.osd.10.smithi069.stdout:Sep 23 05:16:00 smithi069 bash[186633]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-10 2023-09-23T05:16:01.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.915+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:01.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.915+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:01.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.916+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:01.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.916+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:01.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.916+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:01.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.916+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:01.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:00 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:00.916+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:01.954 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.10.service' 2023-09-23T05:16:01.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:01.994 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-23T05:16:01.994 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-23T05:16:01.995 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.11 2023-09-23T05:16:02.370 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:02.370 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:02.370 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:02.370 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:01 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:01.938+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 systemd[1]: Stopping Ceph osd.11 for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.369+0000 7f9aeddff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.369+0000 7f9aeddff700 -1 osd.11 138 *** Got signal Terminated *** 2023-09-23T05:16:02.371 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.369+0000 7f9aeddff700 -1 osd.11 138 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-23T05:16:03.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:03.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:03.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:03.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:03.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:03.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:03.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:02 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:02.939+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:04.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:04.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:04.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:04.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:04.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:04.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:04.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:03 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:03.974+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:05.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:05.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:05.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:05.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:05.413 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:05.414 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:05.414 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:04 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:04.972+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:06.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:06.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.011+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:07.411 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-09-23T05:14:36.718077+0000 front 2023-09-23T05:14:36.717899+0000 (oldest deadline 2023-09-23T05:15:02.017987+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6804 osd.1 since back 2023-09-23T05:14:47.720553+0000 front 2023-09-23T05:14:47.720714+0000 (oldest deadline 2023-09-23T05:15:11.220507+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6804 osd.2 since back 2023-09-23T05:14:56.521825+0000 front 2023-09-23T05:14:56.521795+0000 (oldest deadline 2023-09-23T05:15:20.021920+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6812 osd.4 since back 2023-09-23T05:15:12.125616+0000 front 2023-09-23T05:15:12.125491+0000 (oldest deadline 2023-09-23T05:15:36.225634+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.118:6812 osd.5 since back 2023-09-23T05:15:16.226110+0000 front 2023-09-23T05:15:16.226152+0000 (oldest deadline 2023-09-23T05:15:42.126154+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-09-23T05:15:23.227206+0000 front 2023-09-23T05:15:23.227180+0000 (oldest deadline 2023-09-23T05:15:49.126993+0000) 2023-09-23T05:16:07.412 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:06 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11[164870]: 2023-09-23T05:16:06.964+0000 7f9ae73ef700 -1 osd.11 138 heartbeat_check: no reply from 172.21.15.69:6820 osd.7 since back 2023-09-23T05:15:33.827869+0000 front 2023-09-23T05:15:33.827915+0000 (oldest deadline 2023-09-23T05:15:59.727947+0000) 2023-09-23T05:16:08.136 INFO:journalctl@ceph.osd.11.smithi118.stdout:Sep 23 05:16:08 smithi118 bash[187036]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-osd-11 2023-09-23T05:16:09.396 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@osd.11.service' 2023-09-23T05:16:09.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:09.441 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-23T05:16:09.441 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-23T05:16:09.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.a 2023-09-23T05:16:09.905 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:16:09 smithi005 systemd[1]: Stopping Ceph mds.a for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:10.405 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:16:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: 2023-09-23T05:16:09.954+0000 7f697106b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:10.405 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:16:09 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a[149553]: 2023-09-23T05:16:09.954+0000 7f697106b700 -1 mds.a *** got signal Terminated *** 2023-09-23T05:16:14.655 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 23 05:16:14 smithi005 bash[191805]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-a 2023-09-23T05:16:14.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.a.service' 2023-09-23T05:16:14.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:14.777 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-23T05:16:14.778 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-23T05:16:14.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.d 2023-09-23T05:16:15.155 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 05:16:14 smithi005 systemd[1]: Stopping Ceph mds.d for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:15.655 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 05:16:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-d[148553]: 2023-09-23T05:16:15.286+0000 7f9a67e6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:15.655 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 05:16:15 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-d[148553]: 2023-09-23T05:16:15.286+0000 7f9a67e6f700 -1 mds.d *** got signal Terminated *** 2023-09-23T05:16:17.405 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 23 05:16:17 smithi005 bash[191961]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-d 2023-09-23T05:16:17.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.d.service' 2023-09-23T05:16:17.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:17.719 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-23T05:16:17.719 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-23T05:16:17.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.g 2023-09-23T05:16:17.905 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:16:17 smithi005 systemd[1]: Stopping Ceph mds.g for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:18.655 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:16:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T05:16:18.203+0000 7f8f9e02e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:18.655 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:16:18 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g[149162]: 2023-09-23T05:16:18.203+0000 7f8f9e02e700 -1 mds.g *** got signal Terminated *** 2023-09-23T05:16:22.905 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 23 05:16:22 smithi005 bash[192105]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-g 2023-09-23T05:16:22.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.g.service' 2023-09-23T05:16:23.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:23.059 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-23T05:16:23.059 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-23T05:16:23.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.j 2023-09-23T05:16:23.405 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:23 smithi005 systemd[1]: Stopping Ceph mds.j for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:23.905 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-j[148163]: 2023-09-23T05:16:23.560+0000 7fea43bdc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:23.905 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:23 smithi005 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-j[148163]: 2023-09-23T05:16:23.560+0000 7fea43bdc700 -1 mds.j *** got signal Terminated *** 2023-09-23T05:16:26.226 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:26 smithi005 bash[192251]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-j 2023-09-23T05:16:26.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.j.service' 2023-09-23T05:16:26.523 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:26 smithi005 systemd[1]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.j.service: Succeeded. 2023-09-23T05:16:26.523 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 23 05:16:26 smithi005 systemd[1]: Stopped Ceph mds.j for a18d5e54-59ca-11ee-8db4-212e2dc638e7. 2023-09-23T05:16:26.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:26.546 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-23T05:16:26.546 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-23T05:16:26.546 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.b 2023-09-23T05:16:26.991 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:16:26 smithi069 systemd[1]: Stopping Ceph mds.b for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:27.491 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:16:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: 2023-09-23T05:16:27.124+0000 7f57820fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:27.491 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:16:27 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b[170563]: 2023-09-23T05:16:27.124+0000 7f57820fa700 -1 mds.b *** got signal Terminated *** 2023-09-23T05:16:30.992 INFO:journalctl@ceph.mds.b.smithi069.stdout:Sep 23 05:16:30 smithi069 bash[186903]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-b 2023-09-23T05:16:31.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.b.service' 2023-09-23T05:16:31.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:31.099 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-23T05:16:31.099 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-23T05:16:31.099 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.e 2023-09-23T05:16:31.482 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:16:31 smithi069 systemd[1]: Stopping Ceph mds.e for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:31.741 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:16:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: 2023-09-23T05:16:31.481+0000 7f3dac735700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:31.741 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:16:31 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e[169882]: 2023-09-23T05:16:31.481+0000 7f3dac735700 -1 mds.e *** got signal Terminated *** 2023-09-23T05:16:34.991 INFO:journalctl@ceph.mds.e.smithi069.stdout:Sep 23 05:16:34 smithi069 bash[187045]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-e 2023-09-23T05:16:35.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.e.service' 2023-09-23T05:16:35.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:35.108 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-23T05:16:35.109 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-23T05:16:35.109 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.h 2023-09-23T05:16:35.481 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:16:35 smithi069 systemd[1]: Stopping Ceph mds.h for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:35.741 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:16:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: 2023-09-23T05:16:35.480+0000 7f5793065700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:35.741 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:16:35 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h[170224]: 2023-09-23T05:16:35.480+0000 7f5793065700 -1 mds.h *** got signal Terminated *** 2023-09-23T05:16:38.991 INFO:journalctl@ceph.mds.h.smithi069.stdout:Sep 23 05:16:38 smithi069 bash[187189]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-h 2023-09-23T05:16:39.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.h.service' 2023-09-23T05:16:39.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:39.075 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-23T05:16:39.075 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-23T05:16:39.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.k 2023-09-23T05:16:39.448 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:16:39 smithi069 systemd[1]: Stopping Ceph mds.k for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:39.741 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:16:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: 2023-09-23T05:16:39.446+0000 7f826dcea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:39.741 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:16:39 smithi069 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k[169541]: 2023-09-23T05:16:39.446+0000 7f826dcea700 -1 mds.k *** got signal Terminated *** 2023-09-23T05:16:42.991 INFO:journalctl@ceph.mds.k.smithi069.stdout:Sep 23 05:16:42 smithi069 bash[187333]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-k 2023-09-23T05:16:43.215 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.k.service' 2023-09-23T05:16:43.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:43.249 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-23T05:16:43.250 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-23T05:16:43.250 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.c 2023-09-23T05:16:43.630 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:16:43 smithi118 systemd[1]: Stopping Ceph mds.c for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:43.911 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:16:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: 2023-09-23T05:16:43.628+0000 7f3d81ef4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:43.911 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:16:43 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c[171729]: 2023-09-23T05:16:43.628+0000 7f3d81ef4700 -1 mds.c *** got signal Terminated *** 2023-09-23T05:16:47.146 INFO:journalctl@ceph.mds.c.smithi118.stdout:Sep 23 05:16:46 smithi118 bash[187306]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-c 2023-09-23T05:16:47.152 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.c.service' 2023-09-23T05:16:47.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:47.192 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-23T05:16:47.192 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-23T05:16:47.192 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.f 2023-09-23T05:16:47.412 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:16:47 smithi118 systemd[1]: Stopping Ceph mds.f for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:47.914 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:16:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: 2023-09-23T05:16:47.578+0000 7f0d8dcdc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:47.914 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:16:47 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f[171389]: 2023-09-23T05:16:47.578+0000 7f0d8dcdc700 -1 mds.f *** got signal Terminated *** 2023-09-23T05:16:50.942 INFO:journalctl@ceph.mds.f.smithi118.stdout:Sep 23 05:16:50 smithi118 bash[187453]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-f 2023-09-23T05:16:51.131 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.f.service' 2023-09-23T05:16:51.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:51.165 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-23T05:16:51.165 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-23T05:16:51.165 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.i 2023-09-23T05:16:51.411 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:16:51 smithi118 systemd[1]: Stopping Ceph mds.i for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:51.911 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:16:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: 2023-09-23T05:16:51.553+0000 7fc35040e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:51.912 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:16:51 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i[172068]: 2023-09-23T05:16:51.553+0000 7fc35040e700 -1 mds.i *** got signal Terminated *** 2023-09-23T05:16:55.093 INFO:journalctl@ceph.mds.i.smithi118.stdout:Sep 23 05:16:54 smithi118 bash[187590]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-i 2023-09-23T05:16:55.099 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.i.service' 2023-09-23T05:16:55.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:55.132 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-23T05:16:55.133 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-23T05:16:55.133 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.l 2023-09-23T05:16:55.411 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:16:55 smithi118 systemd[1]: Stopping Ceph mds.l for a18d5e54-59ca-11ee-8db4-212e2dc638e7... 2023-09-23T05:16:55.911 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:16:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: 2023-09-23T05:16:55.501+0000 7f2e95c80700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-23T05:16:55.912 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:16:55 smithi118 ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l[171040]: 2023-09-23T05:16:55.501+0000 7f2e95c80700 -1 mds.l *** got signal Terminated *** 2023-09-23T05:16:58.978 INFO:journalctl@ceph.mds.l.smithi118.stdout:Sep 23 05:16:58 smithi118 bash[187726]: ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7-mds-l 2023-09-23T05:16:59.181 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a18d5e54-59ca-11ee-8db4-212e2dc638e7@mds.l.service' 2023-09-23T05:16:59.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-23T05:16:59.215 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-23T05:16:59.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T05:16:59.635 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:17:03.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T05:17:03.391 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:17:04.996 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force --keep-logs 2023-09-23T05:17:05.379 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:17:07.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:17:07.100 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:17:07.137 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-23T05:17:07.167 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-23T05:17:07.169 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi005/crash 2023-09-23T05:17:07.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T05:17:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T05:17:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2023-09-23T05:17:07.273 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi069/crash 2023-09-23T05:17:07.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T05:17:07.395 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T05:17:07.396 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2023-09-23T05:17:07.397 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi118/crash 2023-09-23T05:17:07.398 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash -- . 2023-09-23T05:17:07.472 INFO:teuthology.orchestra.run.smithi118.stderr:tar: /var/lib/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/crash: Cannot open: No such file or directory 2023-09-23T05:17:07.472 INFO:teuthology.orchestra.run.smithi118.stderr:tar: Error is not recoverable: exiting now 2023-09-23T05:17:07.473 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-23T05:17:07.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-23T05:17:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:grep: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph.log: No such file or directory 2023-09-23T05:17:07.560 INFO:tasks.cephadm:Compressing logs... 2023-09-23T05:17:07.560 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T05:17:07.564 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T05:17:07.567 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T05:17:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T05:17:07.609 INFO:teuthology.orchestra.run.smithi069.stderr:: No such file or directory 2023-09-23T05:17:07.612 INFO:teuthology.orchestra.run.smithi118.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T05:17:07.612 INFO:teuthology.orchestra.run.smithi118.stderr:: No such file or directory 2023-09-23T05:17:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.10.log 2023-09-23T05:17:07.614 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.h.log 2023-09-23T05:17:07.615 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.e.log 2023-09-23T05:17:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.4.log 2023-09-23T05:17:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.1.log 2023-09-23T05:17:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.k.log 2023-09-23T05:17:07.616 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T05:17:07.617 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.7.log 2023-09-23T05:17:07.617 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.b.log 2023-09-23T05:17:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.b.log 2023-09-23T05:17:07.618 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.y.log 2023-09-23T05:17:07.620 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.5.log 2023-09-23T05:17:07.620 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.2.log 2023-09-23T05:17:07.621 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.11.log 2023-09-23T05:17:07.621 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.5.log: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.z.log 2023-09-23T05:17:07.621 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.8.log 2023-09-23T05:17:07.621 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.f.log 2023-09-23T05:17:07.623 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.c.log 2023-09-23T05:17:07.623 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.i.log 2023-09-23T05:17:07.623 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.c.log 2023-09-23T05:17:07.623 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.c.log: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T05:17:07.623 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.l.log 2023-09-23T05:17:07.625 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.y.log: 92.6% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.y.log.gz 2023-09-23T05:17:07.628 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.l.log: 92.8% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.z.log.gz 2023-09-23T05:17:07.628 INFO:teuthology.orchestra.run.smithi118.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T05:17:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-23T05:17:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2023-09-23T05:17:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-09-23T05:17:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.0.log 2023-09-23T05:17:07.640 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.g.log 2023-09-23T05:17:07.642 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.a.log 2023-09-23T05:17:07.642 INFO:teuthology.orchestra.run.smithi005.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-09-23T05:17:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.6.log 2023-09-23T05:17:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.3.log 2023-09-23T05:17:07.645 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.d.log 2023-09-23T05:17:07.646 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.x.log 2023-09-23T05:17:07.647 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.9.log 2023-09-23T05:17:07.648 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log 2023-09-23T05:17:07.649 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.j.log 2023-09-23T05:17:07.650 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.a.log 2023-09-23T05:17:07.653 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.j.log: /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.a.log: 94.4% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.d.log.gz 2023-09-23T05:17:07.658 INFO:teuthology.orchestra.run.smithi005.stderr: 94.5% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.j.log.gz 2023-09-23T05:17:07.768 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T05:17:08.179 INFO:teuthology.orchestra.run.smithi118.stderr: 91.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T05:17:08.197 INFO:teuthology.orchestra.run.smithi069.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T05:17:08.288 INFO:teuthology.orchestra.run.smithi069.stderr: 91.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-volume.log.gz 2023-09-23T05:17:09.339 INFO:teuthology.orchestra.run.smithi005.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-23T05:17:09.385 INFO:teuthology.orchestra.run.smithi005.stderr: 90.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mgr.x.log.gz 2023-09-23T05:17:10.532 INFO:teuthology.orchestra.run.smithi069.stderr: 92.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.b.log.gz 2023-09-23T05:17:13.430 INFO:teuthology.orchestra.run.smithi118.stderr: 91.9% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.c.log.gz 2023-09-23T05:17:15.116 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mon.a.log.gz 2023-09-23T05:17:18.816 INFO:teuthology.orchestra.run.smithi069.stderr: 95.4% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.k.log.gz 2023-09-23T05:17:18.822 INFO:teuthology.orchestra.run.smithi005.stderr: 94.7% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.a.log.gz 2023-09-23T05:17:25.857 INFO:teuthology.orchestra.run.smithi069.stderr: 95.3% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.e.log.gz 2023-09-23T05:17:27.250 INFO:teuthology.orchestra.run.smithi069.stderr: 95.4% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.h.log.gz 2023-09-23T05:17:49.951 INFO:teuthology.orchestra.run.smithi118.stderr: 95.2% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.l.log.gz 2023-09-23T05:18:13.482 INFO:teuthology.orchestra.run.smithi069.stderr: 91.4% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.1.log.gz 2023-09-23T05:18:17.938 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.10.log.gz 2023-09-23T05:18:30.882 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.7.log.gz 2023-09-23T05:18:42.728 INFO:teuthology.orchestra.run.smithi069.stderr: 91.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.4.log.gz 2023-09-23T05:18:49.321 INFO:teuthology.orchestra.run.smithi005.stderr: 91.4% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.0.log.gz 2023-09-23T05:18:57.766 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.6.log.gz 2023-09-23T05:19:02.221 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.3.log.gz 2023-09-23T05:19:11.556 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.9.log.gz 2023-09-23T05:19:37.482 INFO:teuthology.orchestra.run.smithi069.stderr: 93.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.b.log.gz 2023-09-23T05:19:37.485 INFO:teuthology.orchestra.run.smithi069.stderr: 2023-09-23T05:19:37.485 INFO:teuthology.orchestra.run.smithi069.stderr:real 2m29.901s 2023-09-23T05:19:37.485 INFO:teuthology.orchestra.run.smithi069.stderr:user 4m56.931s 2023-09-23T05:19:37.485 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m10.731s 2023-09-23T05:20:06.494 INFO:teuthology.orchestra.run.smithi118.stderr: 91.2% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.11.log.gz 2023-09-23T05:20:29.260 INFO:teuthology.orchestra.run.smithi118.stderr: 90.8% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.8.log.gz 2023-09-23T05:20:47.976 INFO:teuthology.orchestra.run.smithi118.stderr: 91.0% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.2.log.gz 2023-09-23T05:21:27.864 INFO:teuthology.orchestra.run.smithi118.stderr: 90.8% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-osd.5.log.gz 2023-09-23T05:21:53.575 INFO:teuthology.orchestra.run.smithi005.stderr: 94.5% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.g.log.gz 2023-09-23T05:21:53.579 INFO:teuthology.orchestra.run.smithi005.stderr: 2023-09-23T05:21:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:real 4m45.978s 2023-09-23T05:21:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:user 6m34.111s 2023-09-23T05:21:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m45.711s 2023-09-23T05:24:26.151 INFO:teuthology.orchestra.run.smithi118.stderr: 93.8% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.i.log.gz 2023-09-23T05:24:48.680 INFO:teuthology.orchestra.run.smithi118.stderr: 94.9% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.c.log.gz 2023-09-23T05:25:10.406 INFO:teuthology.orchestra.run.smithi118.stderr: 96.1% -- replaced with /var/log/ceph/a18d5e54-59ca-11ee-8db4-212e2dc638e7/ceph-mds.f.log.gz 2023-09-23T05:25:10.414 INFO:teuthology.orchestra.run.smithi118.stderr: 2023-09-23T05:25:10.414 INFO:teuthology.orchestra.run.smithi118.stderr:real 8m2.829s 2023-09-23T05:25:10.414 INFO:teuthology.orchestra.run.smithi118.stderr:user 9m37.239s 2023-09-23T05:25:10.414 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m25.222s 2023-09-23T05:25:10.415 INFO:tasks.cephadm:Archiving logs... 2023-09-23T05:25:10.416 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi005/log 2023-09-23T05:25:10.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T05:26:06.469 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi069/log 2023-09-23T05:26:06.484 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T05:26:53.406 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi118/log 2023-09-23T05:26:53.407 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T05:28:13.643 INFO:tasks.cephadm:Removing cluster... 2023-09-23T05:28:13.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force 2023-09-23T05:28:14.207 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:28:16.176 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force 2023-09-23T05:28:16.794 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:28:17.666 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a18d5e54-59ca-11ee-8db4-212e2dc638e7 --force 2023-09-23T05:28:18.371 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: a18d5e54-59ca-11ee-8db4-212e2dc638e7 2023-09-23T05:28:19.418 INFO:tasks.cephadm:Removing cephadm ... 2023-09-23T05:28:19.418 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T05:28:19.466 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T05:28:19.499 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-23T05:28:19.533 INFO:tasks.cephadm:Teardown complete 2023-09-23T05:28:19.534 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-23T05:28:19.546 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-23T05:28:19.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T05:28:19.550 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T05:28:19.553 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T05:28:19.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-23T05:28:19.697 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-23T05:28:19.697 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T05:28:19.702 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T05:28:19.704 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T05:28:19.720 INFO:teuthology.orchestra.run.smithi118.stderr:bash: ntpq: command not found 2023-09-23T05:28:19.720 INFO:teuthology.orchestra.run.smithi069.stderr:bash: ntpq: command not found 2023-09-23T05:28:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 -709us[ -709us] +/- 91ms 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -1562us[-1560us] +/- 74ms 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +2150us[+2153us] +/- 71ms 2023-09-23T05:28:19.742 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 -713us[ -706us] +/- 91ms 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1585us[-1579us] +/- 74ms 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +2114us[+2121us] +/- 71ms 2023-09-23T05:28:19.751 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T05:28:19.761 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T05:28:19.762 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-09-23T05:28:19.762 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 -662us[ -662us] +/- 91ms 2023-09-23T05:28:19.762 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1533us[-1537us] +/- 74ms 2023-09-23T05:28:19.762 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +2172us[+2172us] +/- 71ms 2023-09-23T05:28:19.762 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T05:28:19.763 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-23T05:28:19.773 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-23T05:28:19.774 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-23T05:28:19.814 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T05:28:20.075 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T05:28:20.213 DEBUG:teuthology.orchestra.run.smithi118:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T05:28:20.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T05:28:20.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:20.516 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T05:28:20.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:20.552 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T05:28:20.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:20.585 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-23T05:28:20.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-23T05:28:20.661 INFO:teuthology.task.internal:Duration was 4078.042159 seconds 2023-09-23T05:28:20.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-23T05:28:20.708 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-23T05:28:20.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T05:28:20.713 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T05:28:20.716 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T05:28:20.794 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T05:28:20.806 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T05:28:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T05:28:21.210 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-23T05:28:21.210 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2023-09-23T05:28:21.210 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T05:28:21.276 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2023-09-23T05:28:21.276 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T05:28:21.319 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2023-09-23T05:28:21.320 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T05:28:21.348 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-23T05:28:21.348 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T05:28:21.353 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T05:28:21.362 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T05:28:21.458 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-23T05:28:21.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T05:28:21.463 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T05:28:21.466 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T05:28:21.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-23T05:28:21.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-23T05:28:21.653 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T05:28:21.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T05:28:21.755 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T05:28:21.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-23T05:28:21.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:21.800 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:21.802 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:21.840 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2023-09-23T05:28:21.861 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2023-09-23T05:28:21.892 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2023-09-23T05:28:21.929 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:21.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:21.971 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:21.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:21.987 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T05:28:22.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T05:28:22.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-23T05:28:22.015 INFO:teuthology.task.internal:Transferring archived files... 2023-09-23T05:28:22.017 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi005 2023-09-23T05:28:22.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T05:28:22.101 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi069 2023-09-23T05:28:22.101 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T05:28:22.139 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/remote/smithi118 2023-09-23T05:28:22.140 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T05:28:22.179 INFO:teuthology.task.internal:Removing archive directory... 2023-09-23T05:28:22.179 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T05:28:22.184 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T05:28:22.186 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T05:28:22.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-23T05:28:22.241 INFO:teuthology.task.internal:Not uploading archives. 2023-09-23T05:28:22.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-23T05:28:22.274 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-23T05:28:22.274 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T05:28:22.278 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T05:28:22.281 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T05:28:22.294 INFO:teuthology.orchestra.run.smithi069.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 05:28 /home/ubuntu/cephtest 2023-09-23T05:28:22.297 INFO:teuthology.orchestra.run.smithi118.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 05:28 /home/ubuntu/cephtest 2023-09-23T05:28:22.317 INFO:teuthology.orchestra.run.smithi005.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 05:28 /home/ubuntu/cephtest 2023-09-23T05:28:22.320 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-23T05:28:22.330 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-23T05:28:22.425 INFO:teuthology.nuke:Checking targets against current locks 2023-09-23T05:28:22.451 DEBUG:teuthology.nuke:shortname: smithi005 2023-09-23T05:28:22.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T05:28:22.475 DEBUG:teuthology.nuke:shortname: smithi069 2023-09-23T05:28:22.475 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T05:28:22.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.825070', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T05:28:22.498 DEBUG:teuthology.nuke:shortname: smithi118 2023-09-23T05:28:22.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T05:28:22.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.822168', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T05:28:22.521 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 04:11:20.823808', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-23T05:28:22.530 INFO:teuthology.orchestra.console.smithi005:Power off 2023-09-23T05:28:22.531 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T05:28:22.557 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2023-09-23T05:28:22.557 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:22.573 INFO:teuthology.orchestra.console.smithi069:Power off 2023-09-23T05:28:22.574 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T05:28:22.588 INFO:teuthology.orchestra.console.smithi118:Power off 2023-09-23T05:28:22.588 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T05:28:22.596 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2023-09-23T05:28:22.596 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2023-09-23T05:28:22.597 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:22.609 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-09-23T05:28:22.609 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:22.620 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-23T05:28:22.630 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-23T05:28:26.597 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:26.621 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:26.631 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:26.706 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2023-09-23T05:28:26.731 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-23T05:28:26.741 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-23T05:28:30.710 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:30.732 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:30.741 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:30.819 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2023-09-23T05:28:30.841 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-09-23T05:28:30.850 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-23T05:28:30.921 INFO:teuthology.orchestra.console.smithi005:Power off completed 2023-09-23T05:28:30.951 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-09-23T05:28:34.846 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T05:28:34.957 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-09-23T05:28:35.057 INFO:teuthology.orchestra.console.smithi069:Power off completed 2023-09-23T05:28:35.164 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 4078.042158603668 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=be869081a06548ce91c468e53c6dbbe4 status: fail success: false 2023-09-23T05:28:35.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T05:28:35.309 INFO:teuthology.run:FAIL